No change yet, just posting a rebase to keep cfbot happy.
Hi, Thomas
I think that the proposed feature is pretty cool not only because it fixes some old issues with lseek() performance and reliability, but also because it opens the door to some new features, such as disk size quota management [1]. Cheap up-to-date size tracking is one of the major problems for it.
I've only read the 1st patch so far. Here are some thoughts about it:
- SMgrSharedRelation - what about calling it SMgrShmemRelation. It looks weird too, but "...SharedRelation" makes me think of shared catalog tables.
- We can exclude temp relations from consideration as they are never shared and use RelFileNode instead of RelFileNodeBackend in SMgrSharedRelationMapping.
Not only it will save us some space, but also it will help to avoid a situation when temp relations occupy whole cache (which may easily happen in some workloads).
I assume you were trying to make a generic solution, but in practice, temp rels differ from regular ones and may require different optimizations. Local caching will be enough for them if ever needed, for example, we can leave smgr_cached_nblocks[] for temp relations.
> int smgr_shared_relations = 1000;
- How bad would it be to keep cache for all relations? Let's test memory overhead on some realistic datasets. I suppose this 1000 default was chosen just for WIP patch.
I wonder if we can use DSM instead of regular shared memory?
- I'd expect that CREATE DATABASE and ALTER DATABASE SET TABLESPACE require special treatment, because they bypass smgr, just like dropdb. Don't see it in the patch.
[1]
https://github.com/greenplum-db/diskquota--
Best regards,
Lubennikova Anastasia