Marking shared buffer lookup table as HASH_FIXED_SIZE
От | Ashutosh Bapat |
---|---|
Тема | Marking shared buffer lookup table as HASH_FIXED_SIZE |
Дата | |
Msg-id | CAExHW5v0jh3F_wj86yC=qBfWk0uiT94qy=Z41uzAHLHh0SerRA@mail.gmail.com обсуждение исходный текст |
Список | pgsql-hackers |
Hi All, StrategyInitialize() calls InitBufTable() passing the maximum number of possible entries in the shared buffer lookup table. The table can not have more entries than the number of available shared buffers + number of partitions as explained in the comment in StrategyInitialize(). If there are more than those entries something is wrong with the hash table like mapping two different pages to the same buffer. InitBufTable() invokes ShmemInitHash() with the same init and max size. This means that the maximum number of entries are allocated right from the beginning. So there should not be any need to allocate more entries after initial setup. Thus the hash table qualifies to be marked as HASH_FIXED_SIZE, so that we don't end up with a corrupted hash table in case of a bug. Here's a patch to do so. We are discussing resizing shared buffers without a restart in [1]. When testing those patches, I noticed that the buffer lookup table was getting enlarged even though the underlying memory segment didn't have enough memory causing other errors. Marking the shared buffer lookup table as fixed size will be good to detect such errors earlier. As a side note, we will have to somehow tweak the HASH_FIXED_SIZE flag if we have to resize the shared buffer table while resizing the shared buffers table. But that's the problem for that patch set, not this. I looked at other callers of ShmemInitHash(). Shared tables in lock.c and wait_event.c use different initial and maximum table sizes. So I think they can not be marked fixed sized. Those in predicate.c are already marked HASH_FIXED_SIZE. pg_stat_statement.c has a shared hash table with the same initial and maximum size. It can be marked as HASH_FIXED_SIZE, but it doesn't seem to be worth the trouble in case it requires resizing in corner cases. Shmem index sets maximum and initial size as the same, but the comment there says that it's a soft limit. So didn't add HASH_FIXED_SIZE to that hash table. So it's only the shared buffer lookup table, that is worth marking as fixed sized. I ran pgbench, scale 100, with shared_buffers = 128kB and 128MB resp. for 10 minutes. I saw "pgbench: error: client 7 script 0 aborted in command 5 query 0: ERROR: no unpinned buffers available" in case of 128kB. I think that is expected with such a small buffer pool and large scale. No other error, crash or Assert failure was seen. The whole buffer cache was used all the time in both the cases. The change looks safe. It will be good to get this committed early to detect any lingering bugs which cause the hash table to grow beyond the initial size. [1] https://www.postgresql.org/message-id/my4hukmejato53ef465ev7lk3sqiqvneh7436rz64wmtc7rbfj%40hmuxsf2ngov2 -- Best Wishes, Ashutosh Bapat
Вложения
В списке pgsql-hackers по дате отправления: