Re: SLRUs in the main buffer pool - Page Header definitions
От | Stephen Frost |
---|---|
Тема | Re: SLRUs in the main buffer pool - Page Header definitions |
Дата | |
Msg-id | ZPsaEGRvllitxB3v@tamriel.snowman.net обсуждение исходный текст |
Ответ на | Re: SLRUs in the main buffer pool - Page Header definitions (Robert Haas <robertmhaas@gmail.com>) |
Ответы |
Re: SLRUs in the main buffer pool - Page Header definitions
(Robert Haas <robertmhaas@gmail.com>)
|
Список | pgsql-hackers |
Greetings, * Robert Haas (robertmhaas@gmail.com) wrote: > On Thu, Aug 24, 2023 at 3:28 PM Stephen Frost <sfrost@snowman.net> wrote: > > Agreed that we'd certainly want to make sure it's all correct and to do > > performance testing but in terms of how many buffers... isn't much of > > the point of this that we have data in memory because it's being used > > and if it's not then we evict it? That is, I wouldn't think we'd have > > set parts of the buffer pool for SLRUs vs. regular data but would let > > the actual demand drive what pages are in the cache and I thought that > > was part of this proposal and part of the reasoning behind making this > > change. > > I think that it's not quite that simple. In the regular buffer pool, > access to pages is controlled by buffer pins and buffer content locks, > but these mechanisms don't exist in the same way in the SLRU code. But > buffer pins drive usage counts which drive eviction decisions. So if > you move SLRU data into the main buffer pool, you either need to keep > the current locking regime and use some new logic to decide how much > of shared_buffers to bequeath to the SLRU pools, OR you need to make > SLRU access use buffer pins and buffer content locks. If you do the > latter, I think you substantially increase the cost of an uncontended > SLRU buffer access, because you now need to pin the buffer, and and > then take and release the content lock, and then release the pin; > whereas today you can do all that by just taking and release the > SLRU's lwlock. That's more atomic operations, and hence more costly, I > think. But even if not, it could perform terribly if SLRU buffers > become more vulnerable to eviction than they are at present, or > alternatively if they take over too much of the buffer pool and force > other important data out. An SLRU buffer access does also update the cur_lru_count for the SLRU, along with the per-page page_lru_count, but those are 32bit and we don't enforce that they're done in order, so presumably those are less expensive than the pinning and usage count updates. This thread started with the issue that our current SLRUs are relatively small though and simply increasing their size would lead to issues as we're just doing simple things like a linear search through them all at times, or at least that's more-or-less what I understood from [1]. More details on the specific 'scaling and sizing challenges' would be nice to have. The current patches were at least claimed to improve performance while also using ReadBuffer_common [2]. Having an idea as to what is specifically leading to that would be interesting though with all these changes likely non-trivial. pgbench may not be the best way to measure this, but it's still interesting to see an improvement like that. Certainly one concern about using the regular buffer pool is that finding a victim page can be expensive and having that as part of an SLRU access could be pretty painful. Though we also have that problem elsewhere too. If we're going to effectively segregate the buffer pool into SLRU parts vs. everything else and then use the existing strategies for SLRUs and have that be different from what everything else is using ... then that doesn't seem like it's really changing things. What would be the point of moving the SLRUs into the main buffer pool then? > SLRUs are a performance hotspot, so even relatively minor changes to > their performance characteristics can, I believe, have pretty > noticeable effects on performance overall. Agreed, we certainly need to have a plan for how to performance test this change and should try to come up with some 'worst case' tests. Thanks, Stephen [1]: https://postgr.es/m/EFAAC0BE-27E9-4186-B925-79B7C696D5AC%40amazon.com [2]: https://postgr.es/m/A09EAE0D-0D3F-4A34-ADE9-8AC1DCBE7D57%40amazon.com
Вложения
В списке pgsql-hackers по дате отправления:
Следующее
От: "Hayato Kuroda (Fujitsu)"Дата:
Сообщение: RE: [PoC] pg_upgrade: allow to upgrade publisher node