Re: Avoiding repeated snapshot computation
От | Pavan Deolasee |
---|---|
Тема | Re: Avoiding repeated snapshot computation |
Дата | |
Msg-id | CABOikdP99LD1BM_6EKJV=dUGEDAmb0J5hVSeWcT1TUFSebZz8w@mail.gmail.com обсуждение исходный текст |
Ответ на | Re: Avoiding repeated snapshot computation (Tom Lane <tgl@sss.pgh.pa.us>) |
Ответы |
Re: Avoiding repeated snapshot computation
(Bruce Momjian <bruce@momjian.us>)
Re: Avoiding repeated snapshot computation (Ants Aasma <ants.aasma@eesti.ee>) |
Список | pgsql-hackers |
On Sun, Nov 27, 2011 at 12:26 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Pavan Deolasee <pavan.deolasee@gmail.com> writes: >> On Sat, Nov 26, 2011 at 10:43 PM, Robert Haas <robertmhaas@gmail.com> wrote: >>> Furthermore, it's >>> hard to understand how this could be a net improvement in the general >>> case, because now both B and F are copying everything twice (once to >>> the shared area and one to backend-local memory) instead of just once >>> (to backend-local memory) and C and D are sleeping (yikes!). > >> Yes, B and F pay a price of double copy. But I think it can be a net >> saving because C and D (and many more hopefully) don't need to >> recompute the snapshot again by going over a potentially large >> ProcArray. > > Like Robert, I'm finding it hard to believe that this isn't going to be > a net pessimization in as many or more cases as it'd be a win. Also, > didn't we just get rid of most of the issue of "going over a large > ProcArray" with the move of hot members to a separate array? > Yeah, separating the PGPROC members has helped a lot. But that does not reduce the number of GetSnapshotData calls. It only makes each call less expensive. As I said, I had seen 15-20% improvement with not even a slightly tuned patch, so I am optimistic about the potential of the approach. > In the end, getting a snapshot is exactly about copying information out > of shared memory. Perhaps it would be more productive to think about > how we could further modify the ProcArray representation to reduce the > impedance mismatch between it and the snapshot representation, rather > than introducing a second shared-memory copy of the same information. > I think that a good idea. We need a representation that needs minimum processing to derive the snapshot. One idea is to maintain a bitmap of currently running transactions in the shared memory. The size of the bitmap has to be significantly larger than the ProcArray itself because there could be gaps. But even then since its just a bit per XID, we can have a bitmap to represent a window of say 16K or 32K consecutive XIDs. We can also have a summary bitmap to quickly find if there is at least one running transaction in any given chunk. If there are long running transactions which don't fit in the bitmap, we can track them separately in an array. Snapshot is then just a copy of this bitmap along with some additional information. Will try to think more about it and see if I can test this idea. But any comments welcome. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com
В списке pgsql-hackers по дате отправления: