Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions
От | Tomas Vondra |
---|---|
Тема | Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions |
Дата | |
Msg-id | 20200329005905.ooxq25qyjwn2mvxz@development обсуждение исходный текст |
Ответ на | Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions (Amit Kapila <amit.kapila16@gmail.com>) |
Ответы |
Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions
|
Список | pgsql-hackers |
On Sat, Mar 28, 2020 at 03:29:34PM +0530, Amit Kapila wrote: >On Sat, Mar 28, 2020 at 2:19 PM Dilip Kumar <dilipbalaut@gmail.com> wrote: >> >> On Sat, Mar 28, 2020 at 11:56 AM Amit Kapila <amit.kapila16@gmail.com> wrote: >> > >> > >> > I have looked at the solution proposed and I would like to share my >> > findings. I think calling ProcArrayApplyXidAssignment for each >> > subtransaction is not a good idea for a couple of reasons: >> > (a) It will just beat the purpose of maintaining KnowAssignedXids >> > array which is to avoid looking at pg_subtrans in >> > TransactionIdIsInProgress() on standby. Basically, if we remove it >> > for each subXid, it will consider the KnowAssignedXids to be >> > overflowed and check pg_subtrans frequently. >> >> Right, I also think this is a problem with this solution. I think we >> may try to avoid this by caching this information. But, then we will >> have to maintain this in some dimensional array which stores >> sub-transaction ids per top transaction or we can maintain a list of >> sub-transaction for each transaction. I haven't thought about how >> much complexity this solution will add. >> > >How about if instead of writing an XLOG_XACT_ASSIGNMENT WAL, we set a >flag in TransactionStateData and then log that as special information >whenever we write next WAL record for a new subtransaction? Then >during recovery, we can only call ProcArrayApplyXidAssignment when we >find that special flag is set in a WAL record. One idea could be to >use a flag bit in XLogRecord.xl_info. If that is feasible then the >solution can work as it is now, without any overhead or change in the >way we maintain KnownAssignedXids. > Ummm, how is that different from what the patch is doing now? I mean, we only write the top-level XID for the first WAL record in each subxact, right? Or what would be the difference with your approach? Anyway, I think you're right the ProcArrayApplyXidAssignment call was done too early, but I think that can be fixed by moving it until after the RecordKnownAssignedTransactionIds call, no? Essentially, right before rm_redo(). You're right calling ProcArrayApplyXidAssignment() may be an issue, because it exclusively acquires the ProcArrayLock. I've actually hinted that might be an issue in my original message, suggesting we might add a local cache of assigned XIDs (a small static array, doing essentially the same thing we used to do on the upstream node). I haven't done that in my WIP patch to keep it simple, but AFACS it'd work. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
В списке pgsql-hackers по дате отправления: