Обсуждение: Is a clearer memory lifespan for outerTuple and innerTuple useful?
Hi, When I am working on "shared detoast value"[0], where I want to avoid detoast the same datum over and over again, I have to decide which memory context should be used to hold the detoast value. later I found I have to use different MemoryContexts for the OuterTuple and innerTuple since OuterTuple usually have a longer lifespan. I found the following code in nodeMergeJoin.c which has pretty similar situation, just that it uses ExprContext rather than MemoryContext. MergeJoinState * ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) /* * we need two additional econtexts in which we can compute the join * expressions from the left and right input tuples. The node's regular * econtext won't do because it gets reset too often. */ mergestate->mj_OuterEContext = CreateExprContext(estate); mergestate->mj_InnerEContext = CreateExprContext(estate); IIUC, we needs a MemoryContext rather than ExprContext in fact. In the attachment, I just use two MemoryContext instead of the two ExprContexts which should be less memory and more precise semantics, and works fine. shall we go in this direction? I attached the 2 MemoryContext in JoinState rather than MergeJoinState, which is for the "shared detoast value"[0] more or less. [0] https://www.postgresql.org/message-id/87ttoyihgm.fsf@163.com -- Best Regards Andy Fan
Вложения
Andy Fan <zhihuifan1213@163.com> writes: > ..., I attached the 2 MemoryContext in > JoinState rather than MergeJoinState, which is for the "shared detoast > value"[0] more or less. > After thinking more, if it is designed for "shared detoast value" patch (happens on ExecInterpExpr stage), the inner_tuple_memory and outer_tuple_memory should be attached to ExprContext rather than JoinState since it is more natual to access ExprConext (compared with JoinState) in ExecInterpExpr. I didn't attach a new version for this, any feedback will be appreciated. -- Best Regards Andy Fan
Andy Fan <zhihuifan1213@163.com> writes: > Andy Fan <zhihuifan1213@163.com> writes: > >> ..., I attached the 2 MemoryContext in >> JoinState rather than MergeJoinState, which is for the "shared detoast >> value"[0] more or less. >> In order to delimit the scope of this discussion, I attached the 2 MemoryContext to MergeJoinState. Since the code was writen by Tom at 2005, so add Tom to the cc-list. -- Best Regards Andy Fan
Вложения
Hi!
Maybe, the alternative way is using a separate kind of context, say name it
'ToastContext' for all custom data related to Toasted values? What do you think?
On Sun, Dec 17, 2023 at 4:52 PM Andy Fan <zhihuifan1213@163.com> wrote:
Andy Fan <zhihuifan1213@163.com> writes:
> Andy Fan <zhihuifan1213@163.com> writes:
>
>> ..., I attached the 2 MemoryContext in
>> JoinState rather than MergeJoinState, which is for the "shared detoast
>> value"[0] more or less.
>>
In order to delimit the scope of this discussion, I attached the 2
MemoryContext to MergeJoinState. Since the code was writen by Tom at
2005, so add Tom to the cc-list.
--
Best Regards
Andy Fan
Nikita Malakhov <hukutoc@gmail.com> writes: > Hi! > > Maybe, the alternative way is using a separate kind of context, say name it > 'ToastContext' for all custom data related to Toasted values? What do > you think? That should be a candidate. The latest research makes me think the 'detoast_values' should have the same life cycles as tts_values, so the memory should be managed by TupleTuleSlot (rather than ExprContext) and be handled in ExecCopySlot / ExecClearSlot stuff. In TupleTableSlot we already have a tts_mctx MemoryContext, reusing it needs using 'pfree' to free the detoast values and but a dedicated memory context pays more costs on the setup, but a more efficient MemoryContextReset. > > On Sun, Dec 17, 2023 at 4:52 PM Andy Fan <zhihuifan1213@163.com> wrote: > > Andy Fan <zhihuifan1213@163.com> writes: > > > Andy Fan <zhihuifan1213@163.com> writes: > > > >> ..., I attached the 2 MemoryContext in > >> JoinState rather than MergeJoinState, which is for the "shared detoast > >> value"[0] more or less. > >> > > In order to delimit the scope of this discussion, I attached the 2 > MemoryContext to MergeJoinState. Since the code was writen by Tom at > 2005, so add Tom to the cc-list. > However this patch can be discussed seperately. -- Best Regards Andy Fan