Обсуждение: Advice wanted on backend memory management

I want to change hashjoin's use of a fixed-size overflow area for tuples
that don't fit into the hashbucket they ought to go in.  Since it's
always possible for an improbably large number of tuples to hash into the
same hashbucket, the overflow area itself can overflow; without the
ability to recover from that, hashjoin is inherently unreliable.
So I think this is an important thing to fix.

To do this, I need to be able to allocate chunks of space that I will
later want to give back all at once (at the end of a hash pass).
Seems to me like a job for palloc and a special memory context ---
but I see no way in mcxt.h to create a new memory context.  How do
I do that?  Also, I'd want the new context to be a "sub-context" of
the regular execution context, in the sense that it should automatically
get released if we exit via elog(ERROR).  What are the appropriate
calls to be using for this?  If there's documentation about this stuff,
I haven't found it :-(
        regards, tom lane

Tom Lane wrote:
> 
> I want to change hashjoin's use of a fixed-size overflow area for tuples
> that don't fit into the hashbucket they ought to go in.  Since it's
> always possible for an improbably large number of tuples to hash into the
> same hashbucket, the overflow area itself can overflow; without the
> ability to recover from that, hashjoin is inherently unreliable.
> So I think this is an important thing to fix.
> 
> To do this, I need to be able to allocate chunks of space that I will
> later want to give back all at once (at the end of a hash pass).
> Seems to me like a job for palloc and a special memory context ---
> but I see no way in mcxt.h to create a new memory context.  How do
> I do that?  Also, I'd want the new context to be a "sub-context" of

No way :(
StartPortalAllocMode could help but - portalmem.c:
/** StartPortalAllocMode *      Starts a new block of portal heap allocation using mode and limit;*      the current
blockis disabled until EndPortalAllocMode is called.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I'm unhappy with this allocation block stacking for quite long time :(

Try to pfree chunks "by hand".

Vadim

Vadim Mikheev <vadim@krs.ru> writes:
> Try to pfree chunks "by hand".

Yeah, that's what I'm trying to avoid.  That would basically mean
duplicating the logic that's in aset.c, which is pretty silly...

After some more looking around, it looks like I could create a
"portal" as is done in vacuum or spi.  But portals seem to have
a heckuva lot of features that I don't understand the uses for.
Anyone have any comments or documentation about them?
        regards, tom lane