Re: Safe memory allocation functions

Andres Freund wrote:
> On 2015-01-16 12:09:25 -0300, Alvaro Herrera wrote:
> > Robert Haas wrote:
> > > On Thu, Jan 15, 2015 at 10:57 AM, Alvaro Herrera
> > > <alvherre@2ndquadrant.com> wrote:
> > 
> > > >> I do think that "safe" is the wrong suffix.  Maybe palloc_soft_fail()
> > > >> or palloc_null() or palloc_no_oom() or palloc_unsafe().
> > > >
> > > > I liked palloc_noerror() better myself FWIW.
> > > 
> > > I don't care for noerror() because it probably still will error in
> > > some circumstances; just not for OOM.
> > 
> > Yes, but that seems fine to me.  We have other functions with "noerror"
> > flags, and they can still fail under some circumstances -- just not if
> > the error is the most commonly considered scenario in which they fail.
> 
> We rely on palloc erroring out on large allocations in a couple places
> as a crosscheck. I don't think this argument holds much water.

I don't understand what that has to do with it.  Surely we're not going
to have palloc_noerror() not error out when presented with a huge
allocation.  My point is just that the "noerror" bit in palloc_noerror()
means that it doesn't fail in OOM, and that there are other causes for
it to error.


One thought I just had is that we also have MemoryContextAllocHuge; are
we going to consider a mixture of both things in the future, i.e. allow
huge allocations to return NULL when OOM?  It sounds a bit useless
currently, but it doesn't seem extremely far-fetched that we will need
further flags in the future.  (Or, perhaps, we will want to have code
that retries a Huge allocation that returns NULL with a smaller size,
just in case it does work.)  Maybe what we need is to turn these things
into flags to a new generic function.  Furthermore, I question whether
we really need a "palloc" variant -- I mean, can we live with just the
MemoryContext API instead?  As with the "Huge" variant (which does not
have a corresponding palloc equivalent), possible use cases seem very
limited so there's probably not much point in providing a shortcut.

So how about something like

#define ALLOCFLAG_HUGE            0x01
#define ALLOCFLAG_NO_ERROR_ON_OOM    0x02
void *
MemoryContextAllocFlags(MemoryContext context, Size size, int flags);

and perhaps even

#define MemoryContextAllocHuge(cxt, sz) \MemoryContextAllocFlags(cxt, sz, ALLOCFLAG_HUGE)
for source-level compatibility.


(Now we all agree that palloc() itself is a very hot spot and shouldn't
be touched at all.  I don't think these new functions are used as commonly
as that, so the fact that they are slightly slower shouldn't be too
troublesome.)

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services