Обсуждение: BUG #2694: Memory allocation error when selecting array of empty arrays

The following bug has been logged online:

Bug reference:      2694
Logged by:          Vitali Stupin
Email address:      Vitali.Stupin@ria.ee
PostgreSQL version: 8.1.4
Operating system:   sparc-sun-solaris2.10
Description:        Memory allocation error when selecting array of empty
arrays
Details:

The error "invalid memory alloc request size 4294967293" apears when
selecting array of empty arrays:

select ARRAY['{}'::text[],'{}'::text[]];

This bug can be also reproduced on:
PostgreSQL 8.1.3 on sparc-sun-solaris2.10;
PostgreSQL 8.1.3 on i686-pc-linux-gnu.

The only difference is that linux version of postgre is trying to allocate
4294967294 (or 2^32 - 2) bytes.


...Sorry for posting the same bug report for the second time, but the
previous post sent almost a month ago (bug reference 2641) still had not
appeared in bug-list.

"Vitali Stupin" <Vitali.Stupin@ria.ee> writes:
> The error "invalid memory alloc request size 4294967293" apears when
> selecting array of empty arrays:
> select ARRAY['{}'::text[],'{}'::text[]];

I can get a core dump off it too, sometimes.  The problem is in
ExecEvalArray, which computes the dimension of the result as [1:2]
even though there are no elements to put in it.

Joe, what do you think about this?  Offhand I think that the only
workable definition is that this case yields another zero-dimensional
array, but maybe there is another choice?

We should probably check all the other array operations to see if they
have comparable problems.

            regards, tom lane

Tom Lane wrote:
> "Vitali Stupin" <Vitali.Stupin@ria.ee> writes:
>
>>The error "invalid memory alloc request size 4294967293" apears when
>>selecting array of empty arrays:
>>select ARRAY['{}'::text[],'{}'::text[]];
>
> I can get a core dump off it too, sometimes.  The problem is in
> ExecEvalArray, which computes the dimension of the result as [1:2]
> even though there are no elements to put in it.
>
> Joe, what do you think about this?  Offhand I think that the only
> workable definition is that this case yields another zero-dimensional
> array, but maybe there is another choice?

Sorry for the slow response -- I'm at the airport just heading home from
a marathon 30 day business trip.

I think producing another zero-dimensional result is the only way that
makes sense unless/until we change multidimensional arrays to really be
arrays of array-datatype elements. Right now they're two different things.

> We should probably check all the other array operations to see if they
> have comparable problems.

Yes -- I'll see if I can find the time over the next couple weeks while
home.

Joe

Joe Conway <mail@joeconway.com> writes:
> Sorry for the slow response -- I'm at the airport just heading home from
> a marathon 30 day business trip.

Yow.  Hope you get some time off...

> Tom Lane wrote:
>> "Vitali Stupin" <Vitali.Stupin@ria.ee> writes:
>>> The error "invalid memory alloc request size 4294967293" apears when
>>> selecting array of empty arrays:
>>> select ARRAY['{}'::text[],'{}'::text[]];
>>
>> Joe, what do you think about this?  Offhand I think that the only
>> workable definition is that this case yields another zero-dimensional
>> array, but maybe there is another choice?

> I think producing another zero-dimensional result is the only way that
> makes sense unless/until we change multidimensional arrays to really be
> arrays of array-datatype elements. Right now they're two different things.

On looking at the code, I notice that this somewhat-related case works:

regression=# select array[null::text[], null::text[]];
 array
-------
 {}
(1 row)

The reason is that null inputs are just ignored in ExecEvalArray.  So
one pretty simple patch would be to ignore zero-dimensional inputs too.
This would have implications for mixed inputs though: instead of

regression=# select array['{}'::text[], '{a,b,c}'::text[]];
ERROR:  multidimensional arrays must have array expressions with matching dimensions

you'd get behavior like

regression=# select array[null::text[], '{a,b,c}'::text[]];
   array
-----------
 {{a,b,c}}
(1 row)

Which of these seems more sane?

            regards, tom lane

Tom Lane wrote:
> Joe Conway <mail@joeconway.com> writes:
>>Sorry for the slow response -- I'm at the airport just heading home from
>>a marathon 30 day business trip.
>
> Yow.  Hope you get some time off...

Yeah, I just took a week. Next week I'm back to work and the week after
that I'm back to Germany for a few...

> On looking at the code, I notice that this somewhat-related case works:
>
> regression=# select array[null::text[], null::text[]];
>  array
> -------
>  {}
> (1 row)
>
> The reason is that null inputs are just ignored in ExecEvalArray.  So
> one pretty simple patch would be to ignore zero-dimensional inputs too.
> This would have implications for mixed inputs though: instead of
>
> regression=# select array['{}'::text[], '{a,b,c}'::text[]];
> ERROR:  multidimensional arrays must have array expressions with matching dimensions
>
> you'd get behavior like
>
> regression=# select array[null::text[], '{a,b,c}'::text[]];
>    array
> -----------
>  {{a,b,c}}
> (1 row)
>
> Which of these seems more sane?

I'm not sure I love either. I would think both NULL and empty array
expressions should be disallowed in this scenario, i.e.:

    regression=# select array['{}'::text[], '{a,b,c}'::text[]];
    ERROR:  multidimensional arrays must have array expressions with
matching dimensions

    regression=# select array[NULL::text[], '{a,b,c}'::text[]];
    ERROR:  multidimensional arrays must have array expressions with
    matching dimensions

In both cases you are trying to construct a multidimensional array with
inconsistent dimensions.

On the other hand, building an N-dimension array from entirely empty
array expressions should just produce an empty array, while using all
NULL expressions should produce an N-dim array full of NULLs.

But as I've opined before, all of this seems to me to be much cleaner if
arrays were always one-dimensional, and array elements could also be
nested arrays (per SQL 2003). If we said that the cardinality of the
nested array is an integral part of the datatype, then I think you would
have:

    regression=# select array['{}'::text[], '{a,b,c}'::text[]];
    ERROR:  nested arrays must have array expressions with matching
    dimensions

    regression=# select array[NULL::text[], '{a,b,c}'::text[]];
        array
     -----------
      {NULL, {a,b,c}}
     (1 row)

So maybe this is the behavior we should shoot for now?

Joe

Joe Conway <mail@joeconway.com> writes:
> ... But as I've opined before, all of this seems to me to be much cleaner if
> arrays were always one-dimensional, and array elements could also be
> nested arrays (per SQL 2003). If we said that the cardinality of the
> nested array is an integral part of the datatype, then I think you would
> have:

>     regression=# select array['{}'::text[], '{a,b,c}'::text[]];
>     ERROR:  nested arrays must have array expressions with matching
>     dimensions

>     regression=# select array[NULL::text[], '{a,b,c}'::text[]];
>         array
>      -----------
>       {NULL, {a,b,c}}
>      (1 row)

> So maybe this is the behavior we should shoot for now?

Perhaps that's something to think about for 8.3, but it's clearly
infeasible for 8.2, to say nothing of the back branches.  What I've
just committed does this:

* if all inputs are empty arrays, return an empty array (instead of failing)

* if mix of empty and nonempty arrays, raise error (same as before)

The case of NULL subarrays is a bit weird.  In the back branches the
ARRAY[] construct was strict and would return a NULL array if any input
was NULL; I think it would be a bad idea to change that.  The HEAD code
was simply skipping over NULL subarrays, which seemed like a good idea
at the time but on reflection is not so hot.  What I've done is to make
it treat NULL subarrays as if they were empty subarrays.

            regards, tom lane