BUG #16167: frontend psql fails 'select * from table' if nrows * ncolumns = 2**32 in a table.

Поиск
Список
Период
Сортировка
От PG Bug reporting form
Тема BUG #16167: frontend psql fails 'select * from table' if nrows * ncolumns = 2**32 in a table.
Дата
Msg-id 16167-ceb3fb3f8732e776@postgresql.org
обсуждение исходный текст
Список pgsql-bugs
The following bug has been logged on the website:

Bug reference:      16167
Logged by:          cili
Email address:      cilizili@protonmail.com
PostgreSQL version: 12.1
Operating system:   CentOS7.7 (1908)
Description:

I tested in 13devel. 
In the fe_utils and libpq, nrows as tuples and ncolumns as attributes are
int type parameters, and the maximum number of tuples is INT_MAX which is
hard coded.
I create a table with 67108864 tuples and 64 attributes in following sample.
I've made sure it is possible, even though it requires a lot of memory and
storage.
It causes integer overflows in the following two places [1] and [2], since
67108864 (0x04000000) times 64 (0x40) is 2**32 (0x100000000) that is larger
than UINT_MAX of 32 bits, then it is truncated to 0. 

[1] src/fe_utils/print.c:L3084 in printTableInit()
    content->cell = pg_malloc0((ncolumns * nrows + 1) *
sizeof(*content->cells));

[2] src/fe_utils/print.c:L3518 in printTableAddCell()
    if (content->cellsadded >= content->ncolumns * content->nrows)
        {
            fprintf(stderr, ...

In the first place, (ncolumns * nrows + 1) will be truncated to 0+1=1, then
it allocates a small memory, such as 8 bytes. 
Before to use it, the client process is terminated with error condition in
the second place where cellsadded has been initialized with 0 and compared
to ncoluns*nrows=0 within integer type.
The error message, "Cannot add header to table content: column count of 0
exceeded."  may not be correct.

>>>>>>>>>>>>>>> insturctions and result <<<<<<<<<<<<<<<<

postgres=# psql postgres
psql (13devel)
Type "help" for help
postgres=# \i test.sql
CREATE FUNCTION
CREATE TABLE
 foo
----

(1 row)

postgres=# select * from bar;
Cannot add header to table content: column count of 0 exceeded.


>>>>>>>>>>>>>>> test.sql <<<<<<<<<<<<<<<<
-- WARNING: This sample will require Mem: 16GB, Swap: 72GB, 22GB of storage,
and a lot of time.
-- It takes 2**32 cells in a table where the tuples(columns) and
attributes(columns) are within the limits specified in the document.

-- create function
CREATE OR REPLACE FUNCTION foo(n INTEGER)
RETURNS VOID AS $$
  BEGIN
    FOR i IN 1..n LOOP
    INSERT INTO bar (
    id, i0, i1, i2, i3, i4, i5, i6,
    i7, i8, i9, i10,i11,i12,i13,i14,
    i15,i16,i17,i18,i19,i20,i21,i22,
    i23,i24,i25,i26,i27,i28,i29,i30,
    i31,i32,i33,i34,i35,i36,i37,i38,
    i39,i40,i41,i42,i43,i44,i45,i46,
    i47,i48,i49,i50,i51,i52,i53,i54,
    i55,i56,i57,i58,i59,i60,i61,i62
    ) VALUES (
    i, 0, 1, 2, 3, 4, 5, 6,
    7, 8, 9, 10,11,12,13,14,
    15,16,17,18,19,20,21,22,
    23,24,25,26,27,28,29,30,
    31,32,33,34,35,36,37,38,
    39,40,41,42,43,44,45,46,
    47,48,49,50,51,52,53,54,
    55,56,57,58,59,60,61,62);
    END LOOP;
  END;
$$ LANGUAGE plpgsql;

-- create table
CREATE TABLE bar (
id INTEGER, i0 INTEGER, i1 INTEGER, i2 INTEGER,
i3 INTEGER, i4 INTEGER, i5 INTEGER, i6 INTEGER,
i7 INTEGER, i8 INTEGER, i9 INTEGER, i10 INTEGER,
i11 INTEGER,i12 INTEGER,i13 INTEGER,i14 INTEGER,
i15 INTEGER,i16 INTEGER,i17 INTEGER,i18 INTEGER,
i19 INTEGER,i20 INTEGER,i21 INTEGER,i22 INTEGER,
i23 INTEGER,i24 INTEGER,i25 INTEGER,i26 INTEGER,
i27 INTEGER,i28 INTEGER,i29 INTEGER,i30 INTEGER,
i31 INTEGER,i32 INTEGER,i33 INTEGER,i34 INTEGER,
i35 INTEGER,i36 INTEGER,i37 INTEGER,i38 INTEGER,
i39 INTEGER,i40 INTEGER,i41 INTEGER,i42 INTEGER,
i43 INTEGER,i44 INTEGER,i45 INTEGER,i46 INTEGER,
i47 INTEGER,i48 INTEGER,i49 INTEGER,i50 INTEGER,
i51 INTEGER,i52 INTEGER,i53 INTEGER,i54 INTEGER,
i55 INTEGER,i56 INTEGER,i57 INTEGER,i58 INTEGER,
i59 INTEGER,i60 INTEGER,i61 INTEGER,i62 INTEGER,
PRIMARY KEY (id));

-- set values into table. It takes a lot of time.
select foo(67108864);

-- Let's test with 'select * from bar;' in interactive command line.


В списке pgsql-bugs по дате отправления:

Предыдущее
От: Heikki Linnakangas
Дата:
Сообщение: Re: Planning time is high in Postgres 11.5 Compared with Postgres10.11
Следующее
От: Tomas Vondra
Дата:
Сообщение: Re: BUG #16104: Invalid DSA Memory Alloc Request in Parallel Hash