Обсуждение: ERROR: invalid memory alloc request size 18446744073709551613

1111111111111111111111111111111111111111111111111111111111111101

On 2020-06-22 11:13:33 -0300, Flaris Feller wrote:
> When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on Intel
> x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL logs.
> This is the postgresq.log file's fragment of log where the error was found.
>
> 2020-06-22 00:29:18 BRT [16987]: [1-1] db=bxs,user=postgres ERRO:  invalid
> memory alloc request size 18446744073709551613

That's 2**64 - 3. So probably the size of some object is (erroneously)
computed as -3 bytes.

> 2020-06-22 00:29:18 BRT [16987]: [2-1] db=bxs,user=postgres COMANDO:  COPY
> public.cham_chamada....

Does this always happen in conjunction with a COPY command or sometimes
with other commands, too? If the former, are you copying into the
database or out of it?

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

Flaris Feller <flaris.feller@gmail.com> writes:
> Em seg., 22 de jun. de 2020 às 12:33, Peter J. Holzer <hjp-pgsql@hjp.at>
> escreveu:
>> On 2020-06-22 11:13:33 -0300, Flaris Feller wrote:
>>> When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on
>>> Intel x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL
>>> logs.

>> That's 2**64 - 3. So probably the size of some object is (erroneously)
>> computed as -3 bytes.

Yeah.  I don't recall the bit-level details offhand, but this is an
extremely common symptom of a corrupted length word in a variable-length
field.

The usual approach is to try to isolate which row or rows contains corrupt
data and then delete it.  I'm not aware of any tools for doing that
automatically, but you can usually home in on a bad row by a process of
binary search, eg testing how many rows you can fetch without seeing the
error.  Also "select * from mytab where ctid = '(m,n)'" is useful for
probing individual rows, once you get close to the problem area.

            regards, tom lane

Flaris Feller <flaris.feller@gmail.com> writes:

Em seg., 22 de jun. de 2020 às 12:33, Peter J. Holzer <hjp-pgsql@hjp.at>

escreveu:

On 2020-06-22 11:13:33 -0300, Flaris Feller wrote:

When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on

Intel x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL

logs.

That's 2**64 - 3. So probably the size of some object is (erroneously)

computed as -3 bytes.

Yeah.  I don't recall the bit-level details offhand, but this is an
extremely common symptom of a corrupted length word in a variable-length
field.

The usual approach is to try to isolate which row or rows contains corrupt
data and then delete it.  I'm not aware of any tools for doing that
automatically, but you can usually home in on a bad row by a process of
binary search, eg testing how many rows you can fetch without seeing the
error.  Also "select * from mytab where ctid = '(m,n)'" is useful for
probing individual rows, once you get close to the problem area.

           regards, tom lane



On Jun 22, 2020, at 8:13 AM, Flaris Feller <flaris.feller@gmail.com> wrote:

Hello all,

When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on Intel x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL logs.

This is the postgresq.log file's fragment of log where the error was found.

2020-06-22 00:29:18 BRT [16987]: [1-1] db=bxs,user=postgres ERRO:  invalid memory alloc request size 18446744073709551613
2020-06-22 00:29:18 BRT [16987]: [2-1] db=bxs,user=postgres COMANDO:  COPY public.cham_chamada....

PostgreSQL was installed using official PGDG repository through yum.

This is the list of postgresql.conf settings used in the cluster.

listen_addresses = '*'
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
log_filename = 'postgresql-%a.log'
log_truncate_on_rotation = on
log_rotation_age = 1d
log_rotation_size = 0
autovacuum = off
datestyle = 'iso, mdy'
lc_messages = 'pt_BR'                  
lc_monetary = 'pt_BR'                  
lc_numeric = 'pt_BR'                    
lc_time = 'pt_BR'                              
default_text_search_config = 'pg_catalog.portuguese'
max_connections=150
shared_buffers=2GB
effective_cache_size=4GB
work_mem=13981kB
maintenance_work_mem=256MB
log_min_duration_statement = 300000
log_line_prefix = '%t [%p]: [%l-1] db=%d,user=%u '
log_lock_waits = on
timezone='America/Sao_paulo'
log_timezone = 'Brazil/East'
min_wal_size = 1GB
max_wal_size = 2GB

Looking at previous list's messages I've found this could be a data corruption issue and I've followed the recommended procedures.

But even removing the corrupted records, doing a vacuum full and re-indexing the table the problem keep going recurrently.

So I would like some guidance to find the root cause of the table corruption on the database.

Any help would be appreciated.

Thanks in advance,

Flaris Feller.

I there an application generating this value perhaps?

1111111111111111111111111111111111111111111111111111111111111101

Looks a little like a signed/un-signed mismatch