Re: Invalid memory alloc

> Hello, I'm processing  a 100Million row table.
> I get error message about memory and I'ld like to know what can cause this issue.
>
> ...
> psql:/home/ubuntu/create_topo.sql:12: NOTICE:  104855000 edges processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE:  104856000 edges processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE:  104857000 edges processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE:  invalid memory alloc request size 1677721600
> psql:/home/ubuntu/create_topo.sql:12: NOTICE:  UPDATE public.way_noded SET source = 88374866,target = 88362922 WHERE
id=  142645362 
>  pgr_createtopology
> --------------------
>  FAIL
> (1 row)
>
> The server  has a 10Gb of shared_buffer.
> Do you thing this quantity of memory allowed should normaly be enough to process the data?
>
> Thanks
> Marc

Hi,

what version of Postgres are you using? Any extensions? I guess you're using PostGIS, right?

This error indicates something is trying to allocate 1600 MB of memory in the backend - that
should never happen, as data chunks that are larger than 1 GB are broken down in smaller pieces.

I hope you're not suffering data corruption, what happens if you do select * from public.way_noded
where id =  142645362 ?

Any other hints? Log messages (from Linux or the postgres backend)?

Bye,
Chris.