Re: BUG #18569: Memory leak in Postgres Enterprise server

On 9/20/24 12:03, Yeddula, Madhusudhan reddy [CONTINGENT WORKER] wrote:
> HI @Daniel Gustafsson <mailto:daniel@yesql.se>
> 
>  
> 
> Postgres server is restarting multiple times when we run the use case
> with 3 JOBS. We have enough resources and never seen CPU and memory is
> gong high.
> 

This thread started by claim of a memory leak, but now you claim you
don't see CPU or memory usage going up? Or do I misunderstand?

If it is a memory leak, and the memory usage grows over time, you should
be able to monitor the system, see which backend(s) allocate the memory
and inspect what queries are being executed.

>  
> 
>       Please find Postgres parameters along with application logs.
> 

Application log is not very useful for investigating this, because it
only says this:

    server closed the connection unexpectedly

    This probably means the server terminated abnormally

    before or while processing the request.

So, the database connection went away, but we're none the wiser why. We
don't even know if it was the query reported in the application log
causing the issue, it might have been some other process causing the
crash, and this query is just a victim.

You need to inspect the database log, which might tell you more - either
which backend actually crashed / why, possibly some additional info.
Also look at dmesg, which should have info about OOM killer, if that's
what killed one of the processes.

>  
> 
> MEMORY :  128GB
> 
> CPU :  32 Cores.
> 
>  
> 
>  
> 
> max_connections = 200
> 
> shared_buffers = 32GB
> 
> effective_cache_size = 96GB
> 
> maintenance_work_mem = 2GB
> 
> checkpoint_completion_target = 0.9
> 
> wal_buffers = 16MB
> 
> default_statistics_target = 100
> 
> random_page_cost = 1.1
> 
> effective_io_concurrency = 200
> 
> work_mem = 41943kB
> 
> huge_pages = try
> 
> min_wal_size = 1GB
> 
> max_wal_size = 4GB
> 
> max_worker_processes = 32
> 
> max_parallel_workers_per_gather = 4
> 
> max_parallel_workers = 32
> 
> max_parallel_maintenance_workers = 4
> 

These parameters seem reasonable. I don't see any obvious problem.


regards

-- 
Tomas Vondra