My apologies for the dearth of details. No on both the cpu and errors. But I do believe it is IO related. I just can't
findit.
I thought maybe it was index splitting so I altered the unique index with filterfactor=40 and reindexed. No change.
I then dropped the unique index. No change.
I thought maybe it was checkpoint timeouts, but there was no correlation.
Oddly enough other jobs running concurrently, are also inserting, most likely into different partitions, are running
about2x faster than others.
I'm rather perplexed.
pg
-----Original Message-----
From: Kenneth Marshall <ktm@rice.edu>
Sent: Wednesday, November 24, 2021 1:20 PM
To: Godfrin, Philippe E <Philippe.Godfrin@nov.com>
Cc: pgsql-general@lists.postgresql.org
Subject: [EXTERNAL] Re: Inserts and bad performance
On Wed, Nov 24, 2021 at 07:15:31PM +0000, Godfrin, Philippe E wrote:
> Greetings
> I am inserting a large number of rows, 5,10, 15 million. The python code commits every 5000 inserts. The table has
partitionedchildren.
>
> At first, when there were a low number of rows inserted, the inserts would run at a good clip - 30 - 50K inserts per
second.Now, after inserting oh say 1.5 Billion rows, the insert rate has dropped to around 5000 inserts per second. I
droppedthe unique index , rebuilt the other indexes and no change. The instance is 16 vcpu and 64GB ram.
>
> I'm perplexed, I can't see to find any reason for the slow down...
> Thanks,
> pg
Hi,
With not much information, it may be I/O related. CPU and RAM cannot fix that once items need to be written to disk.
Arethere any errors in the logs or CPUs maxxed out?
Regards,
Ken