Thank you for the insights regarding TOASTing!
I reduced the number of columns to 4, which makes my tuple size ~1KB.
With this, the compression overhead went away, but even now <25% of the time is being spent in system calls when I analyze strace.
Perf tool shows that there is a non-trivial amount of time spent in DropCachedPlan and AllocSetAlloc. Does this hint to some sort of caching being done by PostgreSQL, and is there a way to disable this caching?
Any other insights would be really helpful!
Thanks in advance,
Rohan
Thomas Kellerer <spam_eater@gmx.net> writes:
> Jerry Sievers schrieb am 20.02.2019 um 21:19:
>>> My workload is simple. I insert 1 million rows into a table with 100
>>> columns, where each column is 256 bytes in length, and every 10
>>> inserts are batched into a transaction.
>> Your test workload qualifies for TOASTing due to the $unrealistically
>> long physical tuple size.
> Hmm. I though TOAST is only applied to single values, not the entire tuple (row)?
> As each column is substantially shorter than the TOAST threshold, I would not expect toasting to kick in here.
Well, the entire tuple would be 25600 bytes plus some overhead, which
cannot fit on a Postgres page (8K, unless the OP changed compile options
without mentioning it). So something has to be done to make it fit, and
that something is going to be toasting any fields that can be toasted.
From memory, our threshold for trying to make tuples narrower is only
a quarter-page anyway, so that the toaster will be trying to get this
down to 2K if it can. That's certainly going to involve compressing
every field, and I wouldn't be surprised if a lot of them get shoved
out-of-line too.
The OP might care to read
https://www.postgresql.org/docs/current/storage-toast.html
regards, tom lane