On 17/06/2021 04:12, Michael Paquier wrote:
> On Wed, Jun 16, 2021 at 11:49:51AM +0300, Heikki Linnakangas wrote:
>> Hmm, do we currently compress each block in a WAL record separately, for
>> records that contain multiple full-page images? That could make a big
>> difference e.g. for GiST index build that WAL-logs 32 pages in each record.
>> If it helps the compression, we should probably start WAL-logging b-tree
>> index build in larger batches, too.
>
> Each block is compressed alone, see XLogCompressBackupBlock() in
> XLogRecordAssemble() where we loop through each block. Compressing a
> group of blocks would not be difficult (the refactoring may be
> trickier than it looks) but I am wondering how we should treat the
> case where we finish by not compressing a group of blocks as there is
> a safety fallback to not enforce a failure if a block cannot be
> compressed. Should we move back to the compression of individual
> blocks or just log all those pages uncompressed without their holes?
Just log all the pages uncompressed in that case. If you didn't save any
bytes by compressing the pages together, surely compressing them one by
one would be even worse.
> I really don't expect a group of blocks to not be compressed, just
> being a bit paranoid here about the fallback we'd better have.
Yeah, there will inevitably be some common bytes in the page and tuple
headers, if you compress multiple pages together. But I don't think the
fallback is that important anyway. Even in the worst case, the
compressed image of something uncompressible should not be much larger
than the original.
- Heikki