On Thu, Sep 13, 2018 at 02:56:42PM -0600, Jerry Jelinek wrote:
> I'll take a look at that. I had been trying to keep the patch as minimal as
> possible, but I'm happy to work through this.
(Please be careful with top-posting)
Jerry, the last status was from three weeks ago with the patch waiting
on the author, so I am marking it as returned with feedback.
--
Michael
I'd like to see if I can get this patch moving forward again. I have made several changes to the patch since the last time this was discussed.
First, since last fall, we have found another performance problem related to initializing WAL files. I've described this issue in more detail below, but in order to handle this new problem, I decided to generalize the patch so the tunable refers to running on a Copy-On-Write filesystem instead of just being specific to WAL recycling. Specifically, I renamed the GUC tunable from 'wal_recycle' to 'wal_cow_fs'. Hopefully this will make it more obvious what is being tuned and will also be more flexible if there are other problems in the future which are related to running on a COW filesystem. I'm happy to choose a different name for the tunable if people don't like 'wal_cow_fs'.
Second, I've modified the WAL recycling code change as requested earlier.
Third, this new patch is rebased onto the current code base.
Finally, the patch now includes bypassing the zero-fill for new WAL files when wal_cow_fs is true. Hopefully it should be obvious why this is unnecessary for a COW filesystem, but here is some more information about how this can cause a performance problem, at least on ZFS. As background, internally ZFS will skip allocating zero-filled blocks, but that is handled fairly late in the filesystem code flow. First, all of the thousands of initialization 8k write system calls are being made by Postgres. ZFS will throttle writes under some circumstances. We found that all of the writes from XLogFileInit, while Postgres is also doing an autovacuum, will trigger write throttling due to the large amount of write traffic induced by the autovacuum. This problem occurs even when WAL files are being recycled. That seems to be related to the fact that the checkpointer – which is responsible for WAL file recycling – is taking so long that it is falling behind its own estimate for WAL activity.
The revised patch is attached. Please let me know if there are any comments.