On Wed, Nov 20, 2019 at 03:50:29PM +0100, Antonin Houska wrote:
> Thus the use of pg_pread() makes the code quite a bit simpler, so I
> re-introduced it. If you decide that an explicit lseek() should be used yet,
> just let me know.
Skimming through the code, it looks like in a good state. The
failures of test_deconding are fixed, and all the changes from Alvaro
have been added.
+ fatal_error("could not read in file %s, offset %u, length %zu: %s",
+ fname, seg->ws_off, (Size) errinfo.wre_req,
+ strerror(errinfo.wre_errno));
You should be able to use %m here instead of strerror().
It seems to me that it is always important to not do changes
completely blindly either so as this does not become an issue for
recovery later on. FWIW, I ran a small set of tests with a WAL
segment sizes of 1MB and 1GB (fsync = off, max_wal_size/min_wal_size
set very high, 1 billion rows in single-column table followed by a
series of updates):
- Created a primary and a standby which archive_mode set.
- Stopped the standby.
- Produced close to 12GB worth of WAL.
- Restarted the standby with restore_command and compared the time it
takes for recovery to complete all the segments with HEAD and your
refactoring:
1GB + HEAD: 7min52s
1GB + patch: 8min10s
1MB + HEAD: 10min17s
1MB + patch: 12min1s
And with WAL segments at 1MB, I was seeing quite a slowdown with the
patch... Then I have done an extra test with pg_waldump with the
segments generated previously with the output redirected to /dev/null.
Going through 512 segments takes 15.730s with HEAD (average of 3 runs)
and 15.851s with the patch.
--
Michael