On Thu, Feb 1, 2018 at 7:21 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > Yes, it would be about 99% of the time. > > But you have it backwards - we are not assuming that case. That is the > only case that has risk - the one where an old WAL record starts at > exactly the place the latest one stops. Otherwise the rest of the WAL > record will certainly fail the CRC check, since it will effectively > have random data in it, as you say.
OK, I get it now. Thanks for explaining. I think I understand now why you think this problem can be solved just by controlling the way we recycle segments, but I'm still not sure if that can really be made fully reliable. Michael seems concerned about what might happen after multiple recyclings, and Tom has raised the issue of old data reappearing after a crash.
I'm not sure if Michael has spotted a real problem or was that just a concern. He himself later rightly pointed out that when a WAL file is switched, the old file is filled with zeros. So I don't see a problem there. May be I am missing something and Michael can explain further.
Regarding Tom's concerns, that could be a problem if a file system crash survives a name change, but not the subsequent data written to the file. For this to be a problem, WAL file A is renamed to B and then renamed to C. File A and C share the same low order bits. Further upon file system crash, the file is correctly named as C, but the data written *before* the rename operation is lost. Is that a real possibility? Can we delay reusing low order bits a little further to address this problem? Of course, if the file system crash can survive many renames and still resurrect old data several renames before, then we shall have the same problem.