A key challenge here is how to determine the starting point for WAL receiver when the startup process starts it while still replaying WAL that's already received. Hao and I wrote a much faster and less intrusive solution to determine the starting point. Scan the first page of each WAL segment file, starting from the one that's currently being read for replay. If the first page is found valid, move to the next segment file and check its first page. Continue this one segment file at a time until either the segment file doesn't exist or the page is not valid. The starting point is then the previous segment's start address.
There is no need to read till the end of WAL, one record at a time, like the original proposal upthread did. The most recently flushed LSN does not need to be persisted on disk.
Please see attached patch which also contains a TAP test to induce replay lag and validate the fix.
Asim