RE: Speed up the removal of WAL files

Thanks, it seems to require a bit more consideration about RemoveOldXLogFiles().  Let me continue this next month.


Regards
Takayuki Tsunakawa


> -----Original Message-----
> From: Michael Paquier [mailto:michael.paquier@gmail.com]
> Sent: Saturday, November 18, 2017 10:37 PM
> To: Fujii Masao
> Cc: Tsunakawa, Takayuki/綱川 貴之; Kyotaro HORIGUCHI;
> pgsql-hackers@postgresql.org
> Subject: Re: Speed up the removal of WAL files
> 
> On Sat, Nov 18, 2017 at 2:57 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> > On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki
> > <tsunakawa.takay@jp.fujitsu.com> wrote:
> >> From: Kyotaro HORIGUCHI [mailto:horiguchi.kyotaro@lab.ntt.co.jp]
> >>> The orinal code recycles some of the to-be-removed files, but the
> >>> patch removes all the victims.  This may impact on performance.
> >>
> >> Yes, I noticed it after submitting the patch and was wondering what to
> do.  Thinking simply, I think it would be just enough to replace
> durable_unlink/durable_rename in RemoveXLogFile() with unlink/rename, and
> sync the pg_wal directory once in RemoveNonParentXlogFiles() and
> RemoveOldXlogFiles().  This will benefit the failover time when fast
> promotion is not performed.  What do you think?
> 
> Note that durable_rename() also flushes the origin file to make sure that
> it does not show up again after a crash.
> 
> > It seems not good idea to just replace durable_rename() with rename()
> > in RemoveOldXlogFiles()->RemoveXlogFiles()->InstallXLogFileSegment().
> > Because that change seems to be able to cause the following problem.
> 
> If archiving is enabled, RemoveOldXlogFiles() would create a .ready flag
> for all segments that have reappeared. Oops, it archived again.