At Wed, 13 May 2020 11:15:18 -0400, Bruce Momjian <bruce@momjian.us> wrote in
> On Wed, May 13, 2020 at 11:56:33AM +0900, Kyotaro Horiguchi wrote:
> > >
> > > Allow skipping of WAL for new tables and indexes if wal_level is
> > > 'minimal' (Kyotaro Horiguchi)
> > >
> > > Relations larger than wal_skip_threshold will have their files
> > > fsync'ed rather than writing their WAL records. Previously this
> > > was done only for COPY operations, but the implementation had a
> > > bug that could cause data loss during crash recovery.
> >
> > I see it. It is giving weight on improvement. Looks good the overall
> > structure of the description above. However, wal-skipping is always
> > done regardless of table size. wal_skip_threshold is an optimization
> > to choose which to use fsync or FPI records (that is, not WAL records
> > in the common sense) at commit for speed.
>
> Well, as far as users are concerned, everything wrtiten to WAL is a WAL
> record.
I think that the significant point here is not that persistence is
ensured by which of fsync or WAL , but whether WAL records are written
for every insertion. The commit-time WA is just an alternative of
fsync, which is faster than fsync'ing separate files for smaller
files.
> > So how about the following?
> >
> > All kinds of bulk-insertion are not WAL-logged then fsync'ed at
> > commit. Using FPI WAL records instead of fsync for relations smaller
> > than wal_skip_threshold. Previously this was done only for COPY
> > operations and always using fsync, but the implementation had a bug
> > that could cause data loss during crash recovery.
>
> That is too much detail for the release notes. We already will link to
> the docs. Why put it here?
It is just an more accurate (not an detailed) version of the
previously proposed description. If we simplify that, I choose to
remove explanation on wal_skip_threshold.
How about this?
WAL-logging is now skipped while all kinds of bulk-insertion, then
relations are sync'ed to disk at commit. Previously this was done
only for COPY operations, but the implementation had a bug that could
cause data loss during crash recovery.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center