At Mon, 11 May 2020 20:12:04 -0400, Bruce Momjian <bruce@momjian.us> wrote in
> On Thu, May 7, 2020 at 09:22:02PM -0700, Noah Misch wrote:
> > On Thu, May 07, 2020 at 09:38:34AM -0400, Bruce Momjian wrote:
> > > > > > - Crash recovery was losing tuples written via COPY TO. This fixes the bug.
> > > > >
> > > > > This was not backpatched?
> > > >
> > > > Right.
> > >
> > > Oh. So you are saying we could lose COPY data on a crash, even after a
> > > commit. That seems bad. Can you show me the commit info? I can't find
> > > it.
> >
> > commit c6b9204
> > Author: Noah Misch <noah@leadboat.com>
> > AuthorDate: Sat Apr 4 12:25:34 2020 -0700
> > Commit: Noah Misch <noah@leadboat.com>
> > CommitDate: Sat Apr 4 12:25:34 2020 -0700
> >
> > Skip WAL for new relfilenodes, under wal_level=minimal.
> >
> > Until now, only selected bulk operations (e.g. COPY) did this. If a
> > given relfilenode received both a WAL-skipping COPY and a WAL-logged
> > operation (e.g. INSERT), recovery could lose tuples from the COPY. See
> > src/backend/access/transam/README section "Skipping WAL for New
> > RelFileNode" for the new coding rules. Maintainers of table access
> > methods should examine that section.
>
> OK, so how do we want to document this? Do I mention in the text below
> the WAL skipping item that this fixes a bug where a mix of simultaneous
> COPY and INSERT into a table could lose rows during crash recovery, or
> create a new item?
FWIW, as dicussed upthread, I suppose that the API change is not going
to be in relnotes.
something like this?
- Fix bug of WAL-skipping optimiazation
Previously a trasaction doing both of COPY and a WAL-logged operations
like INSERT while wal_level=minimal can lead to loss of COPY'ed rows
through crash recovery. Also this fix extends the WAL-skipping
optimiazation to all kinds of bulk insert operations.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center