Re: [HACKERS] Possible gaps/garbage in the output of XLOG reader
От | Antonin Houska |
---|---|
Тема | Re: [HACKERS] Possible gaps/garbage in the output of XLOG reader |
Дата | |
Msg-id | 32276.1516290849@localhost обсуждение исходный текст |
Ответ на | Re: Possible gaps/garbage in the output of XLOG reader (Michael Paquier <michael.paquier@gmail.com>) |
Список | pgsql-hackers |
Michael Paquier <michael.paquier@gmail.com> wrote: > On Thu, Apr 9, 2015 at 7:05 PM, Antonin Houska <ah@cybertec.at> wrote: > > While playing with xlogreader, I was lucky enough to see one of the many > > record validations to fail. After having some fun with gdb, I found out that > > in some cases the reader does not enforce enough data to be in state->readBuf > > before copying into state->readRecordBuf starts. This should not happen if the > > callback always reads XLOG_BLCKSZ bytes, but in fact only *reqLen* is the > > mandatory size of the chunk delivered. > > > > There are probably various ways to fix this problem. Attached is what I did in > > my environment. I hit the problem on 9.4.1, but the patch seems to apply to > > master too. > > This looks similar to what is discussed here: > https://www.postgresql.org/message-id/flat/CABOikdPsPByMiG6J01DKq6om2+BNkxHTPkOyqHM2a4oYwGKsqQ@mail.gmail.com > And there is a different patch which takes a lower-level approach, and > it seems to me cleaner approach in its way of calculating the record > offset when it goes through several XLOG pages. > > Perhaps you could help review it? It is attached to the next commit fest. I don't see an overlap. The problem I reported is still there. Attached is a patch for the current HEAD. Attached is also a patch that I use to simulate the problem. It includes this hunk diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c new file mode 100644 index e42b828..e7b56b6 *** a/src/backend/access/transam/xlog.c --- b/src/backend/access/transam/xlog.c *************** retry: *** 11648,11654 **** } pgstat_report_wait_start(WAIT_EVENT_WAL_READ); ! if (read(readFile, readBuf, XLOG_BLCKSZ) != XLOG_BLCKSZ) { char fname[MAXFNAMELEN]; --- 11648,11654 ---- } pgstat_report_wait_start(WAIT_EVENT_WAL_READ); ! if (read(readFile, readBuf, readLen) != readLen) { char fname[MAXFNAMELEN]; that I posted in [1]. When the reproduce.diff was applied (but not xlogreader_readbuf_gap.diff), I could fire the contained Assert(readOff == XLOG_BLCKSZ) statement this way: 1. Create a new cluster. 2. Create another one and setup streaming replication. 3. Run regression test on master. After some time the Assert() statement fired on the standby. There are probably simpler scenarios but here I want to stress that the problem is related to [1]. [1] https://www.postgresql.org/message-id/16062.1516189589%40localhost -- Antonin Houska Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26, A-2700 Wiener Neustadt Web: https://www.cybertec-postgresql.com
Вложения
В списке pgsql-hackers по дате отправления: