Обсуждение: [GENERAL] streaming replication and archive_status

Поиск
Список
Период
Сортировка

[GENERAL] streaming replication and archive_status

От
Luciano Mittmann
Дата:
Hi All,

anyone knows why there are so many files in the directory pg_xlog/archive_status/
in replication server?

# pg_xlog/archive_status/ | wc -l

75217

Is possible to clean this .done files or just don't need to worry ?

It's not occurs on primary or standalone servers, just on replication.

Thanks.

Re: [GENERAL] streaming replication and archive_status

От
Jeff Janes
Дата:
On Mon, Apr 17, 2017 at 12:22 PM, Luciano Mittmann <mittmann@gmail.com> wrote:
Hi All,

anyone knows why there are so many files in the directory pg_xlog/archive_status/
in replication server?

# pg_xlog/archive_status/ | wc -l

75217

Is possible to clean this .done files or just don't need to worry ?

It's not occurs on primary or standalone servers, just on replication.

What version?  Are you logging checkpoints, and if so what do they say?
 
Cheers,

Jeff

Re: [GENERAL] streaming replication and archive_status

От
Luciano Mittmann
Дата:
Hi Jeff,

checkpoint message on standby node:

[ 2017-04-17 17:21:56 BRT] @ LOG:  restartpoint complete: wrote 21475 buffers (2.6%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=149.816 s, sync=0.064 s, total=149.890 s; sync files=314, longest=0.002 s, average=0.000 s; distance=145449 kB, estimate=236346 kB
[ 2017-04-17 17:21:56 BRT] @ LOG:  recovery restart point at 126/A7072A88
[ 2017-04-17 17:21:56 BRT] @ DETAIL:  last completed transaction was at log time 2017-04-17 17:21:02.289164-03




2017-04-17 17:08 GMT-03:00 Jeff Janes <jeff.janes@gmail.com>:
On Mon, Apr 17, 2017 at 12:22 PM, Luciano Mittmann <mittmann@gmail.com> wrote:
Hi All,

anyone knows why there are so many files in the directory pg_xlog/archive_status/
in replication server?

# pg_xlog/archive_status/ | wc -l

75217

Is possible to clean this .done files or just don't need to worry ?

It's not occurs on primary or standalone servers, just on replication.

What version?  Are you logging checkpoints, and if so what do they say?
 
Cheers,

Jeff

Re: [GENERAL] streaming replication and archive_status

От
Luciano Mittmann
Дата:
Version 9.6.2

Checkpoint on primary server:
[ 2017-04-17 17:23:25 BRT] @ LOG:  checkpoint complete: wrote 19436 buffers (2.4%); 0 transaction log file(s) added, 2 removed, 7 recycled; write=149.506 s, sync=0.310 s, total=149.958 s; sync files=370, longest=0.012 s, average=0.000 s; distance=133971 kB, estimate=226109 kB

thanks,

2017-04-17 17:24 GMT-03:00 Luciano Mittmann <mittmann@gmail.com>:
Hi Jeff,

checkpoint message on standby node:

[ 2017-04-17 17:21:56 BRT] @ LOG:  restartpoint complete: wrote 21475 buffers (2.6%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=149.816 s, sync=0.064 s, total=149.890 s; sync files=314, longest=0.002 s, average=0.000 s; distance=145449 kB, estimate=236346 kB
[ 2017-04-17 17:21:56 BRT] @ LOG:  recovery restart point at 126/A7072A88
[ 2017-04-17 17:21:56 BRT] @ DETAIL:  last completed transaction was at log time 2017-04-17 17:21:02.289164-03




2017-04-17 17:08 GMT-03:00 Jeff Janes <jeff.janes@gmail.com>:
On Mon, Apr 17, 2017 at 12:22 PM, Luciano Mittmann <mittmann@gmail.com> wrote:
Hi All,

anyone knows why there are so many files in the directory pg_xlog/archive_status/
in replication server?

# pg_xlog/archive_status/ | wc -l

75217

Is possible to clean this .done files or just don't need to worry ?

It's not occurs on primary or standalone servers, just on replication.

What version?  Are you logging checkpoints, and if so what do they say?
 
Cheers,

Jeff


Re: [GENERAL] streaming replication and archive_status

От
Jeff Janes
Дата:
2017-04-17 17:08 GMT-03:00 Jeff Janes <jeff.janes@gmail.com>:
On Mon, Apr 17, 2017 at 12:22 PM, Luciano Mittmann <mittmann@gmail.com> wrote:
Hi All,

anyone knows why there are so many files in the directory pg_xlog/archive_status/ 
in replication server? 

# pg_xlog/archive_status/ | wc -l

75217

Is possible to clean this .done files or just don't need to worry ?

It's not occurs on primary or standalone servers, just on replication.

What version?  Are you logging checkpoints, and if so what do they say?


On Mon, Apr 17, 2017 at 1:24 PM, Luciano Mittmann <mittmann@gmail.com> wrote:
Hi Jeff,

checkpoint message on standby node:

[ 2017-04-17 17:21:56 BRT] @ LOG:  restartpoint complete: wrote 21475 buffers (2.6%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=149.816 s, sync=0.064 s, total=149.890 s; sync files=314, longest=0.002 s, average=0.000 s; distance=145449 kB, estimate=236346 kB
[ 2017-04-17 17:21:56 BRT] @ LOG:  recovery restart point at 126/A7072A88
[ 2017-04-17 17:21:56 BRT] @ DETAIL:  last completed transaction was at log time 2017-04-17 17:21:02.289164-03

I wish the checkpoint logging code would tell you how many transaction log file it intentionally retained as well.  I've not seen it "accidentally" retain files, but seeing the number logged would help simplify troubleshooting.

Does each file in pg_xlog/archive_status/ have a corresponding file one directory up?

Cheers,

Jeff

Re: [GENERAL] streaming replication and archive_status

От
Luciano Mittmann
Дата:


2017-04-17 20:04 GMT-03:00 Jeff Janes <jeff.janes@gmail.com>:
2017-04-17 17:08 GMT-03:00 Jeff Janes <jeff.janes@gmail.com>:
On Mon, Apr 17, 2017 at 12:22 PM, Luciano Mittmann <mittmann@gmail.com> wrote:
Hi All,

anyone knows why there are so many files in the directory pg_xlog/archive_status/ 
in replication server? 

# pg_xlog/archive_status/ | wc -l

75217

Is possible to clean this .done files or just don't need to worry ?

It's not occurs on primary or standalone servers, just on replication.

What version?  Are you logging checkpoints, and if so what do they say?


On Mon, Apr 17, 2017 at 1:24 PM, Luciano Mittmann <mittmann@gmail.com> wrote:
Hi Jeff,

checkpoint message on standby node:

[ 2017-04-17 17:21:56 BRT] @ LOG:  restartpoint complete: wrote 21475 buffers (2.6%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=149.816 s, sync=0.064 s, total=149.890 s; sync files=314, longest=0.002 s, average=0.000 s; distance=145449 kB, estimate=236346 kB
[ 2017-04-17 17:21:56 BRT] @ LOG:  recovery restart point at 126/A7072A88
[ 2017-04-17 17:21:56 BRT] @ DETAIL:  last completed transaction was at log time 2017-04-17 17:21:02.289164-03

Gostaria que o código de log de ponto de verificação lhe dissesse quantos arquivos de log de transações ele intencionalmente mantidos também. Eu não vi "acidentalmente" reter arquivos, mas vendo o número registrado ajudaria a simplificar a solução de problemas.

Does each file in pg_xlog/archive_status/ have a corresponding file one directory up?

Cheers,

Jeff

Hi Jeff,

**Does each file in pg_xlog/archive_status/ have a corresponding file one directory up?

no corresponding file on pg_xlog directory. That is the question.. for
some reason or some parameter that I do not know, the files are considered consumed but are not erased later.

regards,

Re: [GENERAL] streaming replication and archive_status

От
Jeff Janes
Дата:
On Tue, Apr 18, 2017 at 5:20 AM, Luciano Mittmann <mittmann@gmail.com> wrote:


Hi Jeff,

**Does each file in pg_xlog/archive_status/ have a corresponding file one directory up?

no corresponding file on pg_xlog directory. That is the question.. for
some reason or some parameter that I do not know, the files are considered consumed but are not erased later.

I can see how a well-timed crash could leave behind a few .done files, but not 75 thousand of them.

Are they still accumulating, or was it only an historical accumulation?
 
Also, is this on Windows?

Cheers,

Jeff

Re: [GENERAL] streaming replication and archive_status

От
Luciano Mittmann
Дата:


2017-04-18 22:46 GMT-03:00 Jeff Janes <jeff.janes@gmail.com>:
On Tue, Apr 18, 2017 at 5:20 AM, Luciano Mittmann <mittmann@gmail.com> wrote:


Hi Jeff,

**Does each file in pg_xlog/archive_status/ have a corresponding file one directory up?

no corresponding file on pg_xlog directory. That is the question.. for
some reason or some parameter that I do not know, the files are considered consumed but are not erased later.

I can see how a well-timed crash could leave behind a few .done files, but not 75 thousand of them.

Are they still accumulating, or was it only an historical accumulation?
 
Also, is this on Windows?

Cheers,

Jeff

Hey Jeff,

they are still accumulating, 78k today.
Running on SLES 12.1

regards,