Обсуждение: Keep wal segments

Поиск
Список
Период
Сортировка

Keep wal segments

От
Prince Pathria
Дата:
Hi,

On postgresql10, I'm facing issue of "requested WAL segment 000000010000007E0000003F has already been removed" a lot. I resync slaves with pg_basebackup and still some days after the same thing happens with same or other replica node.

I have kept 

wal_keep_segment = 1024
wal_buffers = 16MB
min_wal_size = 1GB
max_wal_size =  2GB
archive_mode = on
archive_command =  cp %p /var/lib/pgsql/10/archive/%f
wal_level = replica                     
synchronous_commit = on
synchronous_standby_names = 'pgslave01'

Can I keep wal segments for some duration(eg 24 hours) ?

I'm using two replicas with one in sync and other in potential state.Is one synchronous_standby_names fine?

Is there any option to remove archive wal after 30 days or 50GB?

And how archiving the wal logs help? The missing segment is "000000010000007E0000003F" is in archive dir but how can I make it in use?

Thanks!!

Prince Pathria Systems Architect Evive +91 9478670472 goevive.com

Re: Keep wal segments

От
Flavio Henrique Araque Gurgel
Дата:
Em sex, 21 de set de 2018 às 09:19, Prince Pathria
<prince.pathria@goevive.com> escreveu:
>
> Hi,
>
> On postgresql10, I'm facing issue of "requested WAL segment 000000010000007E0000003F has already been removed" a lot.
Iresync slaves with pg_basebackup and still some days after the same thing happens with same or other replica node. 
>
> I have kept
>
> wal_keep_segment = 1024
> wal_buffers = 16MB
> min_wal_size = 1GB
> max_wal_size =  2GB
> archive_mode = on
> archive_command =  cp %p /var/lib/pgsql/10/archive/%f
> wal_level = replica
> synchronous_commit = on
> synchronous_standby_names = 'pgslave01'
>
> Can I keep wal segments for some duration(eg 24 hours) ?

Think about replication slots to deal with it:
https://www.postgresql.org/docs/current/static/warm-standby.html#STREAMING-REPLICATION-SLOTS

> I'm using two replicas with one in sync and other in potential state.Is one synchronous_standby_names fine?

It seems ok, you'll have one synchronous replica.

> Is there any option to remove archive wal after 30 days or 50GB?

You have to keep wal that corresponds to your oldest basebackup.
Consider using one of the backup solutions like Barman or pgbackrest.
Barman works with basebackup and replication slots and you can set a
retention policy for your backups, it handles wal keeping for you.

> And how archiving the wal logs help? The missing segment is "000000010000007E0000003F" is in archive dir but how can
Imake it in use? 

You have to setup a restore_command to retrieve it from your archive:
https://www.postgresql.org/docs/10/static/archive-recovery-settings.html#RESTORE-COMMAND


Re: Keep wal segments

От
Prince Pathria
Дата:
Thanks Flavio!! I'll try these things up.

Prince Pathria
Systems Architect Evive +91 9478670472 goevive.com


On Fri, Sep 21, 2018 at 12:59 PM Flavio Henrique Araque Gurgel <fhagur@gmail.com> wrote:
Em sex, 21 de set de 2018 às 09:19, Prince Pathria
<prince.pathria@goevive.com> escreveu:
>
> Hi,
>
> On postgresql10, I'm facing issue of "requested WAL segment 000000010000007E0000003F has already been removed" a lot. I resync slaves with pg_basebackup and still some days after the same thing happens with same or other replica node.
>
> I have kept
>
> wal_keep_segment = 1024
> wal_buffers = 16MB
> min_wal_size = 1GB
> max_wal_size =  2GB
> archive_mode = on
> archive_command =  cp %p /var/lib/pgsql/10/archive/%f
> wal_level = replica
> synchronous_commit = on
> synchronous_standby_names = 'pgslave01'
>
> Can I keep wal segments for some duration(eg 24 hours) ?

Think about replication slots to deal with it:
https://www.postgresql.org/docs/current/static/warm-standby.html#STREAMING-REPLICATION-SLOTS

> I'm using two replicas with one in sync and other in potential state.Is one synchronous_standby_names fine?

It seems ok, you'll have one synchronous replica.

> Is there any option to remove archive wal after 30 days or 50GB?

You have to keep wal that corresponds to your oldest basebackup.
Consider using one of the backup solutions like Barman or pgbackrest.
Barman works with basebackup and replication slots and you can set a
retention policy for your backups, it handles wal keeping for you.

> And how archiving the wal logs help? The missing segment is "000000010000007E0000003F" is in archive dir but how can I make it in use?

You have to setup a restore_command to retrieve it from your archive:
https://www.postgresql.org/docs/10/static/archive-recovery-settings.html#RESTORE-COMMAND