Обсуждение: [HACKERS] Removing wal_keep_segments as default configuration in PostgresNode.pm

Hi all,

Right now, PostgresNode.pm uses this set of parameters when initializing a node:
    print $conf "\n# Added by PostgresNode.pm\n";
    print $conf "fsync = off\n";
    print $conf "restart_after_crash = off\n";
    print $conf "log_line_prefix = '%m [%p] %q%a '\n";
    print $conf "log_statement = all\n";
    print $conf "wal_retrieve_retry_interval = '500ms'\n";
    print $conf "port = $port\n";

When streaming is enabled, the following set is used:
        print $conf "wal_level = replica\n"; # replace by logical here
is you need ;)
        print $conf "max_wal_senders = 5\n";
        print $conf "max_replication_slots = 5\n";
        print $conf "max_wal_size = 128MB\n";
        print $conf "shared_buffers = 1MB\n";
        print $conf "wal_log_hints = on\n";
        print $conf "hot_standby = on\n";
        print $conf "max_connections = 10\n";

While all those settings are good defaults in my opinion, I have been
trapped by wal_keep_segments being present when designing a test. The
TAP test in question here was forcing WAL segments to be recycled with
two checkpoints and some forced segment switches to make a
disconnected standby sync back to a primary using some archives, but
then I got surprised that max_wal_size was not respected. Until I
noticed that keep_wal_segments was in play.

I tend to think that while all the other parameters make sense to
deploy instances that need few resources, wal_keep_segments may cause
up to 350MB of WAL segments to be kept in each pg_wal's instance,
while max_wal_size is set at 128MB. The only test in the code tree in
need of wal_keep_segments is actually pg_rewind, which enforces
checkpoints after the rewind to update the source's control file.

So, thoughts about the attached that reworks this portion of PostgresNode.pm?
Thanks,
-- 
Michael

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 9/11/17 21:55, Michael Paquier wrote:
> I tend to think that while all the other parameters make sense to
> deploy instances that need few resources, wal_keep_segments may cause
> up to 350MB of WAL segments to be kept in each pg_wal's instance,
> while max_wal_size is set at 128MB. The only test in the code tree in
> need of wal_keep_segments is actually pg_rewind, which enforces
> checkpoints after the rewind to update the source's control file.
> 
> So, thoughts about the attached that reworks this portion of PostgresNode.pm?

Committed.

Besides the resource usage, it would probably be bad if a
wal_keep_segments setting papered over problems with replication slots
for example.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Nov 2, 2017 at 4:47 PM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 9/11/17 21:55, Michael Paquier wrote:
>> I tend to think that while all the other parameters make sense to
>> deploy instances that need few resources, wal_keep_segments may cause
>> up to 350MB of WAL segments to be kept in each pg_wal's instance,
>> while max_wal_size is set at 128MB. The only test in the code tree in
>> need of wal_keep_segments is actually pg_rewind, which enforces
>> checkpoints after the rewind to update the source's control file.
>>
>> So, thoughts about the attached that reworks this portion of PostgresNode.pm?
>
> Committed.
>
> Besides the resource usage, it would probably be bad if a
> wal_keep_segments setting papered over problems with replication slots
> for example.

Thanks! I almost forgot this patch.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers