Обсуждение: max_slot_wal_keep_size unit is not specified
The following documentation comment has been logged on the website: Page: https://www.postgresql.org/docs/14/runtime-config-replication.html Description: The unit (I assume it's MB) of max_slot_wal_keep_size is not explicitly specified in the docs. If it's intentional then please, disregards, but I've not been able to deduct that from this page alone. Thanks!
On 2021-Nov-22, PG Doc comments form wrote: > The unit (I assume it's MB) of max_slot_wal_keep_size is not explicitly > specified in the docs. If it's intentional then please, disregards, but I've > not been able to deduct that from this page alone. Thanks! Well, that's embarrasing. I'll see about fixing it. It is a size-based unit. You would typically specify some unit (say, MB) together with the number; internally, because the way this works is in terms of whole files, it is rounded down to an integer number of WAL segments. If you don't specify a unit, it is taken to be a number of megabytes. I wonder why did we make it round down rather than up. Does this mean that if you have max_slot_wal_keep_size=8MB and wal segments of 16 MB, the slot gets invalidated with more than zero reserved segments? Thanks for reporting. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "I'm impressed how quickly you are fixing this obscure issue. I came from MS SQL and it would be hard for me to put into words how much of a better job you all are doing on [PostgreSQL]." Steve Midgley, http://archives.postgresql.org/pgsql-sql/2008-08/msg00000.php
At Tue, 23 Nov 2021 11:21:05 -0300, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in > On 2021-Nov-22, PG Doc comments form wrote: > > > The unit (I assume it's MB) of max_slot_wal_keep_size is not explicitly > > specified in the docs. If it's intentional then please, disregards, but I've > > not been able to deduct that from this page alone. Thanks! > > Well, that's embarrasing. I'll see about fixing it. (facepalm..) > It is a size-based unit. You would typically specify some unit (say, > MB) together with the number; internally, because the way this works is > in terms of whole files, it is rounded down to an integer number of WAL > segments. If you don't specify a unit, it is taken to be a number of > megabytes. > > I wonder why did we make it round down rather than up. Does this mean > that if you have max_slot_wal_keep_size=8MB and wal segments of 16 MB, > the slot gets invalidated with more than zero reserved segments? That rounding behavior is the way we are generally doing on similar variables. It is based on the behavior of ConvertToXSegs(). So max_wal_size for example is rounded-up the same way. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
On 2021-Nov-24, Kyotaro Horiguchi wrote: > At Tue, 23 Nov 2021 11:21:05 -0300, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in > > I wonder why did we make it round down rather than up. Does this mean > > that if you have max_slot_wal_keep_size=8MB and wal segments of 16 MB, > > the slot gets invalidated with more than zero reserved segments? > > That rounding behavior is the way we are generally doing on similar > variables. It is based on the behavior of ConvertToXSegs(). So > max_wal_size for example is rounded-up the same way. After thinking on it some more, I think rounding down is correct. If somebody sets max_slot_wal_keep_size=40MB, then by keeping the limit at 32 MB (round down) we're honoring that request better than if we made it 48 MB (round up). But in any case, I think it doesn't matter much: users need a *rough* size limit, not a super-precise one. After all, this is only verified at checkpoint time. We're probably dealing with a few hundreds of megabytes at least, so 16 MB one way or the other don't change things much. Here's a proposed patch. -- Álvaro Herrera 39°49'30"S 73°17'W — https://www.EnterpriseDB.com/ "Las mujeres son como hondas: mientras más resistencia tienen, más lejos puedes llegar con ellas" (Jonas Nightingale, Leap of Faith)
Вложения
At Thu, 25 Nov 2021 11:37:37 -0300, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in > On 2021-Nov-24, Kyotaro Horiguchi wrote: > > > At Tue, 23 Nov 2021 11:21:05 -0300, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in > > > > I wonder why did we make it round down rather than up. Does this mean > > > that if you have max_slot_wal_keep_size=8MB and wal segments of 16 MB, > > > the slot gets invalidated with more than zero reserved segments? > > > > That rounding behavior is the way we are generally doing on similar > > variables. It is based on the behavior of ConvertToXSegs(). So > > max_wal_size for example is rounded-up the same way. (no.. rounded-down X) > After thinking on it some more, I think rounding down is correct. If > somebody sets max_slot_wal_keep_size=40MB, then by keeping the limit at > 32 MB (round down) we're honoring that request better than if we made it > 48 MB (round up). But in any case, I think it doesn't matter much: > users need a *rough* size limit, not a super-precise one. After all, > this is only verified at checkpoint time. We're probably dealing with a > few hundreds of megabytes at least, so 16 MB one way or the other don't > change things much. I thought exactly on that line at development. Regarding the proposed description, we have never explained how the same kind of values specified in megabytes is converted into internal units. If we add that explanation there, we might want the same for max_wal_size, min_wal_size and wal_keep_size. > Here's a proposed patch. + The value is rounded down to an integer number of WAL files + according to <varname>wal_segment_size</varname>. The same section starts with the following sentences. > Specify the maximum size of WAL files > that <link linkend="streaming-replication-slots">replication > slots</link> are allowed to retain in the <filename>pg_wal</filename> > directory at checkpoint time. FWIW I thought (or intended) that "the maximum size of WAL files that slots are *allowed to retain*" connotes the same to the sentences cited above. Premising "16 MB one way or the other don't change things much", I'm not sure we really need to be particular about a few megabites difference especially only for this specific variable. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
On 2021-Nov-26, Kyotaro Horiguchi wrote: > Regarding the proposed description, we have never explained how the > same kind of values specified in megabytes is converted into internal > units. If we add that explanation there, we might want the same for > max_wal_size, min_wal_size and wal_keep_size. True. Since this was for backpatch, it seemed best to copy the existing entries instead of changing them. > The same section starts with the following sentences. > > > Specify the maximum size of WAL files > > that <link linkend="streaming-replication-slots">replication > > slots</link> are allowed to retain in the <filename>pg_wal</filename> > > directory at checkpoint time. > > FWIW I thought (or intended) that "the maximum size of WAL files that > slots are *allowed to retain*" connotes the same to the sentences > cited above. Right you are -- that addition was redundant, so I pushed without that. > Premising "16 MB one way or the other don't change things much", I'm > not sure we really need to be particular about a few megabites > difference especially only for this specific variable. Yeah. -- Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/