Обсуждение: question about zeroes in the wal file names

I'm just curious to better understand the naming convention behind wal
files, because I've seen on a system of mine that the wals created
were:

000000050000020E000000FF
 000000050000020F00000000

while I was expecting 20E0x100. So I digged into the code and I've
seen, from the XLogFileName macro, that the last part is built as the
reminder of the number of segments per wal file:

#define XLogFileName(fname, tli, logSegNo, wal_segsz_bytes)    \
    snprintf(fname, MAXFNAMELEN, "%08X%08X%08X", tli,        \
             (uint32) ((logSegNo) / XLogSegmentsPerXLogId(wal_segsz_bytes)), \
             (uint32) ((logSegNo) % XLogSegmentsPerXLogId(wal_segsz_bytes)))


and with the default wal size of 16 MB that gives a remainder of 256
(FF + 1). Assuming I haven't missed anything, this means that there
are 6 zeroes that will never change in the last 8 chars of the wal
filename.  Is therefore this only done to handle PostgreSQL WAL sizes
of 4 GB each?

Thanks,
Luca

At Sun, 18 Aug 2019 16:17:03 +0200, Luca Ferrari <fluca1978@gmail.com> wrote in
<CAKoxK+67oS+PSygdmRHbn-GSjE_TY+t6Gzk6CRzcw-HVGswzOA@mail.gmail.com>
> I'm just curious to better understand the naming convention behind wal
> files, because I've seen on a system of mine that the wals created
> were:
> 
> 000000050000020E000000FF
>  000000050000020F00000000
> 
> while I was expecting 20E0x100. So I digged into the code and I've
> seen, from the XLogFileName macro, that the last part is built as the
> reminder of the number of segments per wal file:
> 
> #define XLogFileName(fname, tli, logSegNo, wal_segsz_bytes)    \
>     snprintf(fname, MAXFNAMELEN, "%08X%08X%08X", tli,        \
>              (uint32) ((logSegNo) / XLogSegmentsPerXLogId(wal_segsz_bytes)), \
>              (uint32) ((logSegNo) % XLogSegmentsPerXLogId(wal_segsz_bytes)))
> 
> 
> and with the default wal size of 16 MB that gives a remainder of 256
> (FF + 1). Assuming I haven't missed anything, this means that there
> are 6 zeroes that will never change in the last 8 chars of the wal
> filename.  Is therefore this only done to handle PostgreSQL WAL sizes
> of 4 GB each?

I think that's right. I don't see it used in other than WAL file
names.  As the macro name suggests, the 4GB block is internally
called as "xlog id" (XLogId). The block size is determined so
that the location within it can be handled as a 32-bit
integer. Older versions of postgresql defined XLogRecPtr as a
pair of two 32-bit integers.

xlogdefs.h.~REL9_1_0~:17
/*
 * Pointer to a location in the XLOG.  These pointers are 64 bits wide,
 * because we don't want them ever to overflow.
 *
 * NOTE: xrecoff == 0 is used to indicate an invalid pointer.  This is OK
 * because we use page headers in the XLOG, so no XLOG record can start
 * right at the beginning of a file.
 *
 * NOTE: the "log file number" is somewhat misnamed, since the actual files
 * making up the XLOG are much smaller than 4Gb.  Each actual file is an
 * XLogSegSize-byte "segment" of a logical log file having the indicated
 * xlogid.    The log file number and segment number together identify a
 * physical XLOG file.    Segment number and offset within the physical file
 * are computed from xrecoff div and mod XLogSegSize.
 */
typedef struct XLogRecPtr
{
    uint32        xlogid;            /* log file #, 0 based */
    uint32        xrecoff;        /* byte offset of location in log file */
} XLogRecPtr;


regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

On 2019-08-18 16:17, Luca Ferrari wrote:
> I'm just curious to better understand the naming convention behind wal
> files, because I've seen on a system of mine that the wals created
> were:
> 
> 000000050000020E000000FF
>  000000050000020F00000000
> 
> while I was expecting 20E0x100.

You are in principle correct.  This naming system is a historical
accident.  The actual LSN associated with the first file is

0000020EFF000000

and so the next one is naturally

0000020F00000000

The reason the zeroes are in there comes from a time when PostgreSQL
didn't fully support 64-bit integers, and the LSNs and the files were
tracked internally as pairs of 32-bit integers.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services