Re: [HACKERS] increasing the default WAL segment size

Hello,

Please find attached an updated WIP patch. I have incorporated almost all comments. This is to be applied over Robert's patches. I will post performance results later on. 

1. shift (>>) and AND (&) operations: The assign hook of wal_segment_size sets the WalModMask and WalShiftBit. All the modulo and division operations using XLogSegSize has been replaced with these. However, there are many preprocessors which divide with XLogSegSize in xlog_internal.h. I have not changed these because it would mean I will have to reassign the WalShiftBit along with XLogSegSize in all the modules which use these macros. That does not seem to be a good idea. Also, this means shift operator can be used only in couple of places.

2. pg_standby: it deals with WAL files, so I have used the file size to set the XLogSegSize (similar to pg_xlogdump).  Also, macro MaxSegmentsPerLogFile using  XLOG_SEG_SIZE  is now defined in SetWALFileNameForCleanup where it is used. Since XLOG_SEG_SIZE is not preset, the code which throws an message if the file size is greater than XLOG_SEG_SIZE had to be removed.

3. XLOGChooseNumBuffers: This function, called during the creation of Shared Memory Segment, requires XLogSegSize which is set from the ControlFile. Hence temporarily read the ControlFile in XLOGShmemSize before invoking  XLOGChooseNumBuffer. The ControlFile is read again and stored on the Shared Memory later on. 

4. IsValidXLogSegSize: This is a macro to verify the XLogSegSize. This is used in initdb, pg_xlogdump, pg_standby. 

5. Macro for power2: There were couple of ideas to make it centralised.  For now, I have just defined it in xlog_internal.

6. Since CRC is used to verify the ControlFile before reading all the contents from it as is and I do not see the need to re-verify the xlog_seg_size.   

7. min/max_wal_size still take values in KB unit and internally store it as segment count. Though the calculation is now shifted to their respective assign hook as the GUC_UNIT_XSEGS had to be removed.

8. Declaring XLogSegSize: There are 2 internal variables for the same parameter. In original code XLOG_SEG_SIZE is defined in the auto-generated file src/include/pg_config.h. And xlog_internal.h defines:

#define XLogSegSize     ((uint32) XLOG_SEG_SIZE)

To avoid renaming all parts of code, I made the following change in xlog_internal.h

+ extern uint32 XLogSegSize;

+#define XLOG_SEG_SIZE XLogSegSize

 would it be better to just use one variable XLogSegSize everywhere. But few external modules could be using XLOG_SEG_SIZE. Thoughts?

9. Documentation will be added in next version of patch.--