Обсуждение: WAL Record Header Size Reduction
Current WAL Header uses 32 bytes on a 64-bit CPU. It seems possible to reduce this to 24 bytes, without reducing resilience, when full_page_writes = off. This will reduce overall WAL volumes by around 5-15%, depending upon the application with performance gains in various ways. If full_page_writes = off then it this is true xl_tot_len == xl_len + SizeOfXLogRecord since there are no backup blocks. As a result, there is no loss in resilience by removing this field. xlog.h shows this definition currently: typedef struct XLogRecord { pg_crc32 xl_crc; /* CRC for this record */ XLogRecPtr xl_prev; /* ptr to previous record in log */ TransactionId xl_xid; /* xact id */ uint32 xl_tot_len; /* total len of entire record */ uint32 xl_len; /* total len of rmgr data */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ /* Depending on MAXALIGN, there are either 2 or 6 wasted bytes here */ I propose to rearrange the XLogRecord structure to this: pg_crc32 xl_crc; /* CRC for this record */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ XLogRecPtr xl_prev; /* ptr to previous record in log */ TransactionId xl_xid; /* xact id */ uint32 xl_tot_len; /* total len of entire record */ which will occupy 24 bytes, saving 4 bytes on 32-bit and 8 bytes on 64-bit architectures, once alignment is considered. The xl_len field would be included only if backup blocks are included with the record. This is already marked by flags in the xl_info field, so no new flags are required. These changes can be mostly isolated to xlog.c, since only XLogInsert(), ReadRecord() and pg_resetxlog need to know about the changes. The xlog record would dynamically adjust according to whether backup blocks are present, so it can still work as full_page_writes is switched on/off by user or during the period between start/stop backup. The saving is only really relevant when full_page_writes = off, so I'm not worried about changing the xlrec header in that case anyway. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com
Simon Riggs wrote: > Current WAL Header uses 32 bytes on a 64-bit CPU. It seems possible to > reduce this to 24 bytes, without reducing resilience, when > full_page_writes = off. This will reduce overall WAL volumes by around > 5-15%, depending upon the application with performance gains in various > ways. Actually, it would help even when full_page_writes=on, because even then most xlog records don't have backup blocks attached to them. > xlog.h shows this definition currently: > > typedef struct XLogRecord > { > pg_crc32 xl_crc; /* CRC for this record */ > XLogRecPtr xl_prev; /* ptr to previous record in log */ > TransactionId xl_xid; /* xact id */ > uint32 xl_tot_len; /* total len of entire record */ > uint32 xl_len; /* total len of rmgr data */ > uint8 xl_info; /* flag bits, see below */ > RmgrId xl_rmid; /* resource manager for this record */ > > /* Depending on MAXALIGN, there are either 2 or 6 wasted bytes here */ > > I propose to rearrange the XLogRecord structure to this: I think you got your alignment wrong: > pg_crc32 xl_crc; /* CRC for this record */ > uint8 xl_info; /* flag bits, see below */ > RmgrId xl_rmid; /* resource manager for this record */ Because of xl_prev below which is two uint32 fields, there will be 2 bytes of wasted space in here. > XLogRecPtr xl_prev; /* ptr to previous record in log */ > TransactionId xl_xid; /* xact id */ > uint32 xl_tot_len; /* total len of entire record */ ISTM that we would get the effect your looking for by just moving the xl_tot_len field to the end, and only storing it for records with backup blocks: > pg_crc32 xl_crc; /* CRC for this record */> XLogRecPtr xl_prev; /* ptr to previous record in log */> TransactionId xl_xid;/* xact id */> uint32 xl_len; /* total len of rmgr data */> uint8 xl_info; /* flag bits, see below */> RmgrId xl_rmid;/* resource manager for this record */>> uint32 xl_tot_len; /* total len of entire record, if backup blocks indicated in xl_info*/ -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Thu, 2007-01-25 at 14:04 +0000, Heikki Linnakangas wrote: > Simon Riggs wrote: > > Current WAL Header uses 32 bytes on a 64-bit CPU. It seems possible to > > reduce this to 24 bytes, without reducing resilience, when > > full_page_writes = off. This will reduce overall WAL volumes by around > > 5-15%, depending upon the application with performance gains in various > > ways. > > Actually, it would help even when full_page_writes=on, because even then > most xlog records don't have backup blocks attached to them. Sure it would be active, but I meant it probably would not reduce the overall WAL volume by very much. We'll see, I guess. > > xlog.h shows this definition currently: > > > > typedef struct XLogRecord > > { > > pg_crc32 xl_crc; /* CRC for this record */ > > XLogRecPtr xl_prev; /* ptr to previous record in log */ > > TransactionId xl_xid; /* xact id */ > > uint32 xl_tot_len; /* total len of entire record */ > > uint32 xl_len; /* total len of rmgr data */ > > uint8 xl_info; /* flag bits, see below */ > > RmgrId xl_rmid; /* resource manager for this record */ > > > > /* Depending on MAXALIGN, there are either 2 or 6 wasted bytes here */ > > > > I propose to rearrange the XLogRecord structure to this: > > I think you got your alignment wrong: > > > pg_crc32 xl_crc; /* CRC for this record */ > > uint8 xl_info; /* flag bits, see below */ > > RmgrId xl_rmid; /* resource manager for this record */ > > Because of xl_prev below which is two uint32 fields, there will be 2 > bytes of wasted space in here. No escaping that though. > ISTM that we would get the effect your looking for by just moving the > xl_tot_len field to the end, and only storing it for records with backup > blocks: > > > pg_crc32 xl_crc; /* CRC for this record */ > > XLogRecPtr xl_prev; /* ptr to previous record in log */ > > TransactionId xl_xid; /* xact id */ > > uint32 xl_len; /* total len of rmgr data */ > > uint8 xl_info; /* flag bits, see below */ > > RmgrId xl_rmid; /* resource manager for this record */ > >> uint32 xl_tot_len; /* total len of entire record, if backup blocks > indicated in xl_info*/ OK. I was assuming XLogRecPtr was 8-byte aligned, but its a struct with 2 4-byte aligned elements, so I thought I had to move it to get the benefit. Seems not... -- Simon Riggs EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki@enterprisedb.com> writes: > ISTM that we would get the effect your looking for by just moving the > xl_tot_len field to the end, and only storing it for records with backup > blocks: I like that formulation better --- seems like less change-for-the-sake-of-change. regards, tom lane
On Thu, 2007-01-25 at 11:03 -0500, Tom Lane wrote: > Heikki Linnakangas <heikki@enterprisedb.com> writes: > > ISTM that we would get the effect your looking for by just moving the > > xl_tot_len field to the end, and only storing it for records with backup > > blocks: > > I like that formulation better --- seems like less > change-for-the-sake-of-change. Will do. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com