Обсуждение: WAL documentation changes
Hi, the comment WRT WAL recovery and FS journals [1] is a bit misleading in it's current form. First, none of the general purpose filesystems I've seen so far do data journalling per default, since it's a huge performance penalty, even for non-RDBMS workloads. The feature you talk about is ext3 specific (and should be pointed out as such) and only disables write ordering, meaning that metadata and file content updates are not synchronized. best regards, Michael [1] 64b3d98baaf96afea815b0c37ff918f02fda11c9
Michael Renner wrote: > Hi, > > the comment WRT WAL recovery and FS journals [1] is a bit misleading in > it's current form. > > First, none of the general purpose filesystems I've seen so far do data > journalling per default, since it's a huge performance penalty, even for > non-RDBMS workloads. The feature you talk about is ext3 specific (and > should be pointed out as such) and only disables write ordering, meaning > that metadata and file content updates are not synchronized. You are right that my docs were misleading. I have improved them by mentioning that it is _data_ flush that as part of journalling that can be a problem, and documented that the mount option listed is ext3-specific, not linux-specific. Updated docs attached. Please let me know if I can improve it some more. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + Index: doc/src/sgml/wal.sgml =================================================================== RCS file: /cvsroot/pgsql/doc/src/sgml/wal.sgml,v retrieving revision 1.54 diff -c -c -r1.54 wal.sgml *** doc/src/sgml/wal.sgml 6 Dec 2008 21:34:27 -0000 1.54 --- doc/src/sgml/wal.sgml 10 Dec 2008 11:04:08 -0000 *************** *** 139,151 **** <para> Because <acronym>WAL</acronym> restores database file contents after a crash, it is not necessary to use a ! journaled filesystem; in fact, journaling overhead can ! reduce performance. For best performance, turn off ! <emphasis>data</emphasis> journaling as a filesystem mount ! option, e.g. use <literal>data=writeback</> on Linux. ! Meta-data journaling (e.g. file creation and directory ! modification) is still desirable for faster rebooting after ! a crash. </para> </tip> --- 139,151 ---- <para> Because <acronym>WAL</acronym> restores database file contents after a crash, it is not necessary to use a ! journaled filesystem for reliability. In fact, journaling ! overhead can reduce performance, especially if journaling ! causes file system <emphasis>data</emphasis> to be flushed ! to disk. Fortunately, data flushing during journaling can ! often be disabled with a filesystem mount option, e.g. ! <literal>data=writeback</> on a Linux ext3 file system. ! Journaled file systems do improve boot speed after a crash. </para> </tip>
>> First, none of the general purpose filesystems I've seen so far do data >> journalling per default, since it's a huge performance penalty, even for >> non-RDBMS workloads. The feature you talk about is ext3 specific (and >> should be pointed out as such) and only disables write ordering, meaning >> that metadata and file content updates are not synchronized. > > You are right that my docs were misleading. I have improved them by > mentioning that it is _data_ flush that as part of journalling that can > be a problem, and documented that the mount option listed is > ext3-specific, not linux-specific. Actually, I think that some of the other journalling filesystems allow data journalling (I know ReiserFS does), they just don't default to it. For that matter, a few (ZFS in particular) havedata journalling which can't be turned off. While it's not a tuning parameter, users should be warned that they'll take a performance hit from it. --Josh
Josh Berkus wrote: > > >> First, none of the general purpose filesystems I've seen so far do data > >> journalling per default, since it's a huge performance penalty, even for > >> non-RDBMS workloads. The feature you talk about is ext3 specific (and > >> should be pointed out as such) and only disables write ordering, meaning > >> that metadata and file content updates are not synchronized. > > > > You are right that my docs were misleading. I have improved them by > > mentioning that it is _data_ flush that as part of journalling that can > > be a problem, and documented that the mount option listed is > > ext3-specific, not linux-specific. > > Actually, I think that some of the other journalling filesystems allow > data journalling (I know ReiserFS does), they just don't default to it. > For that matter, a few (ZFS in particular) have data journalling which > can't be turned off. While it's not a tuning parameter, users should be > warned that they'll take a performance hit from it. So I assume you are saying the docs are fine now. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce, In your document change which one can be placed on non-journalling file system? data? wal? or both? For me it seems it's not clear. -- Tatsuo Ishii SRA OSS, Inc. Japan > Josh Berkus wrote: > > > > >> First, none of the general purpose filesystems I've seen so far do data > > >> journalling per default, since it's a huge performance penalty, even for > > >> non-RDBMS workloads. The feature you talk about is ext3 specific (and > > >> should be pointed out as such) and only disables write ordering, meaning > > >> that metadata and file content updates are not synchronized. > > > > > > You are right that my docs were misleading. I have improved them by > > > mentioning that it is _data_ flush that as part of journalling that can > > > be a problem, and documented that the mount option listed is > > > ext3-specific, not linux-specific. > > > > Actually, I think that some of the other journalling filesystems allow > > data journalling (I know ReiserFS does), they just don't default to it. > > For that matter, a few (ZFS in particular) have data journalling which > > can't be turned off. While it's not a tuning parameter, users should be > > warned that they'll take a performance hit from it. > > So I assume you are saying the docs are fine now. > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + If your life is a hard drive, Christ can be your backup. + > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers
Tatsuo Ishii wrote: > Bruce, > > In your document change which one can be placed on non-journalling > file system? data? wal? or both? Both. I have updated the docs to mention this, patch attached. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + Index: doc/src/sgml/wal.sgml =================================================================== RCS file: /cvsroot/pgsql/doc/src/sgml/wal.sgml,v retrieving revision 1.55 diff -c -c -r1.55 wal.sgml *** doc/src/sgml/wal.sgml 10 Dec 2008 11:05:49 -0000 1.55 --- doc/src/sgml/wal.sgml 18 Dec 2008 22:15:53 -0000 *************** *** 138,145 **** <tip> <para> Because <acronym>WAL</acronym> restores database file ! contents after a crash, it is not necessary to use a ! journaled filesystem for reliability. In fact, journaling overhead can reduce performance, especially if journaling causes file system <emphasis>data</emphasis> to be flushed to disk. Fortunately, data flushing during journaling can --- 138,145 ---- <tip> <para> Because <acronym>WAL</acronym> restores database file ! contents after a crash, journaled filesystems are necessary for ! reliable storage of the data files or WAL files. In fact, journaling overhead can reduce performance, especially if journaling causes file system <emphasis>data</emphasis> to be flushed to disk. Fortunately, data flushing during journaling can
>>> Bruce Momjian <bruce@momjian.us> wrote: > Tatsuo Ishii wrote: >> In your document change which one can be placed on non-journalling >> file system? data? wal? or both? > > Both. I have updated the docs to mention this, patch attached. Did you mean to say that journaled file systems are *not* necessary? -Kevin
Kevin Grittner wrote: > >>> Bruce Momjian <bruce@momjian.us> wrote: > > Tatsuo Ishii wrote: > >> In your document change which one can be placed on non-journalling > >> file system? data? wal? or both? > > > > Both. I have updated the docs to mention this, patch attached. > > Did you mean to say that journaled file systems are *not* necessary? Yes, not needed for database reliablity. The patch text was attached; was it unclear? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
>>> Bruce Momjian <bruce@momjian.us> wrote: > Kevin Grittner wrote: >> Did you mean to say that journaled file systems are *not* necessary? > > Yes, not needed for database reliablity. The patch text was attached; > was it unclear? I think you accidentally left out the word "not". -Kevin
Kevin Grittner wrote: > >>> Bruce Momjian <bruce@momjian.us> wrote: > > Kevin Grittner wrote: > >> Did you mean to say that journaled file systems are *not* > necessary? > > > > Yes, not needed for database reliablity. The patch text was > attached; > > was it unclear? > > I think you accidentally left out the word "not". Oops, right, added. Good catch. Warping that sentence into something that allowed the mention of WAL and data files was obviously too much for me. ;-) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +