Обсуждение: Question on Postgres Index internals
How does Postgres detect corruption of index data files ? What is the exact mechanism by which it knows that the index is corrupt ? Can it happen that part of the index is corrupt but Postgres does not realize because those specific rows are not accessed (but other rows from that table are accessed)
Thanks for any inputs.
On 19/08/2011 5:46 AM, A J wrote:
Index corruption should not happen. If you are facing index corruption, either you have a hardware problem or you've hit a bug. The vast majority of cases are hardware faults.
Perhaps it'd help if you'd supply a little more detail about the background. Why you're asking this, and why you (presumably) suspect you have index corruption.
--
Craig Ringer
If their structure doesn't make sense, they're corrupt. It doesn't actively seek or detect corruption, because it shouldn't happen, and cannot be reliably discovered if it has.How does Postgres detect corruption of index data files ?
Numerous different ways, depending on the nature of the damage and the type of index. Often it won't know the index is damaged at all.What is the exact mechanism by which it knows that the index is corrupt ?
Yes, that is possible. It is also possible that a damaged index could cause issues like multiple instances of a value that's supposed to be unique, or a foreign key reference to a non-existent tuple.Can it happen that part of the index is corrupt but Postgres does not realize because those specific rows are not accessed (but other rows from that table are accessed)
Index corruption should not happen. If you are facing index corruption, either you have a hardware problem or you've hit a bug. The vast majority of cases are hardware faults.
Perhaps it'd help if you'd supply a little more detail about the background. Why you're asking this, and why you (presumably) suspect you have index corruption.
--
Craig Ringer
So I assume there is no redundant information stored such as checksum to validate the files against the stored checksums.
I don't have an active issue. Just trying to understand in detail how Postgres behaves under index corruption.
From: Craig Ringer <ringerc@ringerc.id.au>
To: A J <s5aly@yahoo.com>
Cc: PG Admin <pgsql-admin@postgresql.org>
Sent: Friday, August 19, 2011 2:40 AM
Subject: Re: [ADMIN] Question on Postgres Index internals
On 19/08/2011 5:46 AM, A J wrote:
Index corruption should not happen. If you are facing index corruption, either you have a hardware problem or you've hit a bug. The vast majority of cases are hardware faults.
Perhaps it'd help if you'd supply a little more detail about the background. Why you're asking this, and why you (presumably) suspect you have index corruption.
--
Craig Ringer
If their structure doesn't make sense, they're corrupt. It doesn't actively seek or detect corruption, because it shouldn't happen, and cannot be reliably discovered if it has.How does Postgres detect corruption of index data files ?
Numerous different ways, depending on the nature of the damage and the type of index. Often it won't know the index is damaged at all.What is the exact mechanism by which it knows that the index is corrupt ?
Yes, that is possible. It is also possible that a damaged index could cause issues like multiple instances of a value that's supposed to be unique, or a foreign key reference to a non-existent tuple.Can it happen that part of the index is corrupt but Postgres does not realize because those specific rows are not accessed (but other rows from that table are accessed)
Index corruption should not happen. If you are facing index corruption, either you have a hardware problem or you've hit a bug. The vast majority of cases are hardware faults.
Perhaps it'd help if you'd supply a little more detail about the background. Why you're asking this, and why you (presumably) suspect you have index corruption.
--
Craig Ringer
On 21/08/2011 11:07 PM, A J wrote:
Nope, there are no checksums.
It's the storage subsystem's job to make sure it doesn't mangle data on disk and reads the data as it was written. It's the OS's job to make sure nobody else overwrites or messes with that data while it's in RAM cache or on disk.
Index corruption means you've hit a hardware fault, OS bug, or possibly a PostgreSQL bug.
Only some kinds of corruption could be detected by application-level checksums. If the OS or storage subsystem just failed to write a page, Pg wouldn't be able to tell, for example. Checksums _would_ detect split page writes, single-bit faults, corrupted disk sectors and the like and could potentially be useful.
Like most forms of fault behaviour, it is undefined in the "C standard" sense of the meaning, though perhaps not quite as willing to just eat data ;-)
--
Craig Ringer
So I assume there is no redundant information stored such as checksum to validate the files against the stored checksums.
Nope, there are no checksums.
It's the storage subsystem's job to make sure it doesn't mangle data on disk and reads the data as it was written. It's the OS's job to make sure nobody else overwrites or messes with that data while it's in RAM cache or on disk.
Index corruption means you've hit a hardware fault, OS bug, or possibly a PostgreSQL bug.
Only some kinds of corruption could be detected by application-level checksums. If the OS or storage subsystem just failed to write a page, Pg wouldn't be able to tell, for example. Checksums _would_ detect split page writes, single-bit faults, corrupted disk sectors and the like and could potentially be useful.
I don't have an active issue. Just trying to understand in detail how Postgres behaves under index corruption.
Like most forms of fault behaviour, it is undefined in the "C standard" sense of the meaning, though perhaps not quite as willing to just eat data ;-)
--
Craig Ringer