Обсуждение: WIP patch for system-catalog vacuuming via a relation map
Attached is the current state of my work on letting system catalogs be processed by new-style VACUUM FULL (a/k/a CLUSTER). I haven't done the WAL support nor worried about interlocking concurrent updates of relation map files, but it passes the regression tests and can do VACUUM FULL of every system catalog. So I think this is proof that the approach is workable and there are no further gotchas to stumble over in dark recesses of the system. The patch adds about 1250 lines of code, mostly in a new file relmapper.c, and will be bigger once WAL support is added. However, I anticipate that we will be able to rip out probably three times that much code by eliminating VACUUM FULL INPLACE (commands/vacuum.c is alone almost 4000 lines, though of course it won't all go away). So it will be a substantial net reduction in code size, plus we will get the previously discussed benefits such as removing reindex-in-place for shared indexes. Barring objections I'm going to press ahead with completing and committing this; then in a separate patch remove VACUUM FULL INPLACE. regards, tom lane
Вложения
Tom Lane wrote: > Barring objections I'm going to press ahead with completing and > committing this; then in a separate patch remove VACUUM FULL INPLACE. With the second patch, we will continue to support reading XVAC_MOVED_OUT and IN hint bits, but never set them, correct? -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera <alvherre@commandprompt.com> writes: > Tom Lane wrote: >> Barring objections I'm going to press ahead with completing and >> committing this; then in a separate patch remove VACUUM FULL INPLACE. > With the second patch, we will continue to support reading > XVAC_MOVED_OUT and IN hint bits, but never set them, correct? Right, the tqual support has to stay for as long as we support binary upgrade from previous releases. (Unless we want to work out some way of ensuring no MOVED bits remain set in a database.) There is one heck of a lot of other infrastructure and complexity that can go away, though. regards, tom lane
> Barring objections I'm going to press ahead with completing and > committing this; then in a separate patch remove VACUUM FULL INPLACE. Was it our determination that we could remove VFI if we eliminated the system catalogs? I'm fine with it, I just thought some people had a marginal use case for VFI. --Josh Berkus
Josh Berkus <josh@agliodbs.com> writes: >> Barring objections I'm going to press ahead with completing and >> committing this; then in a separate patch remove VACUUM FULL INPLACE. > Was it our determination that we could remove VFI if we eliminated the > system catalogs? I'm fine with it, I just thought some people had a > marginal use case for VFI. I thought the consensus was to remove it if possible. There may still be some "marginal" use cases, but they don't justify the work that'd be needed to make it play safely with HS; let alone fixing the other longstanding gotchas with it, like the double-commit risk. regards, tom lane
Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > >> Barring objections I'm going to press ahead with completing and > >> committing this; then in a separate patch remove VACUUM FULL INPLACE. > > > Was it our determination that we could remove VFI if we eliminated the > > system catalogs? I'm fine with it, I just thought some people had a > > marginal use case for VFI. > > I thought the consensus was to remove it if possible. There may still > be some "marginal" use cases, but they don't justify the work that'd > be needed to make it play safely with HS; let alone fixing the other > longstanding gotchas with it, like the double-commit risk. Oh, yea, I remember, HS killed it because of the work required to make FULL work for 9.0. Thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes: > Tom Lane wrote: >> I thought the consensus was to remove it if possible. There may still >> be some "marginal" use cases, but they don't justify the work that'd >> be needed to make it play safely with HS; let alone fixing the other >> longstanding gotchas with it, like the double-commit risk. > I think part of the plan was to also provide an online reorg tool that > works by doing dummy UPDATEs, which means that you can get serialization > errors in serializable mode, but doesn't need to lock the table. Yeah. There's a good deal of interest in incremental/partial vacuuming. But that wouldn't make use of the existing VFI infrastructure either. regards, tom lane
Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: >>> Barring objections I'm going to press ahead with completing and >>> committing this; then in a separate patch remove VACUUM FULL INPLACE. > >> Was it our determination that we could remove VFI if we eliminated the >> system catalogs? I'm fine with it, I just thought some people had a >> marginal use case for VFI. > > I thought the consensus was to remove it if possible. There may still > be some "marginal" use cases, but they don't justify the work that'd > be needed to make it play safely with HS; let alone fixing the other > longstanding gotchas with it, like the double-commit risk. I think part of the plan was to also provide an online reorg tool that works by doing dummy UPDATEs, which means that you can get serialization errors in serializable mode, but doesn't need to lock the table. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com