Re: Index bloat of 4x

I found this thread quite depressing because I had forgotten the VACUUM
FULL only reclaims totally empty pages.  I have applied the following
documentation patch to recommend periodic REINDEX, and backpatched to
8.2.X docs.  I also added some TODO items so hopefully at least we will
keep track of this limitation.

---------------------------------------------------------------------------

Ed L. wrote:
> On Thursday January 18 2007 6:07 am, Bill Moran wrote:
> > Right.  It doesn't _look_ that way from the graph, but that's
> > because I only graph total DB size.  I expect if I graphed
> > data and index size separately, it would be evident.
>
> pg_total_relation_size() might give you what you want there.
>
> > At this point, I'm going to assume that my question of, "Is
> > this 4x bloat strange enough to warrant further investigation"
> > is "no".  It seems like this amount of bloat isn't terribly
> > unusual, and that the people working on improving this sort of
> > thing already have enough examples of it.
>
> I afraid I don't see how any of the answers I saw discussed fit a
> 24x7 operation.  Reindex, drop index, vacuum full, ... they all
> block production queries of one sort or another for significant
> periods of time (minutes) on large (multi/tens of GB) tables,
> and thus are infeasible for true 24x7 operations.  What it seems
> we really need is something to remove the bloat without blocking
> production DML queries, while under significant query load, with
> very large tables.  This bloat issue is by far our biggest
> headache on the DB side.
>
> Ed
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster

--
  Bruce Momjian   bruce@momjian.us
  EnterpriseDB    http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/maintenance.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/maintenance.sgml,v
retrieving revision 1.66
diff -c -c -r1.66 maintenance.sgml
*** doc/src/sgml/maintenance.sgml    16 Jan 2007 18:26:02 -0000    1.66
--- doc/src/sgml/maintenance.sgml    31 Jan 2007 04:12:07 -0000
***************
*** 615,623 ****
     for inefficient use of space: if all but a few index keys on a page have
     been deleted, the page remains allocated.  So a usage pattern in which all
     but a few keys in each range are eventually deleted will see poor use of
!    space.  The potential for bloat is not indefinite — at worst there
!    will be one key per page — but it may still be worthwhile to schedule
!    periodic reindexing for indexes that have such usage patterns.
    </para>

    <para>
--- 615,621 ----
     for inefficient use of space: if all but a few index keys on a page have
     been deleted, the page remains allocated.  So a usage pattern in which all
     but a few keys in each range are eventually deleted will see poor use of
!    space.  For such usage patterns, periodic reindexing is recommended.
    </para>

    <para>
Index: doc/src/sgml/ref/vacuum.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/vacuum.sgml,v
retrieving revision 1.45
diff -c -c -r1.45 vacuum.sgml
*** doc/src/sgml/ref/vacuum.sgml    27 Dec 2006 14:55:17 -0000    1.45
--- doc/src/sgml/ref/vacuum.sgml    31 Jan 2007 04:12:07 -0000
***************
*** 167,172 ****
--- 167,175 ----
      most of the rows in a table and would like the table to physically shrink
      to occupy less disk space.  <command>VACUUM FULL</command> will usually
      shrink the table more than a plain <command>VACUUM</command> would.
+     The <option>FULL</option> option does not shrink indexes; a periodic
+     <command>REINDEX</> is still recommended.  In fact, it is often faster
+     to drop all indexes, <command>VACUUM FULL</>, and recreate the indexes.
     </para>

     <para>