Re: [GENERAL] Strange case of database bloat

Hi;

First, I haven't seen major problems of database bloat in a long time which is why I find this case strange.  I wanted to ask here what may be causing it.

Problem:

==========

Database is in the 100GB to 200GB size range, running on btrfs (not my choice) with nodatacow enabled (which I set up to fix a performance issue).  The workload is a very heavy batch-update workload.

The database bloats linearly.  I have measured this on one  table (of 149M rows).

After vacuum full this table is (including indexes): 17GB

Every 24 hrs, seems to add its original space in size to the file system +/-.

Bloat seems to be affecting both indexes and underlying tables.

Vacuum verbose does not indicate a disproportionate number of rows being unremovable.  So autovacuum is keeping up without too much difficulty.

Troubleshooting so far

=======================

 filefrag finds a single extent on each file, so copy-on-write is not the culprit

Selecting the smallest 10 values of ctid from one of the bloating tables shows the first page used is around page 35 with one row per used page (and large gaps in between).

Questions

===========

I assume that it is the fact that rows update frequently which is the problem here? But why doesn't Postgres re-use any of the empty disk pages?

More importantly, is there anything that can be done to mitigate this issue other than a frequent vacuum full?

--

Best Wishes,

Chris Travers

Efficito:  Hosted Accounting and ERP.  Robust and Flexible.  No vendor lock-in.

http://www.efficito.com/learn_more