pgCluu 3.0 has been published

Ubud, Bali - September 18th, 2019

**pgCluu 3.0 released**

PostgreSQL Cluster Utilization
----------------------------------------

pgCluu is a Perl program used to perform a full audit of a PostgreSQL
Cluster performances. It is divided in two parts, a collector used to
grab statistics on the PostgreSQL server using psql and sar, a reports
builder that will generate all HTML and charts output.

This release is a major release that adds lot of improvement and
new reports. It also improve performances and fully support a CGI
mode with dynamic content for realtime reports.

New or enhanced reports/features:

• The CGI mode to provide dynamic reports on time selection is now
  production ready and has left the beta stage. See documentation
  on how to enable it. Debian packaging enable this mode by default.
• Installation and configuration is fully managed from Makefile.PL
  script. This will help distribution packaging with additional
  configuration directives to control the behavior of pgcluu scripts
  with generic systemd service and timer files.
• Add incremental mode to pgcluu. That mean that you don't have to
  remove the content of the report directory anymore before running
  pgcluu again and when pgcluu detect that you are running in this
  mode. The incremental mode is detected from statistic directory
  when you have used the --rotate-daily or --rotate-hourly option
  to pgcluu_collectd.
• Rewrite Stats I/O reports to display read/hit per table or indexes
  during the audit period reported.
• Move Xlog (Wal bytes written) report under replication menu as
  data comes from pg_stat_replication.
• Explode System and Cluster reports into several reports dedicated
  files to limit the size of the HTML files in incremental mode.
• Add collect and report of vm.nr_overcommit_hugepages.
• Add report on pages scanned to report number of pages scanned by
  the kswapd daemon and number of pages scanned directly per second
  (pgscank/s and pgscand/s). It also reports the number of pages
  the system has reclaimed from cache (pagecache and swapcache) per
  second to satisfy its memory demands (pgsteal/s). On a second
  axis %vmeffto reports the the efficiency of page reclaim.
• All pages reports have been moved as sub menu of the Page menu and
  pgfree has been removed from the page fault report.
• Limit system cache statistics report to pgpgin/s and pgpgout/s and
  a new report is dedicated to Page fault statistics with majflt/s,
  minflt/s and pgfree/s dataset. Dataset minflt/s is calculated from
  sar output as result of "fault/s - majflt/s".
• All statistics about pg_stat_user_* and pg_stat_xact_user_* views
  are now stored as a snapshot at start and at end of the audit
  period and each hour if you are running in incremental mode. The
  old behavior was to append the snapshot at each running interval
  to the data file but this is not necessary and this can use huge
  disk space if you have lot of table and indexes in your database.
  Backward compatibility is preserved.
• New report of json versus jsonb columns in each database.
• Add keepalive kernel configuration in SysInfo report.
• Separate bgwriter "allocated" buffers in a dedicated report for
  better reading.
• Add new report about estimated memory workload based on the value
  of Committed_AS from /proc/meminfo. Committed_AS is the total
  amount of memory estimated to complete the workload. This value
  represents the worst case scenario value, and also includes swap
  memory. The report show the memory workload aver the time.

New command line options:

• Add option -x, --external-menu to pgcluu to save the common menu
  in menu.html and load it into each report using w3-include-html
  attribute from w3.js. This will only work if access to HTML report
  is through a Web server, not using the file:// protocol.
• Add new option --retention to pgcluu to set number of rolling days
  to keep in report directory. Default is to store indefinitely.
• Add option --retention NDAYS to pgcluu_collectd to set the number
  of rolling days to keep in data directory in incremental mode.
  Default is to store indefinitely.

New configuration directives :

• Add configuration directive STATS_REPORT_CACHING to pgcluu.conf
  file. This directive must be enabled in CGI mode. In this case the
  pgcluu script must be run periodically at least each 5 minutes to
  build and update cached statistics. In cache mode pgcluu generate
  binary file in the statistics directory and nothing in the report
  directory. The pgcluu.service and pgcluu.timer can be use, pgcluu
  will read the configuration file and switch automatically to cache
  mode if STATS_REPORT_CACHING is enabled.
• Add MAX_RENDERED_DAYS configuration directive to set the maximum
  number of days in a graph. Using default pgcluu_collectd 1 minute
  interval each, daily graph will have 1440 points. Having too much
  data to render is not a good point for performances with the
  current code. The user can set is own graph period, default is to
  limit to the first seven days of the selected period. This mean a
  maximum of 10080 points for one week, this seems a safe limit.

It also fixes some issues reported by users since last ten months.

-- 
Gilles Darold
http://www.darold.net/