28.2. The Cumulative Statistics System #
- 28.2.1. Statistics Collection Configuration
- 28.2.2. Viewing Statistics
- 28.2.3.
pg_stat_activity- 28.2.4.
pgpro_stat_wal_activity- 28.2.5.
pg_stat_replication- 28.2.6.
pg_stat_replication_slots- 28.2.7.
pg_stat_wal_receiver- 28.2.8.
pg_stat_recovery_prefetch- 28.2.9.
pg_stat_subscription- 28.2.10.
pg_stat_subscription_stats- 28.2.11.
pg_stat_ssl- 28.2.12.
pg_stat_gssapi- 28.2.13.
pg_stat_archiver- 28.2.14.
pg_stat_io- 28.2.15.
pg_stat_bgwriter- 28.2.16.
pg_stat_checkpointer- 28.2.17.
pg_stat_wal- 28.2.18.
pg_stat_database- 28.2.19.
pg_stat_database_conflicts- 28.2.20.
pg_stat_all_tables- 28.2.21.
pg_stat_all_indexes- 28.2.22.
pg_statio_all_tables- 28.2.23.
pg_statio_all_indexes- 28.2.24.
pg_statio_all_sequences- 28.2.25.
pg_stat_user_functions- 28.2.26.
pg_stat_slru- 28.2.27. Statistics Functions
- 28.2.2. Viewing Statistics
Postgres Pro's cumulative statistics system supports collection and reporting of information about server activity. Presently, accesses to tables and indexes in both disk-block and individual-row terms are counted. The total number of rows in each table, and information about vacuum and analyze actions for each table are also counted. If enabled, calls to user-defined functions and the total time spent in each one are counted as well.
Postgres Pro also supports reporting dynamic information about exactly what is going on in the system right now, such as the exact command currently being executed by other server processes, and which other connections exist in the system. This facility is independent of the cumulative statistics system.
28.2.1. Statistics Collection Configuration #
Since collection of statistics adds some overhead to query execution, the system can be configured to collect or not collect information. This is controlled by configuration parameters that are normally set in postgresql.conf. (See Chapter 19 for details about setting configuration parameters.)
The parameter track_activities enables monitoring of the current command being executed by any server process.
The parameter track_counts controls whether cumulative statistics are collected about table and index accesses.
The parameter track_functions enables tracking of usage of user-defined functions.
The parameter track_io_timing enables monitoring of block read, write, extend, and fsync times.
The parameter track_wal_io_timing enables monitoring of WAL write and fsync times.
Normally these parameters are set in postgresql.conf so that they apply to all server processes, but it is possible to turn them on or off in individual sessions using the SET command. (To prevent ordinary users from hiding their activity from the administrator, only superusers are allowed to change these parameters with SET.)
Cumulative statistics are collected in shared memory. Every Postgres Pro process collects statistics locally, then updates the shared data at appropriate intervals. When a server, including a physical replica, shuts down cleanly, a permanent copy of the statistics data is stored in the pg_stat subdirectory, so that statistics can be retained across server restarts. In contrast, when starting from an unclean shutdown (e.g., after an immediate shutdown, a server crash, starting from a base backup, and point-in-time recovery), all statistics counters are reset.
28.2.2. Viewing Statistics #
Several predefined views, listed in Table 28.1, are available to show the current state of the system. There are also several other views, listed in Table 28.2, available to show the accumulated statistics. Alternatively, one can build custom views using the underlying cumulative statistics functions, as discussed in Section 28.2.27.
When using the cumulative statistics views and functions to monitor collected data, it is important to realize that the information does not update instantaneously. Each individual server process flushes out accumulated statistics to shared memory just before going idle, but not more frequently than once per PGSTAT_MIN_INTERVAL milliseconds (1 second unless altered while building the server); so a query or transaction still in progress does not affect the displayed totals and the displayed information lags behind actual activity. However, current-query information collected by track_activities is always up-to-date.
Another important point is that when a server process is asked to display any of the accumulated statistics, accessed values are cached until the end of its current transaction in the default configuration. So the statistics will show static information as long as you continue the current transaction. Similarly, information about the current queries of all sessions is collected when any such information is first requested within a transaction, and the same information will be displayed throughout the transaction. This is a feature, not a bug, because it allows you to perform several queries on the statistics and correlate the results without worrying that the numbers are changing underneath you. When analyzing statistics interactively, or with expensive queries, the time delta between accesses to individual statistics can lead to significant skew in the cached statistics. To minimize skew, stats_fetch_consistency can be set to snapshot, at the price of increased memory usage for caching not-needed statistics data. Conversely, if it's known that statistics are only accessed once, caching accessed statistics is unnecessary and can be avoided by setting stats_fetch_consistency to none. You can invoke pg_stat_clear_snapshot() to discard the current transaction's statistics snapshot or cached values (if any). The next use of statistical information will (when in snapshot mode) cause a new snapshot to be built or (when in cache mode) accessed statistics to be cached.
A transaction can also see its own statistics (not yet flushed out to the shared memory statistics) in the views pg_stat_xact_all_tables, pg_stat_xact_sys_tables, pg_stat_xact_user_tables, and pg_stat_xact_user_functions. These numbers do not act as stated above; instead they update continuously throughout the transaction.
Some of the information in the dynamic statistics views shown in Table 28.1 is security restricted. Ordinary users can only see all the information about their own sessions (sessions belonging to a role that they are a member of). In rows about other sessions, many columns will be null. Note, however, that the existence of a session and its general properties such as its sessions user and database are visible to all users. Superusers and roles with privileges of built-in role pg_read_all_stats (see also Section 21.5) can see all the information about all sessions.
Table 28.1. Dynamic Statistics Views
| View Name | Description |
|---|---|
pg_stat_activity | One row per server process, showing information related to the current activity of that process, such as state and current query. See pg_stat_activity for details. |
pgpro_stat_wal_activity | One row per server process, showing the size of WAL files generated by this process, as well as all the data provided in the pg_stat_activity view. See pgpro_stat_wal_activity for details. |
pg_stat_replication | One row per WAL sender process, showing statistics about replication to that sender's connected standby server. See pg_stat_replication for details. |
pg_stat_wal_receiver | Only one row, showing statistics about the WAL receiver from that receiver's connected server. See pg_stat_wal_receiver for details. |
pg_stat_recovery_prefetch | Only one row, showing statistics about blocks prefetched during recovery. See pg_stat_recovery_prefetch for details. |
pg_stat_subscription | At least one row per subscription, showing information about the subscription workers. See pg_stat_subscription for details. |
pg_stat_ssl | One row per connection (regular and replication), showing information about SSL used on this connection. See pg_stat_ssl for details. |
pg_stat_gssapi | One row per connection (regular and replication), showing information about GSSAPI authentication and encryption used on this connection. See pg_stat_gssapi for details. |
pg_stat_progress_analyze | One row for each backend (including autovacuum worker processes) running ANALYZE, showing current progress. See Section 28.4.1. |
pg_stat_progress_create_index | One row for each backend running CREATE INDEX or REINDEX, showing current progress. See Section 28.4.4. |
pg_stat_progress_vacuum | One row for each backend (including autovacuum worker processes) running VACUUM, showing current progress. See Section 28.4.5. |
pg_stat_progress_cluster | One row for each backend running CLUSTER or VACUUM FULL, showing current progress. See Section 28.4.2. |
pg_stat_progress_basebackup | One row for each WAL sender process streaming a base backup, showing current progress. See Section 28.4.6. |
pg_stat_progress_copy | One row for each backend running COPY, showing current progress. See Section 28.4.3. |
Table 28.2. Collected Statistics Views
| View Name | Description |
|---|---|
pg_stat_archiver | One row only, showing statistics about the WAL archiver process's activity. See pg_stat_archiver for details. |
pg_stat_bgwriter | One row only, showing statistics about the background writer process's activity. See pg_stat_bgwriter for details. |
pg_stat_checkpointer | One row only, showing statistics about the checkpointer process's activity. See pg_stat_checkpointer for details. |
pg_stat_database | One row per database, showing database-wide statistics. See pg_stat_database for details. |
pg_stat_database_conflicts | One row per database, showing database-wide statistics about query cancels due to conflict with recovery on standby servers. See pg_stat_database_conflicts for details. |
pg_stat_io | One row for each combination of backend type, context, and target object containing cluster-wide I/O statistics. See pg_stat_io for details. |
pg_stat_replication_slots | One row per replication slot, showing statistics about the replication slot's usage. See pg_stat_replication_slots for details. |
pg_stat_slru | One row per SLRU, showing statistics of operations. See pg_stat_slru for details. |
pg_stat_subscription_stats | One row per subscription, showing statistics about errors. See pg_stat_subscription_stats for details. |
pg_stat_wal | One row only, showing statistics about WAL activity. See pg_stat_wal for details. |
pg_stat_all_tables | One row for each table in the current database, showing statistics about accesses to that specific table. See pg_stat_all_tables for details. |
pg_stat_sys_tables | Same as pg_stat_all_tables, except that only system tables are shown. |
pg_stat_user_tables | Same as pg_stat_all_tables, except that only user tables are shown. |
pg_stat_xact_all_tables | Similar to pg_stat_all_tables, but counts actions taken so far within the current transaction (which are not yet included in pg_stat_all_tables and related views). The columns for numbers of live and dead rows and vacuum and analyze actions are not present in this view. |
pg_stat_xact_sys_tables | Same as pg_stat_xact_all_tables, except that only system tables are shown. |
pg_stat_xact_user_tables | Same as pg_stat_xact_all_tables, except that only user tables are shown. |
pg_stat_all_indexes | One row for each index in the current database, showing statistics about accesses to that specific index. See pg_stat_all_indexes for details. |
pg_stat_sys_indexes | Same as pg_stat_all_indexes, except that only indexes on system tables are shown. |
pg_stat_user_indexes | Same as pg_stat_all_indexes, except that only indexes on user tables are shown. |
pg_stat_user_functions | One row for each tracked function, showing statistics about executions of that function. See pg_stat_user_functions for details. |
pg_stat_xact_user_functions | Similar to pg_stat_user_functions, but counts only calls during the current transaction (which are not yet included in pg_stat_user_functions). |
pg_statio_all_tables | One row for each table in the current database, showing statistics about I/O on that specific table. See pg_statio_all_tables for details. |
pg_statio_sys_tables | Same as pg_statio_all_tables, except that only system tables are shown. |
pg_statio_user_tables | Same as pg_statio_all_tables, except that only user tables are shown. |
pg_statio_all_indexes | One row for each index in the current database, showing statistics about I/O on that specific index. See pg_statio_all_indexes for details. |
pg_statio_sys_indexes | Same as pg_statio_all_indexes, except that only indexes on system tables are shown. |
pg_statio_user_indexes | Same as pg_statio_all_indexes, except that only indexes on user tables are shown. |
pg_statio_all_sequences | One row for each sequence in the current database, showing statistics about I/O on that specific sequence. See pg_statio_all_sequences for details. |
pg_statio_sys_sequences | Same as pg_statio_all_sequences, except that only system sequences are shown. (Presently, no system sequences are defined, so this view is always empty.) |
pg_statio_user_sequences | Same as pg_statio_all_sequences, except that only user sequences are shown. |
The per-index statistics are particularly useful to determine which indexes are being used and how effective they are.
The pg_stat_io and pg_statio_ set of views are useful for determining the effectiveness of the buffer cache. They can be used to calculate a cache hit ratio. Note that while Postgres Pro's I/O statistics capture most instances in which the kernel was invoked in order to perform I/O, they do not differentiate between data which had to be fetched from disk and that which already resided in the kernel page cache. Users are advised to use the Postgres Pro statistics views in combination with operating system utilities for a more complete picture of their database's I/O performance.
28.2.3. pg_stat_activity #
The pg_stat_activity view will have one row per server process, showing information related to the current activity of that process.
Table 28.3. pg_stat_activity View
Column Type Description |
|---|
OID of the database this backend is connected to |
Name of the database this backend is connected to |
Process ID of this backend |
Process ID of the parallel group leader if this process is a parallel query worker, or process ID of the leader apply worker if this process is a parallel apply worker. |
OID of the user logged into this backend |
Name of the user logged into this backend |
Name of the application that is connected to this backend |
IP address of the client connected to this backend. If this field is null, it indicates either that the client is connected via a Unix socket on the server machine or that this is an internal process such as autovacuum. |
Host name of the connected client, as reported by a reverse DNS lookup of |
TCP port number that the client is using for communication with this backend, or |
Time when this process was started. For client backends, this is the time the client connected to the server. |
Time when this process' current transaction was started, or null if no transaction is active. If the current query is the first of its transaction, this column is equal to the |
Time when the currently active query was started, or if |
Time when the |
The type of event for which the backend is waiting, if any; otherwise NULL. See Table 28.4. |
Wait event name if backend is currently waiting, otherwise NULL. See Table 28.5 through Table 28.13. |
Current overall state of this backend. Possible values are:
|
Top-level transaction identifier of this backend, if any; see Section 68.1. |
The current backend's |
Identifier of this backend's most recent query. If |
Text of this backend's most recent query. If |
Type of current backend. Possible types are |
Note
The wait_event and state columns are independent. If a backend is in the active state, it may or may not be waiting on some event. If the state is active and wait_event is non-null, it means that a query is being executed, but is being blocked somewhere in the system.
Table 28.4. Wait Event Types
| Wait Event Type | Description |
|---|---|
Activity | The server process is idle. This event type indicates a process waiting for activity in its main processing loop. wait_event will identify the specific wait point; see Table 28.5. |
BufferPin | The server process is waiting for exclusive access to a data buffer. Buffer pin waits can be protracted if another process holds an open cursor that last read data from the buffer in question. See Table 28.6. |
Client | The server process is waiting for activity on a socket connected to a user application. Thus, the server expects something to happen that is independent of its internal processes. wait_event will identify the specific wait point; see Table 28.7. |
Extension | The server process is waiting for some condition defined by an extension module. See Table 28.8. |
InjectionPoint | The server process is waiting for an injection point to reach an outcome defined in a test. See Section 40.10.13 for more details. This type has no predefined wait points. |
IO | The server process is waiting for an I/O operation to complete. wait_event will identify the specific wait point; see Table 28.9. |
IPC | The server process is waiting for some interaction with another server process. wait_event will identify the specific wait point; see Table 28.10. |
Lock | The server process is waiting for a heavyweight lock. Heavyweight locks, also known as lock manager locks or simply locks, primarily protect SQL-visible objects such as tables. However, they are also used to ensure mutual exclusion for certain internal operations such as relation extension. wait_event will identify the type of lock awaited; see Table 28.11. |
LWLock | The server process is waiting for a lightweight lock. Most such locks protect a particular data structure in shared memory. wait_event will contain a name identifying the purpose of the lightweight lock. (Some locks have specific names; others are part of a group of locks each with a similar purpose.) See Table 28.12. |
Timeout | The server process is waiting for a timeout to expire. wait_event will identify the specific wait point; see Table 28.13. |
Table 28.5. Wait Events of Type Activity
Activity Wait Event | Description |
|---|---|
ArchiverMain | Waiting in main loop of archiver process. |
AutovacuumMain | Waiting in main loop of autovacuum launcher process. |
BgfreezerMain | Waiting for background freezer to be enabled. |
BgwriterHibernate | Waiting in background writer process, hibernating. |
BgwriterMain | Waiting in main loop of background writer process. |
CfsGcEnable | Waiting for CFS garbage collection to be enabled. |
CheckpointerMain | Waiting in main loop of checkpointer process. |
LogicalApplyMain | Waiting in main loop of logical replication apply process. |
LogicalLauncherMain | Waiting in main loop of logical replication launcher process. |
LogicalParallelApplyMain | Waiting in main loop of logical replication parallel apply process. |
PriorityMain | Waiting in main loop of priority worker. |
RecoveryWalStream | Waiting in main loop of startup process for WAL to arrive, during streaming recovery. |
ReplicationRepairMain | Waiting in main loop of replication repair worker. |
ReplicationSlotsyncMain | Waiting in main loop of slot sync worker. |
ReplicationSlotsyncShutdown | Waiting for slot sync worker to shut down. |
SysloggerMain | Waiting in main loop of syslogger process. |
WalReceiverMain | Waiting in main loop of WAL receiver process. |
WalSenderMain | Waiting in main loop of WAL sender process. |
WalSummarizerWal | Waiting in WAL summarizer for more WAL to be generated. |
WalWriterMain | Waiting in main loop of WAL writer process. |
Table 28.6. Wait Events of Type Bufferpin
BufferPin Wait Event | Description |
|---|---|
BufferPin | Waiting to acquire an exclusive pin on a buffer. |
Table 28.7. Wait Events of Type Client
Client Wait Event | Description |
|---|---|
ClientRead | Waiting to read data from the client. |
ClientWrite | Waiting to write data to the client. |
GssOpenServer | Waiting to read data from the client while establishing a GSSAPI session. |
LibpqwalreceiverConnect | Waiting in WAL receiver to establish connection to remote server. |
LibpqwalreceiverReceive | Waiting in WAL receiver to receive data from remote server. |
SslOpenServer | Waiting for SSL while attempting connection. |
WaitForStandbyConfirmation | Waiting for WAL to be received and flushed by the physical standby. |
WalSenderWaitForWal | Waiting for WAL to be flushed in WAL sender process. |
WalSenderWriteData | Waiting for any activity when processing replies from WAL receiver in WAL sender process. |
Table 28.8. Wait Events of Type Extension
Extension Wait Event | Description |
|---|---|
Extension | Waiting in an extension. |
Table 28.9. Wait Events of Type Io
IO Wait Event | Description |
|---|---|
BasebackupRead | Waiting for base backup to read from a file. |
BasebackupSync | Waiting for data written by a base backup to reach durable storage. |
BasebackupWrite | Waiting for base backup to write to a file. |
BuffblockRepair | Waiting for repair of the corrupted block using replication protocol. |
BuffileRead | Waiting for a read from a buffered file. |
BuffileTruncate | Waiting for a buffered file to be truncated. |
BuffileWrite | Waiting for a write to a buffered file. |
ControlFileRead | Waiting for a read from the pg_control file. |
ControlFileSync | Waiting for the pg_control file to reach durable storage. |
ControlFileSyncUpdate | Waiting for an update to the pg_control file to reach durable storage. |
ControlFileWrite | Waiting for a write to the pg_control file. |
ControlFileWriteUpdate | Waiting for a write to update the pg_control file. |
CopyFileRead | Waiting for a read during a file copy operation. |
CopyFileWrite | Waiting for a write during a file copy operation. |
DataFileExtend | Waiting for a relation data file to be extended. |
DataFileFlush | Waiting for a relation data file to reach durable storage. |
DataFileImmediateSync | Waiting for an immediate synchronization of a relation data file to durable storage. |
DataFilePrefetch | Waiting for an asynchronous prefetch from a relation data file. |
DataFileRead | Waiting for a read from a relation data file. |
DataFileSync | Waiting for changes to a relation data file to reach durable storage. |
DataFileTruncate | Waiting for a relation data file to be truncated. |
DataFileWrite | Waiting for a write to a relation data file. |
DsmAllocate | Waiting for a dynamic shared memory segment to be allocated. |
DsmFillZeroWrite | Waiting to fill a dynamic shared memory backing file with zeroes. |
LockFileAddtodatadirRead | Waiting for a read while adding a line to the data directory lock file. |
LockFileAddtodatadirSync | Waiting for data to reach durable storage while adding a line to the data directory lock file. |
LockFileAddtodatadirWrite | Waiting for a write while adding a line to the data directory lock file. |
LockFileCreateRead | Waiting to read while creating the data directory lock file. |
LockFileCreateSync | Waiting for data to reach durable storage while creating the data directory lock file. |
LockFileCreateWrite | Waiting for a write while creating the data directory lock file. |
LockFileRecheckdatadirRead | Waiting for a read during recheck of the data directory lock file. |
LogicalRewriteCheckpointSync | Waiting for logical rewrite mappings to reach durable storage during a checkpoint. |
LogicalRewriteMappingSync | Waiting for mapping data to reach durable storage during a logical rewrite. |
LogicalRewriteMappingWrite | Waiting for a write of mapping data during a logical rewrite. |
LogicalRewriteSync | Waiting for logical rewrite mappings to reach durable storage. |
LogicalRewriteTruncate | Waiting for truncate of mapping data during a logical rewrite. |
LogicalRewriteWrite | Waiting for a write of logical rewrite mappings. |
RelationMapRead | Waiting for a read of the relation map file. |
RelationMapReplace | Waiting for durable replacement of a relation map file. |
RelationMapWrite | Waiting for a write to the relation map file. |
ReorderBufferRead | Waiting for a read during reorder buffer management. |
ReorderBufferWrite | Waiting for a write during reorder buffer management. |
ReorderLogicalMappingRead | Waiting for a read of a logical mapping during reorder buffer management. |
ReplicationSlotRead | Waiting for a read from a replication slot control file. |
ReplicationSlotRestoreSync | Waiting for a replication slot control file to reach durable storage while restoring it to memory. |
ReplicationSlotSync | Waiting for a replication slot control file to reach durable storage. |
ReplicationSlotWrite | Waiting for a write to a replication slot control file. |
SlruFlushSync | Waiting for SLRU data to reach durable storage during a checkpoint or database shutdown. |
SlruRead | Waiting for a read of an SLRU page. |
SlruSync | Waiting for SLRU data to reach durable storage following a page write. |
SlruWrite | Waiting for a write of an SLRU page. |
SnapbuildRead | Waiting for a read of a serialized historical catalog snapshot. |
SnapbuildSync | Waiting for a serialized historical catalog snapshot to reach durable storage. |
SnapbuildWrite | Waiting for a write of a serialized historical catalog snapshot. |
TimelineHistoryFileSync | Waiting for a timeline history file received via streaming replication to reach durable storage. |
TimelineHistoryFileWrite | Waiting for a write of a timeline history file received via streaming replication. |
TimelineHistoryRead | Waiting for a read of a timeline history file. |
TimelineHistorySync | Waiting for a newly created timeline history file to reach durable storage. |
TimelineHistoryWrite | Waiting for a write of a newly created timeline history file. |
TwophaseFileRead | Waiting for a read of a two phase state file. |
TwophaseFileSync | Waiting for a two phase state file to reach durable storage. |
TwophaseFileWrite | Waiting for a write of a two phase state file. |
VersionFileSync | Waiting for the version file to reach durable storage while creating a database. |
VersionFileWrite | Waiting for the version file to be written while creating a database. |
WalsenderTimelineHistoryRead | Waiting for a read from a timeline history file during a walsender timeline command. |
WalBootstrapSync | Waiting for WAL to reach durable storage during bootstrapping. |
WalBootstrapWrite | Waiting for a write of a WAL page during bootstrapping. |
WalCopyRead | Waiting for a read when creating a new WAL segment by copying an existing one. |
WalCopySync | Waiting for a new WAL segment created by copying an existing one to reach durable storage. |
WalCopyWrite | Waiting for a write when creating a new WAL segment by copying an existing one. |
WalInitSync | Waiting for a newly initialized WAL file to reach durable storage. |
WalInitWrite | Waiting for a write while initializing a new WAL file. |
WalRead | Waiting for a read from a WAL file. |
WalSummaryRead | Waiting for a read from a WAL summary file. |
WalSummaryWrite | Waiting for a write to a WAL summary file. |
WalSync | Waiting for a WAL file to reach durable storage. |
WalSyncMethodAssign | Waiting for data to reach durable storage while assigning a new WAL sync method. |
WalWrite | Waiting for a write to a WAL file. |
Table 28.10. Wait Events of Type Ipc
IPC Wait Event | Description |
|---|---|
AppendReady | Waiting for subplan nodes of an Append plan node to be ready. |
ArchiveCleanupCommand | Waiting for archive_cleanup_command to complete. |
ArchiveCommand | Waiting for archive_command to complete. |
B
|