Re: Protect syscache from bloating with negative cache entries
| От | Kyotaro HORIGUCHI |
|---|---|
| Тема | Re: Protect syscache from bloating with negative cache entries |
| Дата | |
| Msg-id | 20190206.144334.193118280.horiguchi.kyotaro@lab.ntt.co.jp обсуждение исходный текст |
| Ответ на | RE: Protect syscache from bloating with negative cache entries ("Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>) |
| Ответы |
Re: Protect syscache from bloating with negative cache entries
(Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>)
|
| Список | pgsql-hackers |
At Tue, 5 Feb 2019 02:40:35 +0000, "Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com> wrote in
<0A3221C70F24FB45833433255569204D1FB93A16@G01JPEXMBYT05>
> From: bruce@momjian.us [mailto:bruce@momjian.us]
> > On Mon, Feb 4, 2019 at 08:23:39AM +0000, Tsunakawa, Takayuki wrote:
> > > Horiguchi-san, Bruce, all, So, why don't we make
> > > syscache_memory_target the upper limit on the total size of all
> > > catcaches, and rethink the past LRU management?
> >
> > I was going to say that our experience with LRU has been that the
> > overhead is not worth the value, but that was in shared resource cases,
> > which this is not.
>
> That's good news! Then, let's proceed with the approach involving LRU, Horiguchi-san, Ideriha-san.
If you mean accessed-time-ordered list of entries by "LRU", I
still object to involve it since it is too complex in searching
code paths. Invalidation would make things more complex. The
current patch sorts entries by ct->lastaccess and discards
entries not accessed for more than threshold, only at doubling
cache capacity. It is already a kind of LRU in behavior.
This patch intends not to let caches bloat by unnecessary
entries, which is negative ones at first, then less-accessed ones
currently. If you mean by "LRU" something to put a hard limit on
the number or size of a catcache or all caches, it would be
doable by adding sort phase before pruning, like
CatCacheCleanOldEntriesByNum() in the attached as a PoC (first
attched) as food for discussion.
With the second attached script, we can observe what is happening
from another session by the following query.
select relname, size, ntuples, ageclass from pg_stat_syscache where relname =' pg_statistic'::regclass;
> pg_statistic | 1041024 | 7109 | {{1,1109},{3,0},{30,0},{60,0},{90,6000},{0,0
On the other hand, differently from the original pruning, this
happens irrelevantly to hash resize so it will causes another
observable intermittent slowdown than rehashing.
The two should have the same extent of impact on performance when
disabled. I'll take numbers briefly using pgbench.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
From 21f7b5528be03274dae9e58690c35cee9e68c82f Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
Date: Tue, 16 Oct 2018 13:04:30 +0900
Subject: [PATCH 1/4] Remove entries that haven't been used for a certain time
Catcache entries can be left alone for several reasons. It is not
desirable that they eat up memory. With this patch, This adds
consideration of removal of entries that haven't been used for a
certain time before enlarging the hash array.
---
doc/src/sgml/config.sgml | 38 ++++++
src/backend/access/transam/xact.c | 5 +
src/backend/utils/cache/catcache.c | 166 ++++++++++++++++++++++++--
src/backend/utils/misc/guc.c | 23 ++++
src/backend/utils/misc/postgresql.conf.sample | 2 +
src/include/utils/catcache.h | 28 ++++-
6 files changed, 254 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 9b7a7388d5..d0d2374944 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -1662,6 +1662,44 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-syscache-memory-target" xreflabel="syscache_memory_target">
+ <term><varname>syscache_memory_target</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>syscache_memory_target</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the maximum amount of memory to which syscache is expanded
+ without pruning. The value defaults to 0, indicating that pruning is
+ always considered. After exceeding this size, syscache pruning is
+ considered according to
+ <xref linkend="guc-syscache-prune-min-age"/>. If you need to keep
+ certain amount of syscache entries with intermittent usage, try
+ increase this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-syscache-prune-min-age" xreflabel="syscache_prune_min_age">
+ <term><varname>syscache_prune_min_age</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>syscache_prune_min_age</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the minimum amount of unused time in seconds at which a
+ syscache entry is considered to be removed. -1 indicates that syscache
+ pruning is disabled at all. The value defaults to 600 seconds
+ (<literal>10 minutes</literal>). The syscache entries that are not
+ used for the duration can be removed to prevent syscache bloat. This
+ behavior is suppressed until the size of syscache exceeds
+ <xref linkend="guc-syscache-memory-target"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-max-stack-depth" xreflabel="max_stack_depth">
<term><varname>max_stack_depth</varname> (<type>integer</type>)
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 92bda87804..ddc433c59e 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -734,7 +734,12 @@ void
SetCurrentStatementStartTimestamp(void)
{
if (!IsParallelWorker())
+ {
stmtStartTimestamp = GetCurrentTimestamp();
+
+ /* Set this timestamp as aproximated current time */
+ SetCatCacheClock(stmtStartTimestamp);
+ }
else
Assert(stmtStartTimestamp != 0);
}
diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c
index 258a1d64cc..2a996d740a 100644
--- a/src/backend/utils/cache/catcache.c
+++ b/src/backend/utils/cache/catcache.c
@@ -71,9 +71,24 @@
#define CACHE6_elog(a,b,c,d,e,f,g)
#endif
+/*
+ * GUC variable to define the minimum size of hash to cosider entry eviction.
+ * This variable is shared among various cache mechanisms.
+ */
+int cache_memory_target = 0;
+
+/* GUC variable to define the minimum age of entries that will be cosidered to
+ * be evicted in seconds. This variable is shared among various cache
+ * mechanisms.
+ */
+int cache_prune_min_age = 600;
+
/* Cache management header --- pointer is NULL until created */
static CatCacheHeader *CacheHdr = NULL;
+/* Timestamp used for any operation on caches. */
+TimestampTz catcacheclock = 0;
+
static inline HeapTuple SearchCatCacheInternal(CatCache *cache,
int nkeys,
Datum v1, Datum v2,
@@ -490,6 +505,7 @@ CatCacheRemoveCTup(CatCache *cache, CatCTup *ct)
CatCacheFreeKeys(cache->cc_tupdesc, cache->cc_nkeys,
cache->cc_keyno, ct->keys);
+ cache->cc_tupsize -= ct->size;
pfree(ct);
--cache->cc_ntup;
@@ -841,6 +857,7 @@ InitCatCache(int id,
cp->cc_nkeys = nkeys;
for (i = 0; i < nkeys; ++i)
cp->cc_keyno[i] = key[i];
+ cp->cc_tupsize = 0;
/*
* new cache is initialized as far as we can go for now. print some
@@ -858,9 +875,129 @@ InitCatCache(int id,
*/
MemoryContextSwitchTo(oldcxt);
+ /* initilize catcache reference clock if haven't done yet */
+ if (catcacheclock == 0)
+ catcacheclock = GetCurrentTimestamp();
+
return cp;
}
+/*
+ * CatCacheCleanupOldEntries - Remove infrequently-used entries
+ *
+ * Catcache entries can be left alone for several reasons. We remove them if
+ * they are not accessed for a certain time to prevent catcache from
+ * bloating. The eviction is performed with the similar algorithm with buffer
+ * eviction using access counter. Entries that are accessed several times can
+ * live longer than those that have had no access in the same duration.
+ */
+static bool
+CatCacheCleanupOldEntries(CatCache *cp)
+{
+ int i;
+ int nremoved = 0;
+ size_t hash_size;
+#ifdef CATCACHE_STATS
+ /* These variables are only for debugging purpose */
+ int ntotal = 0;
+ /*
+ * nth element in nentries stores the number of cache entries that have
+ * lived unaccessed for corresponding multiple in ageclass of
+ * cache_prune_min_age. The index of nremoved_entry is the value of the
+ * clock-sweep counter, which takes from 0 up to 2.
+ */
+ double ageclass[] = {0.05, 0.1, 1.0, 2.0, 3.0, 0.0};
+ int nentries[] = {0, 0, 0, 0, 0, 0};
+ int nremoved_entry[3] = {0, 0, 0};
+ int j;
+#endif
+
+ /* Return immediately if no pruning is wanted */
+ if (cache_prune_min_age < 0)
+ return false;
+
+ /*
+ * Return without pruning if the size of the hash is below the target.
+ */
+ hash_size = cp->cc_nbuckets * sizeof(dlist_head);
+ if (hash_size + cp->cc_tupsize < (Size) cache_memory_target * 1024L)
+ return false;
+
+ /* Search the whole hash for entries to remove */
+ for (i = 0; i < cp->cc_nbuckets; i++)
+ {
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &cp->cc_bucket[i])
+ {
+ CatCTup *ct = dlist_container(CatCTup, cache_elem, iter.cur);
+ long entry_age;
+ int us;
+
+
+ /*
+ * Calculate the duration from the time of the last access to the
+ * "current" time. Since catcacheclock is not advanced within a
+ * transaction, the entries that are accessed within the current
+ * transaction won't be pruned.
+ */
+ TimestampDifference(ct->lastaccess, catcacheclock, &entry_age, &us);
+
+#ifdef CATCACHE_STATS
+ /* count catcache entries for each age class */
+ ntotal++;
+ for (j = 0 ;
+ ageclass[j] != 0.0 &&
+ entry_age > cache_prune_min_age * ageclass[j] ;
+ j++);
+ if (ageclass[j] == 0.0) j--;
+ nentries[j]++;
+#endif
+
+ /*
+ * Try to remove entries older than cache_prune_min_age seconds.
+ * Entries that are not accessed after last pruning are removed in
+ * that seconds, and that has been accessed several times are
+ * removed after leaving alone for up to three times of the
+ * duration. We don't try shrink buckets since pruning effectively
+ * caps catcache expansion in the long term.
+ */
+ if (entry_age > cache_prune_min_age)
+ {
+#ifdef CATCACHE_STATS
+ Assert (ct->naccess >= 0 && ct->naccess <= 2);
+ nremoved_entry[ct->naccess]++;
+#endif
+ if (ct->naccess > 0)
+ ct->naccess--;
+ else
+ {
+ if (!ct->c_list || ct->c_list->refcount == 0)
+ {
+ CatCacheRemoveCTup(cp, ct);
+ nremoved++;
+ }
+ }
+ }
+ }
+ }
+
+#ifdef CATCACHE_STATS
+ ereport(DEBUG1,
+ (errmsg ("removed %d/%d, age(-%.0fs:%d, -%.0fs:%d, *-%.0fs:%d, -%.0fs:%d, -%.0fs:%d) naccessed(0:%d, 1:%d,
2:%d)",
+ nremoved, ntotal,
+ ageclass[0] * cache_prune_min_age, nentries[0],
+ ageclass[1] * cache_prune_min_age, nentries[1],
+ ageclass[2] * cache_prune_min_age, nentries[2],
+ ageclass[3] * cache_prune_min_age, nentries[3],
+ ageclass[4] * cache_prune_min_age, nentries[4],
+ nremoved_entry[0], nremoved_entry[1], nremoved_entry[2]),
+ errhidestmt(true)));
+#endif
+
+ return nremoved > 0;
+}
+
/*
* Enlarge a catcache, doubling the number of buckets.
*/
@@ -1274,6 +1411,11 @@ SearchCatCacheInternal(CatCache *cache,
*/
dlist_move_head(bucket, &ct->cache_elem);
+ /* Update access information for pruning */
+ if (ct->naccess < 2)
+ ct->naccess++;
+ ct->lastaccess = catcacheclock;
+
/*
* If it's a positive entry, bump its refcount and return it. If it's
* negative, we can report failure to the caller.
@@ -1819,11 +1961,13 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
CatCTup *ct;
HeapTuple dtp;
MemoryContext oldcxt;
+ int tupsize = 0;
/* negative entries have no tuple associated */
if (ntp)
{
int i;
+ int tupsize;
Assert(!negative);
@@ -1842,13 +1986,14 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
/* Allocate memory for CatCTup and the cached tuple in one go */
oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
- ct = (CatCTup *) palloc(sizeof(CatCTup) +
- MAXIMUM_ALIGNOF + dtp->t_len);
+ tupsize = sizeof(CatCTup) + MAXIMUM_ALIGNOF + dtp->t_len;
+ ct = (CatCTup *) palloc(tupsize);
ct->tuple.t_len = dtp->t_len;
ct->tuple.t_self = dtp->t_self;
ct->tuple.t_tableOid = dtp->t_tableOid;
ct->tuple.t_data = (HeapTupleHeader)
MAXALIGN(((char *) ct) + sizeof(CatCTup));
+ ct->size = tupsize;
/* copy tuple contents */
memcpy((char *) ct->tuple.t_data,
(const char *) dtp->t_data,
@@ -1876,8 +2021,8 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
{
Assert(negative);
oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
- ct = (CatCTup *) palloc(sizeof(CatCTup));
-
+ tupsize = sizeof(CatCTup);
+ ct = (CatCTup *) palloc(tupsize);
/*
* Store keys - they'll point into separately allocated memory if not
* by-value.
@@ -1898,17 +2043,24 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
ct->dead = false;
ct->negative = negative;
ct->hash_value = hashValue;
+ ct->naccess = 0;
+ ct->lastaccess = catcacheclock;
+ ct->size = tupsize;
dlist_push_head(&cache->cc_bucket[hashIndex], &ct->cache_elem);
cache->cc_ntup++;
CacheHdr->ch_ntup++;
+ cache->cc_tupsize += tupsize;
/*
- * If the hash table has become too full, enlarge the buckets array. Quite
- * arbitrarily, we enlarge when fill factor > 2.
+ * If the hash table has become too full, try cleanup by removing
+ * infrequently used entries to make a room for the new entry. If it
+ * failed, enlarge the bucket array instead. Quite arbitrarily, we try
+ * this when fill factor > 2.
*/
- if (cache->cc_ntup > cache->cc_nbuckets * 2)
+ if (cache->cc_ntup > cache->cc_nbuckets * 2 &&
+ !CatCacheCleanupOldEntries(cache))
RehashCatCache(cache);
return ct;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 8681ada33a..06c589f725 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -81,6 +81,7 @@
#include "tsearch/ts_cache.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
+#include "utils/catcache.h"
#include "utils/guc_tables.h"
#include "utils/float.h"
#include "utils/memutils.h"
@@ -2204,6 +2205,28 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"cache_memory_target", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the minimum syscache size to keep."),
+ gettext_noop("Cache is not pruned before exceeding this size."),
+ GUC_UNIT_KB
+ },
+ &cache_memory_target,
+ 0, 0, MAX_KILOBYTES,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"cache_prune_min_age", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the minimum unused duration of cache entries before removal."),
+ gettext_noop("Cache entries that live unused for longer than this seconds are considered to be
removed."),
+ GUC_UNIT_S
+ },
+ &cache_prune_min_age,
+ 600, -1, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/*
* We use the hopefully-safely-small value of 100kB as the compiled-in
* default for max_stack_depth. InitializeGUCOptions will increase it if
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index c7f53470df..108d332f2c 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -128,6 +128,8 @@
#work_mem = 4MB # min 64kB
#maintenance_work_mem = 64MB # min 1MB
#autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
+#cache_memory_target = 0kB # in kB
+#cache_prune_min_age = 600s # -1 disables pruning
#max_stack_depth = 2MB # min 100kB
#shared_memory_type = mmap # the default is the first option
# supported by the operating system:
diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h
index 65d816a583..5d24809900 100644
--- a/src/include/utils/catcache.h
+++ b/src/include/utils/catcache.h
@@ -22,6 +22,7 @@
#include "access/htup.h"
#include "access/skey.h"
+#include "datatype/timestamp.h"
#include "lib/ilist.h"
#include "utils/relcache.h"
@@ -61,6 +62,7 @@ typedef struct catcache
slist_node cc_next; /* list link */
ScanKeyData cc_skey[CATCACHE_MAXKEYS]; /* precomputed key info for heap
* scans */
+ int cc_tupsize; /* total amount of catcache tuples */
/*
* Keep these at the end, so that compiling catcache.c with CATCACHE_STATS
@@ -119,7 +121,9 @@ typedef struct catctup
bool dead; /* dead but not yet removed? */
bool negative; /* negative cache entry? */
HeapTupleData tuple; /* tuple management header */
-
+ int naccess; /* # of access to this entry, up to 2 */
+ TimestampTz lastaccess; /* approx. timestamp of the last usage */
+ int size; /* palloc'ed size off this tuple */
/*
* The tuple may also be a member of at most one CatCList. (If a single
* catcache is list-searched with varying numbers of keys, we may have to
@@ -189,6 +193,28 @@ typedef struct catcacheheader
/* this extern duplicates utils/memutils.h... */
extern PGDLLIMPORT MemoryContext CacheMemoryContext;
+/* for guc.c, not PGDLLPMPORT'ed */
+extern int cache_prune_min_age;
+extern int cache_memory_target;
+
+/* to use as access timestamp of catcache entries */
+extern TimestampTz catcacheclock;
+
+/*
+ * SetCatCacheClock - set timestamp for catcache access record
+ */
+static inline void
+SetCatCacheClock(TimestampTz ts)
+{
+ catcacheclock = ts;
+}
+
+static inline TimestampTz
+GetCatCacheClock(void)
+{
+ return catcacheclock;
+}
+
extern void CreateCacheMemoryContext(void);
extern CatCache *InitCatCache(int id, Oid reloid, Oid indexoid,
--
2.16.3
From 9f243e2fa6c6aaa5e333662f63c28c18ea72ed0f Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
Date: Tue, 16 Oct 2018 15:48:28 +0900
Subject: [PATCH 2/4] Syscache usage tracking feature.
Collects syscache usage statictics and show it using the view
pg_stat_syscache. The feature is controlled by the GUC variable
track_syscache_usage_interval.
---
doc/src/sgml/config.sgml | 15 ++
src/backend/catalog/system_views.sql | 17 +++
src/backend/postmaster/pgstat.c | 201 ++++++++++++++++++++++++--
src/backend/tcop/postgres.c | 23 +++
src/backend/utils/adt/pgstatfuncs.c | 134 +++++++++++++++++
src/backend/utils/cache/catcache.c | 115 +++++++++++----
src/backend/utils/cache/syscache.c | 24 +++
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 ++
src/backend/utils/misc/guc.c | 10 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 9 ++
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 6 +-
src/include/utils/catcache.h | 9 +-
src/include/utils/syscache.h | 19 +++
src/include/utils/timeout.h | 1 +
src/test/regress/expected/rules.out | 24 ++-
18 files changed, 576 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index d0d2374944..5ff3ebeb4e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -6687,6 +6687,21 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-track-syscache-usage-interval" xreflabel="track_syscache_usage_interval">
+ <term><varname>track_syscache_usage_interval</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>track_syscache_usage_interval</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the interval to collect system cache usage statistics in
+ milliseconds. This parameter is 0 by default, which means disabled.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-track-io-timing" xreflabel="track_io_timing">
<term><varname>track_io_timing</varname> (<type>boolean</type>)
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3e229c693c..f5d1aaf96f 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -906,6 +906,22 @@ CREATE VIEW pg_stat_progress_vacuum AS
FROM pg_stat_get_progress_info('VACUUM') AS S
LEFT JOIN pg_database D ON S.datid = D.oid;
+CREATE VIEW pg_stat_syscache AS
+ SELECT
+ S.pid AS pid,
+ S.relid::regclass AS relname,
+ S.indid::regclass AS cache_name,
+ S.size AS size,
+ S.ntup AS ntuples,
+ S.searches AS searches,
+ S.hits AS hits,
+ S.neg_hits AS neg_hits,
+ S.ageclass AS ageclass,
+ S.last_update AS last_update
+ FROM pg_stat_activity A
+ JOIN LATERAL (SELECT A.pid, * FROM pg_get_syscache_stats(A.pid)) S
+ ON (A.pid = S.pid);
+
CREATE VIEW pg_user_mappings AS
SELECT
U.oid AS umid,
@@ -1185,6 +1201,7 @@ GRANT EXECUTE ON FUNCTION pg_ls_waldir() TO pg_monitor;
GRANT EXECUTE ON FUNCTION pg_ls_archive_statusdir() TO pg_monitor;
GRANT EXECUTE ON FUNCTION pg_ls_tmpdir() TO pg_monitor;
GRANT EXECUTE ON FUNCTION pg_ls_tmpdir(oid) TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_get_syscache_stats(int) TO pg_monitor;
GRANT pg_read_all_settings TO pg_monitor;
GRANT pg_read_all_stats TO pg_monitor;
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 81c6499251..a1939958b7 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -66,6 +66,7 @@
#include "utils/ps_status.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/timestamp.h"
@@ -124,6 +125,7 @@
bool pgstat_track_activities = false;
bool pgstat_track_counts = false;
int pgstat_track_functions = TRACK_FUNC_OFF;
+int pgstat_track_syscache_usage_interval = 0;
int pgstat_track_activity_query_size = 1024;
/* ----------
@@ -236,6 +238,11 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
+/* bitmap symbols to specify target file types remove */
+#define PGSTAT_REMFILE_DBSTAT 1 /* remove only databsae stats files */
+#define PGSTAT_REMFILE_SYSCACHE 2 /* remove only syscache stats files */
+#define PGSTAT_REMFILE_ALL 3 /* remove both type of files */
+
/*
* Info about current "snapshot" of stats file
*/
@@ -335,6 +342,7 @@ static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
+static void pgstat_remove_syscache_statsfile(void);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
@@ -630,10 +638,13 @@ startup_failed:
}
/*
- * subroutine for pgstat_reset_all
+ * remove stats files
+ *
+ * clean up stats files in specified directory. target is one of
+ * PGSTAT_REFILE_DBSTAT/SYSCACHE/ALL and restricts files to remove.
*/
static void
-pgstat_reset_remove_files(const char *directory)
+pgstat_reset_remove_files(const char *directory, int target)
{
DIR *dir;
struct dirent *entry;
@@ -644,25 +655,39 @@ pgstat_reset_remove_files(const char *directory)
{
int nchars;
Oid tmp_oid;
+ int filetype = 0;
/*
* Skip directory entries that don't match the file names we write.
* See get_dbstat_filename for the database-specific pattern.
*/
if (strncmp(entry->d_name, "global.", 7) == 0)
+ {
+ filetype = PGSTAT_REMFILE_DBSTAT;
nchars = 7;
+ }
else
{
+ char head[2];
+
nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
+ (void) sscanf(entry->d_name, "%c%c_%u.%n",
+ head, head + 1, &tmp_oid, &nchars);
+
/* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
+ if (nchars < 3 || !isdigit(entry->d_name[3]))
continue;
+
+ if (strncmp(head, "db", 2) == 0)
+ filetype = PGSTAT_REMFILE_DBSTAT;
+ else if (strncmp(head, "cc", 2) == 0)
+ filetype = PGSTAT_REMFILE_SYSCACHE;
}
+ /* skip if this is not a target */
+ if ((filetype & target) == 0)
+ continue;
+
if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
strcmp(entry->d_name + nchars, "stat") != 0)
continue;
@@ -683,8 +708,9 @@ pgstat_reset_remove_files(const char *directory)
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
+ pgstat_reset_remove_files(pgstat_stat_directory, PGSTAT_REMFILE_ALL);
+ pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY,
+ PGSTAT_REMFILE_ALL);
}
#ifdef EXEC_BACKEND
@@ -2963,6 +2989,10 @@ pgstat_beshutdown_hook(int code, Datum arg)
if (OidIsValid(MyDatabaseId))
pgstat_report_stat(true);
+ /* clear syscache statistics files and temprary settings */
+ if (MyBackendId != InvalidBackendId)
+ pgstat_remove_syscache_statsfile();
+
/*
* Clear my status entry, following the protocol of bumping st_changecount
* before and after. We use a volatile pointer here to ensure the
@@ -4287,6 +4317,9 @@ PgstatCollectorMain(int argc, char *argv[])
pgStatRunningInCollector = true;
pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
+ /* Remove left-over syscache stats files */
+ pgstat_reset_remove_files(pgstat_stat_directory, PGSTAT_REMFILE_SYSCACHE);
+
/*
* Loop to process messages until we get SIGQUIT or detect ungraceful
* death of our parent postmaster.
@@ -6377,3 +6410,153 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/*
+ * return the filename for a syscache stat file; filename is the output
+ * buffer, of length len.
+ */
+void
+pgstat_get_syscachestat_filename(bool permanent, bool tempname, int backendid,
+ char *filename, int len)
+{
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/cc_%u.%s",
+ permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
+ pgstat_stat_directory,
+ backendid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
+}
+
+/* removes syscache stats files of this backend */
+static void
+pgstat_remove_syscache_statsfile(void)
+{
+ char fname[MAXPGPATH];
+
+ pgstat_get_syscachestat_filename(false, false, MyBackendId,
+ fname, MAXPGPATH);
+ unlink(fname); /* don't care of the result */
+}
+
+/*
+ * pgstat_write_syscache_stats() -
+ * Write the syscache statistics files.
+ *
+ * If 'force' is false, this function skips writing a file and resturns the
+ * time remaining in the current interval in milliseconds. If'force' is true,
+ * writes a file regardless of the remaining time and reset the interval.
+ */
+long
+pgstat_write_syscache_stats(bool force)
+{
+ static TimestampTz last_report = 0;
+ TimestampTz now;
+ long elapsed;
+ long secs;
+ int usecs;
+ int cacheId;
+ FILE *fpout;
+ char statfile[MAXPGPATH];
+ char tmpfile[MAXPGPATH];
+
+ /* Return if we don't want it */
+ if (!force && pgstat_track_syscache_usage_interval <= 0)
+ {
+ /* disabled. remove the statistics file if any */
+ if (last_report > 0)
+ {
+ last_report = 0;
+ pgstat_remove_syscache_statsfile();
+ }
+ return 0;
+ }
+
+ /* Check aginst the interval */
+ now = GetCurrentTransactionStopTimestamp();
+ TimestampDifference(last_report, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs / 1000;
+
+ if (!force && elapsed < pgstat_track_syscache_usage_interval)
+ {
+ /* not yet the time, inform the remaining time to the caller */
+ return pgstat_track_syscache_usage_interval - elapsed;
+ }
+
+ /* now update the stats */
+ last_report = now;
+
+ pgstat_get_syscachestat_filename(false, true,
+ MyBackendId, tmpfile, MAXPGPATH);
+ pgstat_get_syscachestat_filename(false, false,
+ MyBackendId, statfile, MAXPGPATH);
+
+ /*
+ * This function can be called from ProcessInterrupts(). Inhibit recursive
+ * interrupts to avoid recursive entry.
+ */
+ HOLD_INTERRUPTS();
+
+ fpout = AllocateFile(tmpfile, PG_BINARY_W);
+ if (fpout == NULL)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open temporary statistics file \"%s\": %m",
+ tmpfile)));
+ /*
+ * Failure writing this file is not critical. Just skip this time and
+ * tell caller to wait for the next interval.
+ */
+ RESUME_INTERRUPTS();
+ return pgstat_track_syscache_usage_interval;
+ }
+
+ /* write out every catcache stats */
+ for (cacheId = 0 ; cacheId < SysCacheSize ; cacheId++)
+ {
+ SysCacheStats *stats;
+
+ stats = SysCacheGetStats(cacheId);
+ Assert (stats);
+
+ /* write error is checked later using ferror() */
+ fputc('T', fpout);
+ (void)fwrite(&cacheId, sizeof(int), 1, fpout);
+ (void)fwrite(&last_report, sizeof(TimestampTz), 1, fpout);
+ (void)fwrite(stats, sizeof(*stats), 1, fpout);
+ }
+ fputc('E', fpout);
+
+ if (ferror(fpout))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write syscache statistics file \"%s\": %m",
+ tmpfile)));
+ FreeFile(fpout);
+ unlink(tmpfile);
+ }
+ else if (FreeFile(fpout) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close syscache statistics file \"%s\": %m",
+ tmpfile)));
+ unlink(tmpfile);
+ }
+ else if (rename(tmpfile, statfile) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not rename syscache statistics file \"%s\" to \"%s\": %m",
+ tmpfile, statfile)));
+ unlink(tmpfile);
+ }
+
+ RESUME_INTERRUPTS();
+ return 0;
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cfd507b2..fb77a0ce4c 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3157,6 +3157,12 @@ ProcessInterrupts(void)
}
+ if (IdleSyscacheStatsUpdateTimeoutPending)
+ {
+ IdleSyscacheStatsUpdateTimeoutPending = false;
+ pgstat_write_syscache_stats(true);
+ }
+
if (ParallelMessagePending)
HandleParallelMessages();
}
@@ -3733,6 +3739,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_catcache_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4173,9 +4180,19 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long timeout;
+
ProcessCompletedNotifies();
pgstat_report_stat(false);
+ timeout = pgstat_write_syscache_stats(false);
+
+ if (timeout > 0)
+ {
+ disable_idle_catcache_update_timeout = true;
+ enable_timeout_after(IDLE_CATCACHE_UPDATE_TIMEOUT,
+ timeout);
+ }
set_ps_display("idle", false);
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4218,6 +4235,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_catcache_update_timeout)
+ {
+ disable_timeout(IDLE_CATCACHE_UPDATE_TIMEOUT, false);
+ disable_idle_catcache_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b6ba856ebe..6526cfefb4 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -14,6 +14,8 @@
*/
#include "postgres.h"
+#include <sys/stat.h>
+
#include "access/htup_details.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_type.h"
@@ -28,6 +30,7 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/inet.h"
+#include "utils/syscache.h"
#include "utils/timestamp.h"
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -1899,3 +1902,134 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
PG_RETURN_DATUM(HeapTupleGetDatum(
heap_form_tuple(tupdesc, values, nulls)));
}
+
+Datum
+pgstat_get_syscache_stats(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SYSCACHE_SIZE 9
+ int pid = PG_GETARG_INT32(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ PgBackendStatus *beentry;
+ int beid;
+ char fname[MAXPGPATH];
+ FILE *fpin;
+ char c;
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("materialize mode required, but it is not " \
+ "allowed in this context")));
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* find beentry for given pid*/
+ beentry = NULL;
+ for (beid = 1;
+ (beentry = pgstat_fetch_stat_beentry(beid)) &&
+ beentry->st_procpid != pid ;
+ beid++);
+
+ /*
+ * we silently return empty result on failure or insufficient privileges
+ */
+ if (!beentry ||
+ (!has_privs_of_role(GetUserId(), beentry->st_userid) &&
+ !is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS)))
+ goto no_data;
+
+ pgstat_get_syscachestat_filename(false, false, beid, fname, MAXPGPATH);
+
+ if ((fpin = AllocateFile(fname, PG_BINARY_R)) == NULL)
+ {
+ if (errno != ENOENT)
+ ereport(WARNING,
+ (errcode_for_file_access(),
+ errmsg("could not open statistics file \"%s\": %m",
+ fname)));
+ /* also return empty on no statistics file */
+ goto no_data;
+ }
+
+ /* read the statistics file into tuplestore */
+ while ((c = fgetc(fpin)) == 'T')
+ {
+ TimestampTz last_update;
+ SysCacheStats stats;
+ int cacheid;
+ Datum values[PG_GET_SYSCACHE_SIZE];
+ bool nulls[PG_GET_SYSCACHE_SIZE] = {0};
+ Datum datums[SYSCACHE_STATS_NAGECLASSES * 2];
+ bool arrnulls[SYSCACHE_STATS_NAGECLASSES * 2] = {0};
+ int dims[] = {SYSCACHE_STATS_NAGECLASSES, 2};
+ int lbs[] = {1, 1};
+ ArrayType *arr;
+ int i, j;
+
+ fread(&cacheid, sizeof(int), 1, fpin);
+ fread(&last_update, sizeof(TimestampTz), 1, fpin);
+ if (fread(&stats, 1, sizeof(stats), fpin) != sizeof(stats))
+ {
+ ereport(WARNING,
+ (errmsg("corrupted syscache statistics file \"%s\"",
+ fname)));
+ goto no_data;
+ }
+
+ i = 0;
+ values[i++] = ObjectIdGetDatum(stats.reloid);
+ values[i++] = ObjectIdGetDatum(stats.indoid);
+ values[i++] = Int64GetDatum(stats.size);
+ values[i++] = Int64GetDatum(stats.ntuples);
+ values[i++] = Int64GetDatum(stats.nsearches);
+ values[i++] = Int64GetDatum(stats.nhits);
+ values[i++] = Int64GetDatum(stats.nneg_hits);
+
+ for (j = 0 ; j < SYSCACHE_STATS_NAGECLASSES ; j++)
+ {
+ datums[j * 2] = Int32GetDatum((int32) stats.ageclasses[j]);
+ datums[j * 2 + 1] = Int32GetDatum((int32) stats.nclass_entries[j]);
+ }
+
+ arr = construct_md_array(datums, arrnulls, 2, dims, lbs,
+ INT4OID, sizeof(int32), true, 'i');
+ values[i++] = PointerGetDatum(arr);
+
+ values[i++] = TimestampTzGetDatum(last_update);
+
+ Assert (i == PG_GET_SYSCACHE_SIZE);
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+
+ /* check for the end of file. abandon the result if file is broken */
+ if (c != 'E' || fgetc(fpin) != EOF)
+ tuplestore_clear(tupstore);
+
+ FreeFile(fpin);
+
+no_data:
+ tuplestore_donestoring(tupstore);
+ return (Datum) 0;
+}
diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c
index 2a996d740a..4ccda06795 100644
--- a/src/backend/utils/cache/catcache.c
+++ b/src/backend/utils/cache/catcache.c
@@ -89,6 +89,10 @@ static CatCacheHeader *CacheHdr = NULL;
/* Timestamp used for any operation on caches. */
TimestampTz catcacheclock = 0;
+/* age classes for pruning */
+static double ageclass[SYSCACHE_STATS_NAGECLASSES]
+ = {0.05, 0.1, 1.0, 2.0, 3.0, 0.0};
+
static inline HeapTuple SearchCatCacheInternal(CatCache *cache,
int nkeys,
Datum v1, Datum v2,
@@ -619,9 +623,7 @@ CatCacheInvalidate(CatCache *cache, uint32 hashValue)
else
CatCacheRemoveCTup(cache, ct);
CACHE1_elog(DEBUG2, "CatCacheInvalidate: invalidated");
-#ifdef CATCACHE_STATS
cache->cc_invals++;
-#endif
/* could be multiple matches, so keep looking! */
}
}
@@ -697,9 +699,7 @@ ResetCatalogCache(CatCache *cache)
}
else
CatCacheRemoveCTup(cache, ct);
-#ifdef CATCACHE_STATS
cache->cc_invals++;
-#endif
}
}
}
@@ -906,10 +906,11 @@ CatCacheCleanupOldEntries(CatCache *cp)
* cache_prune_min_age. The index of nremoved_entry is the value of the
* clock-sweep counter, which takes from 0 up to 2.
*/
- double ageclass[] = {0.05, 0.1, 1.0, 2.0, 3.0, 0.0};
- int nentries[] = {0, 0, 0, 0, 0, 0};
+ int nentries[SYSCACHE_STATS_NAGECLASSES] = {0, 0, 0, 0, 0, 0};
int nremoved_entry[3] = {0, 0, 0};
int j;
+
+ Assert(ageclass[SYSCACHE_STATS_NAGECLASSES - 1] == 0.0);
#endif
/* Return immediately if no pruning is wanted */
@@ -923,7 +924,11 @@ CatCacheCleanupOldEntries(CatCache *cp)
if (hash_size + cp->cc_tupsize < (Size) cache_memory_target * 1024L)
return false;
- /* Search the whole hash for entries to remove */
+ /*
+ * Search the whole hash for entries to remove. This is a quite time
+ * consuming task during catcache lookup, but accetable since now we are
+ * going to expand the hash table.
+ */
for (i = 0; i < cp->cc_nbuckets; i++)
{
dlist_mutable_iter iter;
@@ -936,21 +941,21 @@ CatCacheCleanupOldEntries(CatCache *cp)
/*
- * Calculate the duration from the time of the last access to the
- * "current" time. Since catcacheclock is not advanced within a
- * transaction, the entries that are accessed within the current
- * transaction won't be pruned.
+ * Calculate the duration from the time from the last access to
+ * the "current" time. Since catcacheclock is not advanced within
+ * a transaction, the entries that are accessed within the current
+ * transaction always get 0 as the result.
*/
TimestampDifference(ct->lastaccess, catcacheclock, &entry_age, &us);
#ifdef CATCACHE_STATS
/* count catcache entries for each age class */
ntotal++;
- for (j = 0 ;
- ageclass[j] != 0.0 &&
- entry_age > cache_prune_min_age * ageclass[j] ;
- j++);
- if (ageclass[j] == 0.0) j--;
+
+ j = 0;
+ while (j < SYSCACHE_STATS_NAGECLASSES - 1 &&
+ entry_age > cache_prune_min_age * ageclass[j])
+ j++;
nentries[j]++;
#endif
@@ -983,14 +988,17 @@ CatCacheCleanupOldEntries(CatCache *cp)
}
#ifdef CATCACHE_STATS
+ StaticAssertStmt(SYSCACHE_STATS_NAGECLASSES == 6,
+ "number of syscache age class must be 6");
ereport(DEBUG1,
- (errmsg ("removed %d/%d, age(-%.0fs:%d, -%.0fs:%d, *-%.0fs:%d, -%.0fs:%d, -%.0fs:%d) naccessed(0:%d, 1:%d,
2:%d)",
+ (errmsg ("removed %d/%d, age(-%.0fs:%d, -%.0fs:%d, *-%.0fs:%d, -%.0fs:%d, -%.0fs:%d, rest:%d)
naccessed(0:%d,1:%d, 2:%d)",
nremoved, ntotal,
ageclass[0] * cache_prune_min_age, nentries[0],
ageclass[1] * cache_prune_min_age, nentries[1],
ageclass[2] * cache_prune_min_age, nentries[2],
ageclass[3] * cache_prune_min_age, nentries[3],
ageclass[4] * cache_prune_min_age, nentries[4],
+ nentries[5],
nremoved_entry[0], nremoved_entry[1], nremoved_entry[2]),
errhidestmt(true)));
#endif
@@ -1367,9 +1375,7 @@ SearchCatCacheInternal(CatCache *cache,
if (unlikely(cache->cc_tupdesc == NULL))
CatalogCacheInitializeCache(cache);
-#ifdef CATCACHE_STATS
cache->cc_searches++;
-#endif
/* Initialize local parameter array */
arguments[0] = v1;
@@ -1429,9 +1435,7 @@ SearchCatCacheInternal(CatCache *cache,
CACHE3_elog(DEBUG2, "SearchCatCache(%s): found in bucket %d",
cache->cc_relname, hashIndex);
-#ifdef CATCACHE_STATS
cache->cc_hits++;
-#endif
return &ct->tuple;
}
@@ -1440,9 +1444,7 @@ SearchCatCacheInternal(CatCache *cache,
CACHE3_elog(DEBUG2, "SearchCatCache(%s): found neg entry in bucket %d",
cache->cc_relname, hashIndex);
-#ifdef CATCACHE_STATS
cache->cc_neg_hits++;
-#endif
return NULL;
}
@@ -1570,9 +1572,7 @@ SearchCatCacheMiss(CatCache *cache,
CACHE3_elog(DEBUG2, "SearchCatCache(%s): put in bucket %d",
cache->cc_relname, hashIndex);
-#ifdef CATCACHE_STATS
cache->cc_newloads++;
-#endif
return &ct->tuple;
}
@@ -1683,9 +1683,7 @@ SearchCatCacheList(CatCache *cache,
Assert(nkeys > 0 && nkeys < cache->cc_nkeys);
-#ifdef CATCACHE_STATS
cache->cc_lsearches++;
-#endif
/* Initialize local parameter array */
arguments[0] = v1;
@@ -1742,9 +1740,7 @@ SearchCatCacheList(CatCache *cache,
CACHE2_elog(DEBUG2, "SearchCatCacheList(%s): found list",
cache->cc_relname);
-#ifdef CATCACHE_STATS
cache->cc_lhits++;
-#endif
return cl;
}
@@ -2252,3 +2248,64 @@ PrintCatCacheListLeakWarning(CatCList *list)
list->my_cache->cc_relname, list->my_cache->id,
list, list->refcount);
}
+
+/*
+ * CatCacheGetStats - fill in SysCacheStats struct.
+ *
+ * This is a support routine for SysCacheGetStats, substantially fills in the
+ * result. The classification here is based on the same criteria to
+ * CatCacheCleanupOldEntries().
+ */
+void
+CatCacheGetStats(CatCache *cache, SysCacheStats *stats)
+{
+ int i, j;
+
+ Assert(ageclass[SYSCACHE_STATS_NAGECLASSES - 1] == 0.0);
+
+ /* fill in the stats struct */
+ stats->size = cache->cc_tupsize + cache->cc_nbuckets * sizeof(dlist_head);
+ stats->ntuples = cache->cc_ntup;
+ stats->nsearches = cache->cc_searches;
+ stats->nhits = cache->cc_hits;
+ stats->nneg_hits = cache->cc_neg_hits;
+
+ /* cache_prune_min_age can be changed on-session, fill it every time */
+ for (i = 0 ; i < SYSCACHE_STATS_NAGECLASSES ; i++)
+ stats->ageclasses[i] = (int) (cache_prune_min_age * ageclass[i]);
+
+ /*
+ * nth element in nclass_entries stores the number of cache entries that
+ * have lived unaccessed for corresponding multiple in ageclass of
+ * cache_prune_min_age.
+ */
+ memset(stats->nclass_entries, 0, sizeof(int) * SYSCACHE_STATS_NAGECLASSES);
+
+ /* Scan the whole hash */
+ for (i = 0; i < cache->cc_nbuckets; i++)
+ {
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &cache->cc_bucket[i])
+ {
+ CatCTup *ct = dlist_container(CatCTup, cache_elem, iter.cur);
+ long entry_age;
+ int us;
+
+ /*
+ * Calculate the duration from the time from the last access to
+ * the "current" time. Since catcacheclock is not advanced within
+ * a transaction, the entries that are accessed within the current
+ * transaction won't be pruned.
+ */
+ TimestampDifference(ct->lastaccess, catcacheclock, &entry_age, &us);
+
+ j = 0;
+ while (j < SYSCACHE_STATS_NAGECLASSES - 1 &&
+ entry_age > stats->ageclasses[j])
+ j++;
+
+ stats->nclass_entries[j]++;
+ }
+ }
+}
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index ac98c19155..7b38a06708 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -20,6 +20,9 @@
*/
#include "postgres.h"
+#include <sys/stat.h>
+#include <unistd.h>
+
#include "access/htup_details.h"
#include "access/sysattr.h"
#include "catalog/indexing.h"
@@ -1534,6 +1537,27 @@ RelationSupportsSysCache(Oid relid)
return false;
}
+/*
+ * SysCacheGetStats - returns stats of specified syscache
+ *
+ * This routine returns the address of its local static memory.
+ */
+SysCacheStats *
+SysCacheGetStats(int cacheId)
+{
+ static SysCacheStats stats;
+
+ Assert(cacheId >=0 && cacheId < SysCacheSize);
+
+ memset(&stats, 0, sizeof(stats));
+
+ stats.reloid = cacheinfo[cacheId].reloid;
+ stats.indoid = cacheinfo[cacheId].indoid;
+
+ CatCacheGetStats(SysCache[cacheId], &stats);
+
+ return &stats;
+}
/*
* OID comparator for pg_qsort
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index fd51934aaf..f039ecd805 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -32,6 +32,7 @@ volatile sig_atomic_t QueryCancelPending = false;
volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
+volatile sig_atomic_t IdleSyscacheStatsUpdateTimeoutPending = false;
volatile sig_atomic_t ConfigReloadPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index c0b6231458..dee7f19475 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -72,6 +72,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleSyscacheStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -628,6 +629,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_CATCACHE_UPDATE_TIMEOUT,
+ IdleSyscacheStatsUpdateTimeoutHandler);
}
/*
@@ -1239,6 +1242,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleSyscacheStatsUpdateTimeoutHandler(void)
+{
+ IdleSyscacheStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 06c589f725..32e41253a6 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -3168,6 +3168,16 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"track_syscache_usage_interval", PGC_SUSET, STATS_COLLECTOR,
+ gettext_noop("Sets the interval between syscache usage collection, in milliseconds. Zero disables syscache
usagetracking."),
+ NULL
+ },
+ &pgstat_track_syscache_usage_interval,
+ 0, 0, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"gin_pending_list_limit", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the maximum size of the pending list for GIN index."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 108d332f2c..4d4fb42251 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -560,6 +560,7 @@
#track_io_timing = off
#track_functions = none # none, pl, all
#track_activity_query_size = 1024 # (change requires restart)
+#track_syscache_usage_interval = 0 # zero disables tracking
#stats_temp_directory = 'pg_stat_tmp'
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b8de13f03b..6099a828d2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9669,6 +9669,15 @@
proargmodes => '{o,o,o,o,o,o,o,o,o,o,o}',
proargnames =>
'{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn}',
prosrc => 'pg_get_replication_slots' },
+{ oid => '3425',
+ descr => 'syscache statistics',
+ proname => 'pg_get_syscache_stats', prorows => '100', proisstrict => 'f',
+ proretset => 't', provolatile => 'v', prorettype => 'record',
+ proargtypes => 'int4',
+ proallargtypes => '{int4,oid,oid,int8,int8,int8,int8,int8,_int4,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,relid,indid,size,ntup,searches,hits,neg_hits,ageclass,last_update}',
+ prosrc => 'pgstat_get_syscache_stats' },
{ oid => '3786', descr => 'set up a logical replication slot',
proname => 'pg_create_logical_replication_slot', provolatile => 'v',
proparallel => 'u', prorettype => 'record', proargtypes => 'name name bool',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index c9e35003a5..69b9a976f0 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -82,6 +82,7 @@ extern PGDLLIMPORT volatile sig_atomic_t InterruptPending;
extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleSyscacheStatsUpdateTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 88a75fb798..b6bfd7d644 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1144,6 +1144,7 @@ extern bool pgstat_track_activities;
extern bool pgstat_track_counts;
extern int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern int pgstat_track_syscache_usage_interval;
extern char *pgstat_stat_directory;
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
@@ -1228,7 +1229,8 @@ extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
extern void pgstat_initstats(Relation rel);
extern char *pgstat_clip_activity(const char *raw_activity);
-
+extern void pgstat_get_syscachestat_filename(bool permanent,
+ bool tempname, int backendid, char *filename, int len);
/* ----------
* pgstat_report_wait_start() -
*
@@ -1363,5 +1365,5 @@ extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
-
+extern long pgstat_write_syscache_stats(bool force);
#endif /* PGSTAT_H */
diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h
index 5d24809900..4d51975920 100644
--- a/src/include/utils/catcache.h
+++ b/src/include/utils/catcache.h
@@ -65,10 +65,8 @@ typedef struct catcache
int cc_tupsize; /* total amount of catcache tuples */
/*
- * Keep these at the end, so that compiling catcache.c with CATCACHE_STATS
- * doesn't break ABI for other modules
+ * Statistics entries
*/
-#ifdef CATCACHE_STATS
long cc_searches; /* total # searches against this cache */
long cc_hits; /* # of matches against existing entry */
long cc_neg_hits; /* # of matches against negative entry */
@@ -81,7 +79,6 @@ typedef struct catcache
long cc_invals; /* # of entries invalidated from cache */
long cc_lsearches; /* total # list-searches */
long cc_lhits; /* # of matches against existing lists */
-#endif
} CatCache;
@@ -254,4 +251,8 @@ extern void PrepareToInvalidateCacheTuple(Relation relation,
extern void PrintCatCacheLeakWarning(HeapTuple tuple);
extern void PrintCatCacheListLeakWarning(CatCList *list);
+/* defined in syscache.h */
+typedef struct syscachestats SysCacheStats;
+extern void CatCacheGetStats(CatCache *cache, SysCacheStats *syscachestats);
+
#endif /* CATCACHE_H */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 95ee48954e..71b399c902 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -112,6 +112,24 @@ enum SysCacheIdentifier
#define SysCacheSize (USERMAPPINGUSERSERVER + 1)
};
+#define SYSCACHE_STATS_NAGECLASSES 6
+/* Struct for catcache tracking information */
+typedef struct syscachestats
+{
+ Oid reloid; /* target relation */
+ Oid indoid; /* index */
+ size_t size; /* size of the catcache */
+ int ntuples; /* number of tuples resides in the catcache */
+ int nsearches; /* number of searches */
+ int nhits; /* number of cache hits */
+ int nneg_hits; /* number of negative cache hits */
+ /* age classes in seconds */
+ int ageclasses[SYSCACHE_STATS_NAGECLASSES];
+ /* number of tuples fall into the corresponding age class */
+ int nclass_entries[SYSCACHE_STATS_NAGECLASSES];
+} SysCacheStats;
+
+
extern void InitCatalogCache(void);
extern void InitCatalogCachePhase2(void);
@@ -164,6 +182,7 @@ extern void SysCacheInvalidate(int cacheId, uint32 hashValue);
extern bool RelationInvalidatesSnapshotsOnly(Oid relid);
extern bool RelationHasSysCache(Oid relid);
extern bool RelationSupportsSysCache(Oid relid);
+extern SysCacheStats *SysCacheGetStats(int cacheId);
/*
* The use of the macros below rather than direct calls to the corresponding
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 9244a2a7b7..0ab441a364 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_CATCACHE_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2c8e21baa7..7bd77e9972 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1921,6 +1921,28 @@ pg_stat_sys_tables| SELECT pg_stat_all_tables.relid,
pg_stat_all_tables.autoanalyze_count
FROM pg_stat_all_tables
WHERE ((pg_stat_all_tables.schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR
(pg_stat_all_tables.schemaname~ '^pg_toast'::text));
+pg_stat_syscache| SELECT s.pid,
+ (s.relid)::regclass AS relname,
+ (s.indid)::regclass AS cache_name,
+ s.size,
+ s.ntup AS ntuples,
+ s.searches,
+ s.hits,
+ s.neg_hits,
+ s.ageclass,
+ s.last_update
+ FROM (pg_stat_activity a
+ JOIN LATERAL ( SELECT a.pid,
+ pg_get_syscache_stats.relid,
+ pg_get_syscache_stats.indid,
+ pg_get_syscache_stats.size,
+ pg_get_syscache_stats.ntup,
+ pg_get_syscache_stats.searches,
+ pg_get_syscache_stats.hits,
+ pg_get_syscache_stats.neg_hits,
+ pg_get_syscache_stats.ageclass,
+ pg_get_syscache_stats.last_update
+ FROM pg_get_syscache_stats(a.pid) pg_get_syscache_stats(relid, indid, size, ntup, searches, hits, neg_hits,
ageclass,last_update)) s ON ((a.pid = s.pid)));
pg_stat_user_functions| SELECT p.oid AS funcid,
n.nspname AS schemaname,
p.proname AS funcname,
@@ -2352,7 +2374,7 @@ pg_settings|pg_settings_n|CREATE RULE pg_settings_n AS
ON UPDATE TO pg_catalog.pg_settings DO INSTEAD NOTHING;
pg_settings|pg_settings_u|CREATE RULE pg_settings_u AS
ON UPDATE TO pg_catalog.pg_settings
- WHERE (new.name = old.name) DO SELECT set_config(old.name, new.setting, false) AS set_config;
+ WHERE (new.name = old.name) DO SELECT set_config(old.name, new.setting, false, false) AS set_config;
rtest_emp|rtest_emp_del|CREATE RULE rtest_emp_del AS
ON DELETE TO public.rtest_emp DO INSERT INTO rtest_emplog (ename, who, action, newsal, oldsal)
VALUES (old.ename, CURRENT_USER, 'fired'::bpchar, '$0.00'::money, old.salary);
--
2.16.3
From 5be729e44acf3f9c94dd9d13fa84cb4ae598406f Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
Date: Wed, 6 Feb 2019 14:36:29 +0900
Subject: [PATCH 3/4] PoC add prune-by-number-of-entries feature
Adds prune based on the number of cache entries on top of the current
pruning patch. It is controlled by two GUC variables.
cache_entry_limit: limit of the number of entries per catcache
cache_entry_limit_prune_ratio: how much of entries to remove at pruning
---
src/backend/utils/cache/catcache.c | 100 ++++++++++++++++++++++++++++++++++++-
src/backend/utils/misc/guc.c | 40 +++++++++++++++
src/include/utils/catcache.h | 2 +
3 files changed, 141 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c
index 4ccda06795..d15eac87d8 100644
--- a/src/backend/utils/cache/catcache.c
+++ b/src/backend/utils/cache/catcache.c
@@ -77,6 +77,11 @@
*/
int cache_memory_target = 0;
+
+/* PoC entry limit */
+int cache_entry_limit = 0;
+double cache_entry_limit_prune_ratio = 0.8;
+
/* GUC variable to define the minimum age of entries that will be cosidered to
* be evicted in seconds. This variable is shared among various cache
* mechanisms.
@@ -882,6 +887,95 @@ InitCatCache(int id,
return cp;
}
+/*
+ * CatCacheCleanupOldEntriesByNum -
+ * Poc remove infrequently-used entries by number of entries.
+ */
+static bool
+CatCacheCleanupOldEntriesByNum(CatCache *cp, int cache_entry_limit)
+{
+ int i;
+ int n;
+ int oldndelelem = cp->cc_ntup;
+ int ndelelem;
+ CatCTup **ct_array;
+
+ ndelelem = oldndelelem - (int)(cache_entry_limit * cache_entry_limit_prune_ratio);
+
+ /* lower limit: quite arbitrary */
+ if (ndelelem < 256)
+ ndelelem = 256;
+
+ /*
+ * partial sort array: [0] contains latest access entry
+ * [1] contains ealiest access entry
+ */
+ ct_array = (CatCTup **) palloc(ndelelem * sizeof(CatCTup*));
+ n = 0;
+
+ /*
+ * Collect entries to be removed, which have older lastaccess.
+ * Using heap bound sort like tuplesort.c.
+ */
+ for (i = 0; i < cp->cc_nbuckets; i++)
+ {
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &cp->cc_bucket[i])
+ {
+ CatCTup *ct = dlist_container(CatCTup, cache_elem, iter.cur);
+
+ if (n < ndelelem)
+ {
+ int j = n++;
+
+ while (j > 0)
+ {
+ int i = (j - 1) >> 1;
+
+ if (ct->lastaccess >= ct_array[i]->lastaccess)
+ break;
+ ct_array[j] = ct_array[i];
+ j = i;
+ }
+ ct_array[j] = ct;
+ }
+ else if (ct->lastaccess > ct_array[0]->lastaccess)
+ {
+ unsigned int i;
+
+ i = 0;
+
+ for (;;)
+ {
+ unsigned int j = 2 * i + 1;
+
+ if (j >= n)
+ break;
+ if (j + 1 < n &&
+ ct_array[j]->lastaccess > ct_array[j + 1]->lastaccess)
+ j++;
+ if (ct->lastaccess <= ct_array[j]->lastaccess)
+ break;
+ ct_array[i] = ct_array[j];
+ i = j;
+ }
+ ct_array[i] = ct;
+ }
+ }
+ }
+
+ /* Now we have the list of elements to be deleted */
+ for (i = 0 ; i < ndelelem ; i++)
+ CatCacheRemoveCTup(cp, ct_array[i]);
+
+ pfree(ct_array);
+
+ elog(LOG, "Catcache pruned by entry number: id=%d, %d => %d", cp->id, oldndelelem, cp->cc_ntup);
+
+ return true;
+}
+
/*
* CatCacheCleanupOldEntries - Remove infrequently-used entries
*
@@ -923,7 +1017,7 @@ CatCacheCleanupOldEntries(CatCache *cp)
hash_size = cp->cc_nbuckets * sizeof(dlist_head);
if (hash_size + cp->cc_tupsize < (Size) cache_memory_target * 1024L)
return false;
-
+
/*
* Search the whole hash for entries to remove. This is a quite time
* consuming task during catcache lookup, but accetable since now we are
@@ -2049,6 +2143,10 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
CacheHdr->ch_ntup++;
cache->cc_tupsize += tupsize;
+ /* cap number of entries */
+ if (cache_entry_limit > 0 && cache->cc_ntup > cache_entry_limit)
+ CatCacheCleanupOldEntriesByNum(cache, cache_entry_limit);
+
/*
* If the hash table has become too full, try cleanup by removing
* infrequently used entries to make a room for the new entry. If it
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 32e41253a6..7bb239a07e 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2227,6 +2227,36 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"cache_entry_limit", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the maximum entries of catcache."),
+ NULL
+ },
+ &cache_entry_limit,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"cache_entry_limit", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the maximum entries of catcache."),
+ NULL
+ },
+ &cache_entry_limit,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"cache_entry_limit", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the maximum entries of catcache."),
+ NULL
+ },
+ &cache_entry_limit,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/*
* We use the hopefully-safely-small value of 100kB as the compiled-in
* default for max_stack_depth. InitializeGUCOptions will increase it if
@@ -3401,6 +3431,16 @@ static struct config_real ConfigureNamesReal[] =
NULL, NULL, NULL
},
+ {
+ {"cache_entry_limit_prune_ratio", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the maximum entries of catcache."),
+ NULL
+ },
+ &cache_entry_limit_prune_ratio,
+ 0.8, 0.0, 1.0,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0.0, 0.0, 0.0, NULL, NULL, NULL
diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h
index 4d51975920..1f7fb51ac0 100644
--- a/src/include/utils/catcache.h
+++ b/src/include/utils/catcache.h
@@ -193,6 +193,8 @@ extern PGDLLIMPORT MemoryContext CacheMemoryContext;
/* for guc.c, not PGDLLPMPORT'ed */
extern int cache_prune_min_age;
extern int cache_memory_target;
+extern int cache_entry_limit;
+extern double cache_entry_limit_prune_ratio;
/* to use as access timestamp of catcache entries */
extern TimestampTz catcacheclock;
--
2.16.3
В списке pgsql-hackers по дате отправления:
Следующее
От: Kyotaro HORIGUCHIДата:
Сообщение: Re: Protect syscache from bloating with negative cache entries