Rethinking MemoryContext creation
| От | Tom Lane |
|---|---|
| Тема | Rethinking MemoryContext creation |
| Дата | |
| Msg-id | 2264.1512870796@sss.pgh.pa.us обсуждение исходный текст |
| Ответы |
Re: Rethinking MemoryContext creation
(Peter Geoghegan <pg@bowt.ie>)
Re: Rethinking MemoryContext creation (Tom Lane <tgl@sss.pgh.pa.us>) |
| Список | pgsql-hackers |
While fooling around with a different problem, I got annoyed at how slow
MemoryContext creation and deletion is. A big chunk of that seems to be
the palloc/pfree cycle needed to allocate the context header block in
TopMemoryContext. After thinking about it for a bit, I decided that
it'd be smarter to include the context header in the context's first
malloc() allocation, thereby eliminating the palloc/pfree cycle
completely. This is not completely cost-free, because it means that
we must allocate a context's first block ("keeper" block) immediately,
even if no pallocs ever get done in it. However, optimizing for the
unused-context case seems like the wrong thing.
To do this, we need to move the responsibility for allocating and
releasing the context header into the individual context-type-specific
modules, which leads to a certain amount of code duplication, but not a
lot. A big advantage is that we can simplify the overly-complicated
creation API that had the context module calling into mcxt.c which then
recursed back to the context module. Further down the pike there might
be some additional win, since this gives the context module more control
over where to put the header (shared memory maybe?).
Another point worth making is that getting the context headers out of
TopMemoryContext reduces the cost of transiently having a lot of
contexts. Since TopMemoryContext is never reset, and aset.c doesn't
give memory back to libc short of a reset, creating a lot of contexts
permanently bloats TopMemoryContext even if they all get deleted later.
I didn't spend a lot of effort on slab.c or generation.c, just having
them do a separate malloc and free for the context header. If anyone's
really excited about that, they could be improved later.
To cut to the chase: I experimented with simply creating and deleting an
aset.c memory context in a tight loop, optionally with one palloc in the
context before deleting it. On my machine, the time per loop looked like
HEAD with patch
no palloc at all 9.2 us 10.3 us
palloc 100 bytes 18.5 us 11.6 us
palloc 10000 bytes 17.2 us 18.5 us
So the unused-context case does get a bit slower, as we trade off
a malloc/free cycle to save a palloc/pfree cycle. The case where
we've made some use of the context gets considerably quicker though.
Also, the last line shows a situation where the lone palloc request
does not fit in the context's initial allocation so that we're forced
to make two malloc requests not one. This is a bit slower too, but
not by much, and I think it's really an unusual case. (As soon as
you've made any requests that do fit in the initial allocation,
you'd be back to the patch winning.)
Overall I'm seeing about a 5% improvement in a "pgbench -S" scenario,
although that number is a bit shaky since the run-to-run variation
is a few percent anyway.
Thoughts, objections? Anyone want to do some other benchmarking?
regards, tom lane
diff --git a/src/backend/utils/mmgr/README b/src/backend/utils/mmgr/README
index 296fa19..7f08f5d 100644
*** a/src/backend/utils/mmgr/README
--- b/src/backend/utils/mmgr/README
*************** every other context is a direct or indir
*** 177,184 ****
here is essentially the same as "malloc", because this context will never
be reset or deleted. This is for stuff that should live forever, or for
stuff that the controlling module will take care of deleting at the
! appropriate time. An example is fd.c's tables of open files, as well as
! the context management nodes for memory contexts themselves. Avoid
allocating stuff here unless really necessary, and especially avoid
running with CurrentMemoryContext pointing here.
--- 177,183 ----
here is essentially the same as "malloc", because this context will never
be reset or deleted. This is for stuff that should live forever, or for
stuff that the controlling module will take care of deleting at the
! appropriate time. An example is fd.c's tables of open files. Avoid
allocating stuff here unless really necessary, and especially avoid
running with CurrentMemoryContext pointing here.
*************** a maximum block size. Selecting smaller
*** 420,430 ****
space in contexts that aren't expected to hold very much (an example
is the relcache's per-relation contexts).
! Also, it is possible to specify a minimum context size. If this
! value is greater than zero then a block of that size will be grabbed
! immediately upon context creation, and cleared but not released during
! context resets. This feature is needed for ErrorContext (see above),
! but will most likely not be used for other contexts.
We expect that per-tuple contexts will be reset frequently and typically
will not allocate very much space per tuple cycle. To make this usage
--- 419,428 ----
space in contexts that aren't expected to hold very much (an example
is the relcache's per-relation contexts).
! Also, it is possible to specify a minimum context size, in case for
! some reason that should be larger than the initial block size.
! An aset.c context will always contain at least max(initBlockSize,
! minContextSize) space (less some header overhead).
We expect that per-tuple contexts will be reset frequently and typically
will not allocate very much space per tuple cycle. To make this usage
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index 1bd1c34..e0ef036 100644
*** a/src/backend/utils/mmgr/aset.c
--- b/src/backend/utils/mmgr/aset.c
*************** typedef void *AllocPointer;
*** 113,119 ****
*
* Note: header.isReset means there is nothing for AllocSetReset to do.
* This is different from the aset being physically empty (empty blocks list)
! * because we may still have a keeper block. It's also different from the set
* being logically empty, because we don't attempt to detect pfree'ing the
* last active chunk.
*/
--- 113,119 ----
*
* Note: header.isReset means there is nothing for AllocSetReset to do.
* This is different from the aset being physically empty (empty blocks list)
! * because we will still have a keeper block. It's also different from the set
* being logically empty, because we don't attempt to detect pfree'ing the
* last active chunk.
*/
*************** typedef struct AllocSetContext
*** 128,134 ****
Size maxBlockSize; /* maximum block size */
Size nextBlockSize; /* next block size to allocate */
Size allocChunkLimit; /* effective chunk size limit */
! AllocBlock keeper; /* if not NULL, keep this block over resets */
} AllocSetContext;
typedef AllocSetContext *AllocSet;
--- 128,134 ----
Size maxBlockSize; /* maximum block size */
Size nextBlockSize; /* next block size to allocate */
Size allocChunkLimit; /* effective chunk size limit */
! AllocBlock keeper; /* keep this block over resets */
} AllocSetContext;
typedef AllocSetContext *AllocSet;
*************** typedef struct AllocChunkData
*** 221,227 ****
static void *AllocSetAlloc(MemoryContext context, Size size);
static void AllocSetFree(MemoryContext context, void *pointer);
static void *AllocSetRealloc(MemoryContext context, void *pointer, Size size);
- static void AllocSetInit(MemoryContext context);
static void AllocSetReset(MemoryContext context);
static void AllocSetDelete(MemoryContext context);
static Size AllocSetGetChunkSpace(MemoryContext context, void *pointer);
--- 221,226 ----
*************** static void AllocSetCheck(MemoryContext
*** 236,246 ****
/*
* This is the virtual function table for AllocSet contexts.
*/
! static MemoryContextMethods AllocSetMethods = {
AllocSetAlloc,
AllocSetFree,
AllocSetRealloc,
- AllocSetInit,
AllocSetReset,
AllocSetDelete,
AllocSetGetChunkSpace,
--- 235,244 ----
/*
* This is the virtual function table for AllocSet contexts.
*/
! static const MemoryContextMethods AllocSetMethods = {
AllocSetAlloc,
AllocSetFree,
AllocSetRealloc,
AllocSetReset,
AllocSetDelete,
AllocSetGetChunkSpace,
*************** AllocSetContextCreate(MemoryContext pare
*** 345,351 ****
--- 343,352 ----
Size initBlockSize,
Size maxBlockSize)
{
+ Size headerSize;
+ Size firstBlockSize;
AllocSet set;
+ AllocBlock block;
/* Assert we padded AllocChunkData properly */
StaticAssertStmt(ALLOC_CHUNKHDRSZ == MAXALIGN(ALLOC_CHUNKHDRSZ),
*************** AllocSetContextCreate(MemoryContext pare
*** 370,387 ****
maxBlockSize);
if (minContextSize != 0 &&
(minContextSize != MAXALIGN(minContextSize) ||
! minContextSize <= ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ))
elog(ERROR, "invalid minContextSize for memory context: %zu",
minContextSize);
! /* Do the type-independent part of context creation */
! set = (AllocSet) MemoryContextCreate(T_AllocSetContext,
! sizeof(AllocSetContext),
! &AllocSetMethods,
! parent,
! name);
- /* Save allocation parameters */
set->initBlockSize = initBlockSize;
set->maxBlockSize = maxBlockSize;
set->nextBlockSize = initBlockSize;
--- 371,426 ----
maxBlockSize);
if (minContextSize != 0 &&
(minContextSize != MAXALIGN(minContextSize) ||
! minContextSize < initBlockSize ||
! minContextSize > maxBlockSize))
elog(ERROR, "invalid minContextSize for memory context: %zu",
minContextSize);
! /* Size of the memory context header, including name storage */
! headerSize = MAXALIGN(sizeof(AllocSetContext) + strlen(name) + 1);
!
! /* Determine size of initial block */
! firstBlockSize = headerSize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
! firstBlockSize = Max(firstBlockSize, initBlockSize);
! firstBlockSize = Max(firstBlockSize, minContextSize);
!
! /*
! * Allocate the initial block. Unlike other aset.c blocks, it starts with
! * the context header and its block header follows that.
! */
! set = (AllocSet) malloc(firstBlockSize);
! if (set == NULL)
! {
! MemoryContextStats(TopMemoryContext);
! ereport(ERROR,
! (errcode(ERRCODE_OUT_OF_MEMORY),
! errmsg("out of memory"),
! errdetail("Failed while creating memory context \"%s\".",
! name)));
! }
!
! /*
! * Avoid writing code that can fail between here and MemoryContextCreate;
! * we'd leak the initial block if we ereport in this stretch.
! */
!
! /* Fill in the initial block's block header */
! block = (AllocBlock) (((char *) set) + headerSize);
! block->aset = set;
! block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
! block->endptr = ((char *) set) + firstBlockSize;
! block->prev = NULL;
! block->next = NULL;
! set->blocks = block;
! /* Mark block as not to be released at reset time */
! set->keeper = block;
!
! /* Mark unallocated space NOACCESS; leave the block header alone. */
! VALGRIND_MAKE_MEM_NOACCESS(block->freeptr, block->endptr - block->freeptr);
!
! /* Finish filling in aset-specific parts of the context header */
! MemSetAligned(set->freelist, 0, sizeof(set->freelist));
set->initBlockSize = initBlockSize;
set->maxBlockSize = maxBlockSize;
set->nextBlockSize = initBlockSize;
*************** AllocSetContextCreate(MemoryContext pare
*** 410,483 ****
(Size) ((maxBlockSize - ALLOC_BLOCKHDRSZ) / ALLOC_CHUNK_FRACTION))
set->allocChunkLimit >>= 1;
! /*
! * Grab always-allocated space, if requested
! */
! if (minContextSize > 0)
! {
! Size blksize = minContextSize;
! AllocBlock block;
!
! block = (AllocBlock) malloc(blksize);
! if (block == NULL)
! {
! MemoryContextStats(TopMemoryContext);
! ereport(ERROR,
! (errcode(ERRCODE_OUT_OF_MEMORY),
! errmsg("out of memory"),
! errdetail("Failed while creating memory context \"%s\".",
! name)));
! }
! block->aset = set;
! block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
! block->endptr = ((char *) block) + blksize;
! block->prev = NULL;
! block->next = set->blocks;
! if (block->next)
! block->next->prev = block;
! set->blocks = block;
! /* Mark block as not to be released at reset time */
! set->keeper = block;
!
! /* Mark unallocated space NOACCESS; leave the block header alone. */
! VALGRIND_MAKE_MEM_NOACCESS(block->freeptr,
! blksize - ALLOC_BLOCKHDRSZ);
! }
return (MemoryContext) set;
}
/*
- * AllocSetInit
- * Context-type-specific initialization routine.
- *
- * This is called by MemoryContextCreate() after setting up the
- * generic MemoryContext fields and before linking the new context
- * into the context tree. We must do whatever is needed to make the
- * new context minimally valid for deletion. We must *not* risk
- * failure --- thus, for example, allocating more memory is not cool.
- * (AllocSetContextCreate can allocate memory when it gets control
- * back, however.)
- */
- static void
- AllocSetInit(MemoryContext context)
- {
- /*
- * Since MemoryContextCreate already zeroed the context node, we don't
- * have to do anything here: it's already OK.
- */
- }
-
- /*
* AllocSetReset
* Frees all memory which is allocated in the given set.
*
* Actually, this routine has some discretion about what to do.
* It should mark all allocated chunks freed, but it need not necessarily
* give back all the resources the set owns. Our actual implementation is
! * that we hang onto any "keeper" block specified for the set. In this way,
! * we don't thrash malloc() when a context is repeatedly reset after small
! * allocations, which is typical behavior for per-tuple contexts.
*/
static void
AllocSetReset(MemoryContext context)
--- 449,477 ----
(Size) ((maxBlockSize - ALLOC_BLOCKHDRSZ) / ALLOC_CHUNK_FRACTION))
set->allocChunkLimit >>= 1;
! /* Finally, do the type-independent part of context creation */
! MemoryContextCreate((MemoryContext) set,
! T_AllocSetContext,
! headerSize,
! sizeof(AllocSetContext),
! &AllocSetMethods,
! parent,
! name);
return (MemoryContext) set;
}
/*
* AllocSetReset
* Frees all memory which is allocated in the given set.
*
* Actually, this routine has some discretion about what to do.
* It should mark all allocated chunks freed, but it need not necessarily
* give back all the resources the set owns. Our actual implementation is
! * that we give back all but the "keeper" block (which we must keep, since
! * it also holds the context header). In this way, we don't thrash malloc()
! * when a context is repeatedly reset after small allocations, which is
! * typical behavior for per-tuple contexts.
*/
static void
AllocSetReset(MemoryContext context)
*************** AllocSetReset(MemoryContext context)
*** 497,503 ****
block = set->blocks;
! /* New blocks list is either empty or just the keeper block */
set->blocks = set->keeper;
while (block != NULL)
--- 491,497 ----
block = set->blocks;
! /* New blocks list will be just the keeper block */
set->blocks = set->keeper;
while (block != NULL)
*************** AllocSetReset(MemoryContext context)
*** 540,546 ****
* in preparation for deletion of the set.
*
* Unlike AllocSetReset, this *must* free all resources of the set.
- * But note we are not responsible for deleting the context node itself.
*/
static void
AllocSetDelete(MemoryContext context)
--- 534,539 ----
*************** AllocSetDelete(MemoryContext context)
*** 555,565 ****
AllocSetCheck(context);
#endif
! /* Make it look empty, just in case... */
! MemSetAligned(set->freelist, 0, sizeof(set->freelist));
! set->blocks = NULL;
! set->keeper = NULL;
!
while (block != NULL)
{
AllocBlock next = block->next;
--- 548,554 ----
AllocSetCheck(context);
#endif
! /* Free all blocks except the keeper */
while (block != NULL)
{
AllocBlock next = block->next;
*************** AllocSetDelete(MemoryContext context)
*** 567,575 ****
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
#endif
! free(block);
block = next;
}
}
/*
--- 556,571 ----
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
#endif
!
! /* Free the block, unless it's the keeper */
! if (block != set->keeper)
! free(block);
!
block = next;
}
+
+ /* Finally, free the context header + keeper block */
+ free(set);
}
/*
*************** AllocSetAlloc(MemoryContext context, Siz
*** 807,824 ****
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
block->endptr = ((char *) block) + blksize;
- /*
- * If this is the first block of the set, make it the "keeper" block.
- * Formerly, a keeper block could only be created during context
- * creation, but allowing it to happen here lets us have fast reset
- * cycling even for contexts created with minContextSize = 0; that way
- * we don't have to force space to be allocated in contexts that might
- * never need any space. Don't mark an oversize block as a keeper,
- * however.
- */
- if (set->keeper == NULL && blksize == set->initBlockSize)
- set->keeper = block;
-
/* Mark unallocated space NOACCESS. */
VALGRIND_MAKE_MEM_NOACCESS(block->freeptr,
blksize - ALLOC_BLOCKHDRSZ);
--- 803,808 ----
*************** AllocSetStats(MemoryContext context, int
*** 1205,1215 ****
AllocSet set = (AllocSet) context;
Size nblocks = 0;
Size freechunks = 0;
! Size totalspace = 0;
Size freespace = 0;
AllocBlock block;
int fidx;
for (block = set->blocks; block != NULL; block = block->next)
{
nblocks++;
--- 1189,1202 ----
AllocSet set = (AllocSet) context;
Size nblocks = 0;
Size freechunks = 0;
! Size totalspace;
Size freespace = 0;
AllocBlock block;
int fidx;
+ /* Include context header in totalspace */
+ totalspace = MAXALIGN(sizeof(AllocSetContext) + strlen(context->name) + 1);
+
for (block = set->blocks; block != NULL; block = block->next)
{
nblocks++;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 19390fa..ed9560a 100644
*** a/src/backend/utils/mmgr/generation.c
--- b/src/backend/utils/mmgr/generation.c
*************** struct GenerationChunk
*** 149,155 ****
static void *GenerationAlloc(MemoryContext context, Size size);
static void GenerationFree(MemoryContext context, void *pointer);
static void *GenerationRealloc(MemoryContext context, void *pointer, Size size);
- static void GenerationInit(MemoryContext context);
static void GenerationReset(MemoryContext context);
static void GenerationDelete(MemoryContext context);
static Size GenerationGetChunkSpace(MemoryContext context, void *pointer);
--- 149,154 ----
*************** static void GenerationCheck(MemoryContex
*** 164,174 ****
/*
* This is the virtual function table for Generation contexts.
*/
! static MemoryContextMethods GenerationMethods = {
GenerationAlloc,
GenerationFree,
GenerationRealloc,
- GenerationInit,
GenerationReset,
GenerationDelete,
GenerationGetChunkSpace,
--- 163,172 ----
/*
* This is the virtual function table for Generation contexts.
*/
! static const MemoryContextMethods GenerationMethods = {
GenerationAlloc,
GenerationFree,
GenerationRealloc,
GenerationReset,
GenerationDelete,
GenerationGetChunkSpace,
*************** GenerationContextCreate(MemoryContext pa
*** 210,215 ****
--- 208,214 ----
const char *name,
Size blockSize)
{
+ Size headerSize;
GenerationContext *set;
/* Assert we padded GenerationChunk properly */
*************** GenerationContextCreate(MemoryContext pa
*** 231,259 ****
elog(ERROR, "invalid blockSize for memory context: %zu",
blockSize);
! /* Do the type-independent part of context creation */
! set = (GenerationContext *) MemoryContextCreate(T_GenerationContext,
! sizeof(GenerationContext),
! &GenerationMethods,
! parent,
! name);
! set->blockSize = blockSize;
! return (MemoryContext) set;
! }
! /*
! * GenerationInit
! * Context-type-specific initialization routine.
! */
! static void
! GenerationInit(MemoryContext context)
! {
! GenerationContext *set = (GenerationContext *) context;
set->block = NULL;
dlist_init(&set->blocks);
}
/*
--- 230,275 ----
elog(ERROR, "invalid blockSize for memory context: %zu",
blockSize);
! /*
! * Allocate the context header. Unlike aset.c, we don't try to put this
! * into the first regular block, since that would prevent us from freeing
! * the first generation of allocations.
! */
! /* Size of the memory context header, including name storage */
! headerSize = sizeof(GenerationContext) + strlen(name) + 1;
! set = (GenerationContext *) malloc(headerSize);
! if (set == NULL)
! {
! MemoryContextStats(TopMemoryContext);
! ereport(ERROR,
! (errcode(ERRCODE_OUT_OF_MEMORY),
! errmsg("out of memory"),
! errdetail("Failed while creating memory context \"%s\".",
! name)));
! }
! /*
! * Avoid writing code that can fail between here and MemoryContextCreate;
! * we'd leak the header if we ereport in this stretch.
! */
+ /* Fill in GenerationContext-specific header fields */
+ set->blockSize = blockSize;
set->block = NULL;
dlist_init(&set->blocks);
+
+ /* Finally, do the type-independent part of context creation */
+ MemoryContextCreate((MemoryContext) set,
+ T_GenerationContext,
+ headerSize,
+ sizeof(GenerationContext),
+ &GenerationMethods,
+ parent,
+ name);
+
+ return (MemoryContext) set;
}
/*
*************** GenerationReset(MemoryContext context)
*** 296,311 ****
/*
* GenerationDelete
! * Frees all memory which is allocated in the given set, in preparation
! * for deletion of the set. We simply call GenerationReset() which does
! * all the dirty work.
*/
static void
GenerationDelete(MemoryContext context)
{
! /* just reset to release all the GenerationBlocks */
GenerationReset(context);
! /* we are not responsible for deleting the context node itself */
}
/*
--- 312,326 ----
/*
* GenerationDelete
! * Free all memory which is allocated in the given context.
*/
static void
GenerationDelete(MemoryContext context)
{
! /* Reset to release all the GenerationBlocks */
GenerationReset(context);
! /* And free the context header */
! free(context);
}
/*
*************** GenerationIsEmpty(MemoryContext context)
*** 659,665 ****
/*
* GenerationStats
! * Compute stats about memory consumption of an Generation.
*
* level: recursion level (0 at top level); used for print indentation.
* print: true to print stats to stderr.
--- 674,680 ----
/*
* GenerationStats
! * Compute stats about memory consumption of a Generation context.
*
* level: recursion level (0 at top level); used for print indentation.
* print: true to print stats to stderr.
*************** GenerationStats(MemoryContext context, i
*** 676,685 ****
Size nblocks = 0;
Size nchunks = 0;
Size nfreechunks = 0;
! Size totalspace = 0;
Size freespace = 0;
dlist_iter iter;
dlist_foreach(iter, &set->blocks)
{
GenerationBlock *block = dlist_container(GenerationBlock, node, iter.cur);
--- 691,703 ----
Size nblocks = 0;
Size nchunks = 0;
Size nfreechunks = 0;
! Size totalspace;
Size freespace = 0;
dlist_iter iter;
+ /* Include context header in totalspace */
+ totalspace = sizeof(GenerationContext) + strlen(context->name) + 1;
+
dlist_foreach(iter, &set->blocks)
{
GenerationBlock *block = dlist_container(GenerationBlock, node, iter.cur);
diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c
index c5c311f..d56e810 100644
*** a/src/backend/utils/mmgr/mcxt.c
--- b/src/backend/utils/mmgr/mcxt.c
*************** MemoryContextInit(void)
*** 91,99 ****
AssertState(TopMemoryContext == NULL);
/*
! * First, initialize TopMemoryContext, which will hold the MemoryContext
! * nodes for all other contexts. (There is special-case code in
! * MemoryContextCreate() to handle this call.)
*/
TopMemoryContext = AllocSetContextCreate((MemoryContext) NULL,
"TopMemoryContext",
--- 91,97 ----
AssertState(TopMemoryContext == NULL);
/*
! * First, initialize TopMemoryContext, which is the parent of all others.
*/
TopMemoryContext = AllocSetContextCreate((MemoryContext) NULL,
"TopMemoryContext",
*************** MemoryContextResetChildren(MemoryContext
*** 191,200 ****
* Delete a context and its descendants, and release all space
* allocated therein.
*
! * The type-specific delete routine removes all subsidiary storage
! * for the context, but we have to delete the context node itself,
! * as well as recurse to get the children. We must also delink the
! * node from its parent, if it has one.
*/
void
MemoryContextDelete(MemoryContext context)
--- 189,197 ----
* Delete a context and its descendants, and release all space
* allocated therein.
*
! * The type-specific delete routine removes all storage for the context,
! * but we have to recurse to handle the children.
! * We must also delink the context from its parent, if it has one.
*/
void
MemoryContextDelete(MemoryContext context)
*************** MemoryContextDelete(MemoryContext contex
*** 223,230 ****
MemoryContextSetParent(context, NULL);
context->methods->delete_context(context);
VALGRIND_DESTROY_MEMPOOL(context);
- pfree(context);
}
/*
--- 220,227 ----
MemoryContextSetParent(context, NULL);
context->methods->delete_context(context);
+
VALGRIND_DESTROY_MEMPOOL(context);
}
/*
*************** MemoryContextContains(MemoryContext cont
*** 587,686 ****
return ptr_context == context;
}
! /*--------------------
* MemoryContextCreate
* Context-type-independent part of context creation.
*
* This is only intended to be called by context-type-specific
* context creation routines, not by the unwashed masses.
*
! * The context creation procedure is a little bit tricky because
! * we want to be sure that we don't leave the context tree invalid
! * in case of failure (such as insufficient memory to allocate the
! * context node itself). The procedure goes like this:
! * 1. Context-type-specific routine first calls MemoryContextCreate(),
! * passing the appropriate tag/size/methods values (the methods
! * pointer will ordinarily point to statically allocated data).
! * The parent and name parameters usually come from the caller.
! * 2. MemoryContextCreate() attempts to allocate the context node,
! * plus space for the name. If this fails we can ereport() with no
! * damage done.
! * 3. We fill in all of the type-independent MemoryContext fields.
! * 4. We call the type-specific init routine (using the methods pointer).
! * The init routine is required to make the node minimally valid
! * with zero chance of failure --- it can't allocate more memory,
! * for example.
! * 5. Now we have a minimally valid node that can behave correctly
! * when told to reset or delete itself. We link the node to its
! * parent (if any), making the node part of the context tree.
! * 6. We return to the context-type-specific routine, which finishes
* up type-specific initialization. This routine can now do things
* that might fail (like allocate more memory), so long as it's
* sure the node is left in a state that delete will handle.
*
! * This protocol doesn't prevent us from leaking memory if step 6 fails
! * during creation of a top-level context, since there's no parent link
! * in that case. However, if you run out of memory while you're building
! * a top-level context, you might as well go home anyway...
! *
! * Normally, the context node and the name are allocated from
! * TopMemoryContext (NOT from the parent context, since the node must
! * survive resets of its parent context!). However, this routine is itself
! * used to create TopMemoryContext! If we see that TopMemoryContext is NULL,
! * we assume we are creating TopMemoryContext and use malloc() to allocate
! * the node.
*
! * Note that the name field of a MemoryContext does not point to
! * separately-allocated storage, so it should not be freed at context
! * deletion.
! *--------------------
*/
! MemoryContext
! MemoryContextCreate(NodeTag tag, Size size,
! MemoryContextMethods *methods,
MemoryContext parent,
const char *name)
{
! MemoryContext node;
! Size needed = size + strlen(name) + 1;
!
! /* creating new memory contexts is not allowed in a critical section */
Assert(CritSectionCount == 0);
! /* Get space for node and name */
! if (TopMemoryContext != NULL)
! {
! /* Normal case: allocate the node in TopMemoryContext */
! node = (MemoryContext) MemoryContextAlloc(TopMemoryContext,
! needed);
! }
! else
! {
! /* Special case for startup: use good ol' malloc */
! node = (MemoryContext) malloc(needed);
! Assert(node != NULL);
! }
! /* Initialize the node as best we can */
! MemSet(node, 0, size);
node->type = tag;
node->methods = methods;
! node->parent = NULL; /* for the moment */
node->firstchild = NULL;
node->prevchild = NULL;
! node->nextchild = NULL;
! node->isReset = true;
! node->name = ((char *) node) + size;
! strcpy(node->name, name);
! /* Type-specific routine finishes any other essential initialization */
! node->methods->init(node);
! /* OK to link node to parent (if any) */
! /* Could use MemoryContextSetParent here, but doesn't seem worthwhile */
if (parent)
{
- node->parent = parent;
node->nextchild = parent->firstchild;
if (parent->firstchild != NULL)
parent->firstchild->prevchild = node;
--- 584,654 ----
return ptr_context == context;
}
! /*
* MemoryContextCreate
* Context-type-independent part of context creation.
*
* This is only intended to be called by context-type-specific
* context creation routines, not by the unwashed masses.
*
! * The memory context creation procedure goes like this:
! * 1. Context-type-specific routine makes some initial space allocation,
! * including enough space for the context header. If it fails,
! * it can ereport() with no damage done.
! * 2. Context-type-specific routine sets up all type-specific fields of
! * the header (those beyond MemoryContextData proper), as well as any
! * other management fields it needs to have a fully valid context.
! * Usually, failure in this step is impossible, but if it's possible
! * the initial space allocation should be freed before ereport'ing.
! * 3. Context-type-specific routine calls MemoryContextCreate() to fill in
! * the generic header fields and link the context into the context tree.
! * 4. We return to the context-type-specific routine, which finishes
* up type-specific initialization. This routine can now do things
* that might fail (like allocate more memory), so long as it's
* sure the node is left in a state that delete will handle.
*
! * node: the as-yet-uninitialized common part of the context header node.
! * tag: NodeTag code identifying the memory context type.
! * size: total size of context header, including context-type-specific fields
! * as well as space for the context name.
! * nameoffset: where within the "size" space to insert the context name.
! * methods: context-type-specific methods (usually statically allocated).
! * parent: parent context, or NULL if this will be a top-level context.
! * name: name of context (for debugging only).
*
! * Context routines generally assume that MemoryContextCreate can't fail,
! * so this can contain Assert but not elog/ereport.
*/
! void
! MemoryContextCreate(MemoryContext node,
! NodeTag tag, Size size, Size nameoffset,
! const MemoryContextMethods *methods,
MemoryContext parent,
const char *name)
{
! /* Creating new memory contexts is not allowed in a critical section */
Assert(CritSectionCount == 0);
! /* Check size is sane */
! Assert(nameoffset >= sizeof(MemoryContextData));
! Assert(size >= nameoffset + strlen(name) + 1);
! /* Initialize all standard fields of memory context header */
node->type = tag;
+ node->isReset = true;
node->methods = methods;
! node->parent = parent;
node->firstchild = NULL;
node->prevchild = NULL;
! node->reset_cbs = NULL;
! /* Insert context name into space reserved for it */
! node->name = ((char *) node) + nameoffset;
! strcpy(node->name, name);
! /* OK to link node into context tree */
if (parent)
{
node->nextchild = parent->firstchild;
if (parent->firstchild != NULL)
parent->firstchild->prevchild = node;
*************** MemoryContextCreate(NodeTag tag, Size si
*** 688,698 ****
/* inherit allowInCritSection flag from parent */
node->allowInCritSection = parent->allowInCritSection;
}
VALGRIND_CREATE_MEMPOOL(node, 0, false);
-
- /* Return to type-specific creation routine to finish up */
- return node;
}
/*
--- 656,668 ----
/* inherit allowInCritSection flag from parent */
node->allowInCritSection = parent->allowInCritSection;
}
+ else
+ {
+ node->nextchild = NULL;
+ node->allowInCritSection = false;
+ }
VALGRIND_CREATE_MEMPOOL(node, 0, false);
}
/*
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index ee21752..3532e83 100644
*** a/src/backend/utils/mmgr/slab.c
--- b/src/backend/utils/mmgr/slab.c
*************** typedef struct SlabChunk
*** 126,132 ****
static void *SlabAlloc(MemoryContext context, Size size);
static void SlabFree(MemoryContext context, void *pointer);
static void *SlabRealloc(MemoryContext context, void *pointer, Size size);
- static void SlabInit(MemoryContext context);
static void SlabReset(MemoryContext context);
static void SlabDelete(MemoryContext context);
static Size SlabGetChunkSpace(MemoryContext context, void *pointer);
--- 126,131 ----
*************** static void SlabCheck(MemoryContext cont
*** 140,150 ****
/*
* This is the virtual function table for Slab contexts.
*/
! static MemoryContextMethods SlabMethods = {
SlabAlloc,
SlabFree,
SlabRealloc,
- SlabInit,
SlabReset,
SlabDelete,
SlabGetChunkSpace,
--- 139,148 ----
/*
* This is the virtual function table for Slab contexts.
*/
! static const MemoryContextMethods SlabMethods = {
SlabAlloc,
SlabFree,
SlabRealloc,
SlabReset,
SlabDelete,
SlabGetChunkSpace,
*************** SlabContextCreate(MemoryContext parent,
*** 194,200 ****
--- 192,201 ----
int chunksPerBlock;
Size fullChunkSize;
Size freelistSize;
+ Size nameOffset;
+ Size headerSize;
SlabContext *slab;
+ int i;
/* Assert we padded SlabChunk properly */
StaticAssertStmt(sizeof(SlabChunk) == MAXALIGN(sizeof(SlabChunk)),
*************** SlabContextCreate(MemoryContext parent,
*** 227,265 ****
/* make sure the chunks actually fit on the block */
Assert((fullChunkSize * chunksPerBlock) + sizeof(SlabBlock) <= blockSize);
! /* Do the type-independent part of context creation */
! slab = (SlabContext *)
! MemoryContextCreate(T_SlabContext,
! (offsetof(SlabContext, freelist) + freelistSize),
! &SlabMethods,
! parent,
! name);
! slab->blockSize = blockSize;
slab->chunkSize = chunkSize;
slab->fullChunkSize = fullChunkSize;
slab->chunksPerBlock = chunksPerBlock;
- slab->nblocks = 0;
slab->minFreeChunks = 0;
!
! return (MemoryContext) slab;
! }
!
! /*
! * SlabInit
! * Context-type-specific initialization routine.
! */
! static void
! SlabInit(MemoryContext context)
! {
! int i;
! SlabContext *slab = castNode(SlabContext, context);
!
! Assert(slab);
/* initialize the freelist slots */
for (i = 0; i < (slab->chunksPerBlock + 1); i++)
dlist_init(&slab->freelist[i]);
}
/*
--- 228,280 ----
/* make sure the chunks actually fit on the block */
Assert((fullChunkSize * chunksPerBlock) + sizeof(SlabBlock) <= blockSize);
! /*
! * Allocate the context header. Unlike aset.c, we don't try to put this
! * into the first regular block; not worth the extra complication.
! */
! /* Size of the memory context header, including name storage */
! nameOffset = offsetof(SlabContext, freelist) + freelistSize;
! headerSize = nameOffset + strlen(name) + 1;
!
! slab = (SlabContext *) malloc(headerSize);
! if (slab == NULL)
! {
! MemoryContextStats(TopMemoryContext);
! ereport(ERROR,
! (errcode(ERRCODE_OUT_OF_MEMORY),
! errmsg("out of memory"),
! errdetail("Failed while creating memory context \"%s\".",
! name)));
! }
!
! /*
! * Avoid writing code that can fail between here and MemoryContextCreate;
! * we'd leak the header if we ereport in this stretch.
! */
!
! /* Fill in SlabContext-specific header fields */
slab->chunkSize = chunkSize;
slab->fullChunkSize = fullChunkSize;
+ slab->blockSize = blockSize;
slab->chunksPerBlock = chunksPerBlock;
slab->minFreeChunks = 0;
! slab->nblocks = 0;
/* initialize the freelist slots */
for (i = 0; i < (slab->chunksPerBlock + 1); i++)
dlist_init(&slab->freelist[i]);
+
+ /* Finally, do the type-independent part of context creation */
+ MemoryContextCreate((MemoryContext) slab,
+ T_SlabContext,
+ headerSize,
+ nameOffset,
+ &SlabMethods,
+ parent,
+ name);
+
+ return (MemoryContext) slab;
}
/*
*************** SlabReset(MemoryContext context)
*** 308,321 ****
/*
* SlabDelete
! * Frees all memory which is allocated in the given slab, in preparation
! * for deletion of the slab. We simply call SlabReset().
*/
static void
SlabDelete(MemoryContext context)
{
! /* just reset the context */
SlabReset(context);
}
/*
--- 323,337 ----
/*
* SlabDelete
! * Free all memory which is allocated in the given context.
*/
static void
SlabDelete(MemoryContext context)
{
! /* Reset to release all the SlabBlocks */
SlabReset(context);
+ /* And free the context header */
+ free(context);
}
/*
*************** SlabIsEmpty(MemoryContext context)
*** 613,619 ****
/*
* SlabStats
! * Compute stats about memory consumption of an Slab.
*
* level: recursion level (0 at top level); used for print indentation.
* print: true to print stats to stderr.
--- 629,635 ----
/*
* SlabStats
! * Compute stats about memory consumption of a Slab context.
*
* level: recursion level (0 at top level); used for print indentation.
* print: true to print stats to stderr.
*************** SlabStats(MemoryContext context, int lev
*** 626,636 ****
SlabContext *slab = castNode(SlabContext, context);
Size nblocks = 0;
Size freechunks = 0;
! Size totalspace = 0;
Size freespace = 0;
int i;
! Assert(slab);
for (i = 0; i <= slab->chunksPerBlock; i++)
{
--- 642,655 ----
SlabContext *slab = castNode(SlabContext, context);
Size nblocks = 0;
Size freechunks = 0;
! Size totalspace;
Size freespace = 0;
int i;
! /* Include context header in totalspace */
! totalspace = offsetof(SlabContext, freelist)
! + sizeof(dlist_head) * (slab->chunksPerBlock + 1)
! + strlen(context->name) + 1;
for (i = 0; i <= slab->chunksPerBlock; i++)
{
diff --git a/src/include/nodes/memnodes.h b/src/include/nodes/memnodes.h
index e22d9fb..f49d55e 100644
*** a/src/include/nodes/memnodes.h
--- b/src/include/nodes/memnodes.h
*************** typedef struct MemoryContextMethods
*** 57,63 ****
/* call this free_p in case someone #define's free() */
void (*free_p) (MemoryContext context, void *pointer);
void *(*realloc) (MemoryContext context, void *pointer, Size size);
- void (*init) (MemoryContext context);
void (*reset) (MemoryContext context);
void (*delete_context) (MemoryContext context);
Size (*get_chunk_space) (MemoryContext context, void *pointer);
--- 57,62 ----
*************** typedef struct MemoryContextData
*** 76,82 ****
/* these two fields are placed here to minimize alignment wastage: */
bool isReset; /* T = no space alloced since last reset */
bool allowInCritSection; /* allow palloc in critical section */
! MemoryContextMethods *methods; /* virtual function table */
MemoryContext parent; /* NULL if no parent (toplevel context) */
MemoryContext firstchild; /* head of linked list of children */
MemoryContext prevchild; /* previous child of same parent */
--- 75,81 ----
/* these two fields are placed here to minimize alignment wastage: */
bool isReset; /* T = no space alloced since last reset */
bool allowInCritSection; /* allow palloc in critical section */
! const MemoryContextMethods *methods; /* virtual function table */
MemoryContext parent; /* NULL if no parent (toplevel context) */
MemoryContext firstchild; /* head of linked list of children */
MemoryContext prevchild; /* previous child of same parent */
diff --git a/src/include/utils/memutils.h b/src/include/utils/memutils.h
index d177b0c..4ac759e 100644
*** a/src/include/utils/memutils.h
--- b/src/include/utils/memutils.h
*************** GetMemoryChunkContext(void *pointer)
*** 132,139 ****
* context creation. It's intended to be called from context-type-
* specific creation routines, and noplace else.
*/
! extern MemoryContext MemoryContextCreate(NodeTag tag, Size size,
! MemoryContextMethods *methods,
MemoryContext parent,
const char *name);
--- 132,140 ----
* context creation. It's intended to be called from context-type-
* specific creation routines, and noplace else.
*/
! extern void MemoryContextCreate(MemoryContext node,
! NodeTag tag, Size size, Size nameoffset,
! const MemoryContextMethods *methods,
MemoryContext parent,
const char *name);
В списке pgsql-hackers по дате отправления: