On 07/13/2016 08:36 PM, Tomas Vondra wrote:
...
>
> I do have a few rough ideas how this might be improved, but it's a bit
> rough at this point. The distinguishing feature of the slab allocator is
> that all objects have exactly the same size, which makes the "many
> freelist" idea in AllocSet a bit pointless. (Let's ignore that we use
> the same context for three different objects for now.)
>
> So I'm thinking about adding a new MemoryContext implementation, e.g.
> SlabMemoryContext, which will only allow allocating pieces with a
> particular size. Then we can get rid of the AllocSet pieces that deal
> with freelists, and also make sure we know which blocks contain free
> chunks so that we can reuse chunks from more full blocks, etc. But I
> haven't tried coding anything, so maybe it's all stupid.
So, I've spent a few hours experimenting with this idea, and I believe
it seems like a good way forward. Attached is an early WIP patch that
does this:
1) adds SlabContext, a new MemoryContext implementation handling
allocations of chunks of equal size. Thanks to that the implementation
is much simpler than AllocSetContext (no multi-size freelists, doubling
etc.) and there's also a bunch of optimization ideas (freelist built
into blocks, ability to free the blocks, which asets can't do).
2) rips out the custom slab from reorderbuffer.c and replaces it with
four memory contexts, one for each type of objects:
change_context: SlabContext for ReorderBufferChange objects
txn_context: SlabContext for ReorderBufferChangeTXN
tup_context: SlabContextCreate for tuples <= MaxHeapTupleSize
tup_context2: AllocSetContext for tuples > MaxHeapTupleSize
Obviously, all of this is very rough at this point. Nevertheless it
compiles and does not crash when running the test case from the initial
report.
So I've done some simple tests with different number of invocations of
the function, and the comparison looks like this:
N | master | patched
-----------------------------
10000 | 100ms | 100ms
50000 | 15000ms | 350ms
100000 | 146000ms | 700ms
200000 | ? | 1400ms
I got bored waiting for the 200k case on master after ~15 minutes, which
is why I put there the question mark.
I'm not sure whether the SlabContext is needed here - perhaps AllocSets
would do equally well, at least in this case. I however wonder whether
the existing local slab cache can actually improve anything, because
what I see is a long sequence of pallocs() followed by a long sequence
of pfrees(), which makes any reuse impossible. But I guess there are
other cases where the palloc() and pfree() calls are mixed.
The other thing is that splitting the tuple context into two parts seems
a bit wasteful - this mostly follows the MaxHeapTupleSize idea from the
existing slab code, but how many tuples actually are this big? I believe
a single AllocSetContext would work just fine.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services