Обсуждение: BUG #14231: logical replication wal sender process spins when using error traps in function

VGhlIGZvbGxvd2luZyBidWcgaGFzIGJlZW4gbG9nZ2VkIG9uIHRoZSB3ZWJz
aXRlOgoKQnVnIHJlZmVyZW5jZTogICAgICAxNDIzMQpMb2dnZWQgYnk6ICAg
ICAgICAgIE1pY2hhZWwgRGF5CkVtYWlsIGFkZHJlc3M6ICAgICAgYmxha2VA
cmNtYWlsLmNvbQpQb3N0Z3JlU1FMIHZlcnNpb246IDkuNC42Ck9wZXJhdGlu
ZyBzeXN0ZW06ICAgRnJlZUJTRCAxMC4xLVJFTEVBU0UKRGVzY3JpcHRpb246
ICAgICAgICAKClRoZSBmb2xsb3dpbmcgZnVuY3Rpb24gY2FuIGJlIHVzZWQg
dG8gcmVwcm9kdWNlIGFuIGlzc3VlIHdpdGggbG9naWNhbApyZXBsaWNhdGlv
biAodmVyc2lvbiA5LjQuNikuDQoNCkNvbm5lY3QgYSBsb2dpY2FsIGNsaWVu
dCB3aXRoIHRlc3RfZGVjb2RpbmcgcGx1Z2luIGFuZCBydW4gdGhlIGNvZGUg
YmVsb3cuDQpJdCB3aWxsIGNhdXNlIHRoZSByZXBsaWNhdGlvbiBwcm9jZXNz
IHRvIHNwaW4sIHVzaW5nIDEwMCUgQ1BVIGZvciBzb21lIGxvbmcKcGVyaW9k
IG9mIHRpbWUuDQpSZWNvdmVyeSB0aW1lIHNlZW1zIHRvIGV4cG9uZW50aWFs
bHkgaW5jcmVhc2UgZGVwZW5kaW5nIG9uIHRoZSBudW1iZXIgb2YKdGltZXMg
dGhlIGZ1bmN0aW9uIGlzIGNhbGxlZC4NCkl0IGlzIGltcG9zc2libGUgdG8g
cHJlbWF0dXJlbHkgc3RvcCB0aGUgcHJvY2VzcyB3aXRob3V0IHJlc3RhcnRp
bmcgdGhlCnNlcnZlciAoaXQgZG9lcyBub3QgdGFrZSBub3RpY2Ugb2YgYSBk
aXNjb25uZWN0KS4NCiANCkNSRUFURSBUQUJMRSBJRiBOT1QgRVhJU1RTIHRl
c3QxIChpZCBzZXJpYWwgbm90IG51bGwsIHZhbHVlIHRleHQpOw0KIA0KQ1JF
QVRFIE9SIFJFUExBQ0UgRlVOQ1RJT04gdGVzdDEoKQ0KICBSRVRVUk5TIHZv
aWQgQVMgJCQNCkJFR0lODQogICAgQkVHSU4NCiAgICAgIElOU0VSVCBJTlRP
IHRlc3QxICh2YWx1ZSkgdmFsdWVzIChyYW5kb20oKTo6dGV4dCk7DQogICAg
RVhDRVBUSU9OIFdIRU4gdW5pcXVlX3Zpb2xhdGlvbiBUSEVODQogICAgICAg
IC0tIEpVU1QgaGF2aW5nIHRoaXMgZXhjZXB0aW9uIGhhbmRsaW5nIGJsb2Nr
IGNhdXNlcyBpdCB0byBmYWlsDQogICAgRU5EOw0KRU5EOw0KJCQgTEFOR1VB
R0UgcGxwZ3NxbDsNCiANCmV4cGxhaW4gYW5hbHl6ZSBTRUxFQ1QgdGVzdDEo
KSBGUk9NIGdlbmVyYXRlX3NlcmllcygxLCA1MDAwMCk7Cgo=

>>>>> "blake" == blake  <blake@rcmail.com> writes:

 blake> Connect a logical client with test_decoding plugin and run the
 blake> code below.  It will cause the replication process to spin,
 blake> using 100% CPU for some long period of time.

So what I've found in my analysis so far since you mentioned this on IRC
is:

1. The time is being spent in ReorderBufferCleanupTXN and the functions
it calls. This is called once for the transaction and (recursively) once
per subtransaction.

2. Within each of those calls, the main culprit seems to be pathological
behavior of the retail pfree() calls of allocated memory. The loop in
AllocSetFree which chases down the allocated block list (for chunks over
the chunk limit) is being executed nearly 900 million times for what
seems to be about 42 thousand calls.

A quick scan of the code suggests that the worst case is when blocks are
being freed in FIFO order, which seems quite plausible in this case, and
the performance is potentially O(N^2).

So I think this is primarily an artifact of doing so much retail
palloc/pfree in a single memory context.

--
Andrew (irc:RhodiumToad)

On Wed, Jul 6, 2016 at 2:00 PM, Andrew Gierth
<andrew@tao11.riddles.org.uk> wrote:
> So I think this is primarily an artifact of doing so much retail
> palloc/pfree in a single memory context.

I wonder what the effect of commit
25c539233044c235e97fd7c9dc600fb5f08fe065, "Improve performance in
freeing memory contexts" might be here. In other words, maybe this is
not so bad on 9.6.

--
Peter Geoghegan

Andrew Gierth wrote:

> A quick scan of the code suggests that the worst case is when blocks are
> being freed in FIFO order, which seems quite plausible in this case, and
> the performance is potentially O(N^2).
>
> So I think this is primarily an artifact of doing so much retail
> palloc/pfree in a single memory context.

As I recall, this is the main reason Andres stuck the slab cache in
reorderbuffer.  Maybe it'd be worthwhile to implement a different
MemoryContext tailored to this use case, and remove the slab cache
stuff.

--
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2016-07-06 17:26:00 -0400, Alvaro Herrera wrote:
> Andrew Gierth wrote:
>
> > A quick scan of the code suggests that the worst case is when blocks are
> > being freed in FIFO order, which seems quite plausible in this case, and
> > the performance is potentially O(N^2).

Hrmpf.

> > So I think this is primarily an artifact of doing so much retail
> > palloc/pfree in a single memory context.
>
> As I recall, this is the main reason Andres stuck the slab cache in
> reorderbuffer.

That was primarily for allocation performance, but that's not far off.
Michael, what you could do is to adjust
static const Size max_cached_changes = 4096 * 2;
static const Size max_cached_tuplebufs = 4096 * 2;        /* ~8MB */
static const Size max_cached_transactions = 512;
to some ridiculous sizes, and see how that affects performance.


> Maybe it'd be worthwhile to implement a different
> MemoryContext tailored to this use case, and remove the slab cache
> stuff.

Hm. I'm not sure how much you could do with a memory context
itself. Seems to need an actual slab allocator, something not easily
done via the MemoryContext interface.

Andres

On 2016-07-06 14:23:33 -0700, Peter Geoghegan wrote:
> On Wed, Jul 6, 2016 at 2:00 PM, Andrew Gierth
> <andrew@tao11.riddles.org.uk> wrote:
> > So I think this is primarily an artifact of doing so much retail
> > palloc/pfree in a single memory context.
>
> I wonder what the effect of commit
> 25c539233044c235e97fd7c9dc600fb5f08fe065, "Improve performance in
> freeing memory contexts" might be here. In other words, maybe this is
> not so bad on 9.6.

Seems unrelated. We're just doing pfrees, not MemoryContextDelete/Reset/Whatnot.

On Wed, Jul 6, 2016 at 2:26 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>> So I think this is primarily an artifact of doing so much retail
>> palloc/pfree in a single memory context.
>
> As I recall, this is the main reason Andres stuck the slab cache in
> reorderbuffer.  Maybe it'd be worthwhile to implement a different
> MemoryContext tailored to this use case, and remove the slab cache
> stuff.

I withdraw my remarks on that commit -- I don't think that's what it is now.

I had luck adding a new child memory context for tuplesort.c in commit
0011c0091e886b874e485a46ff2c94222ffbf550. This allowed tuplesort.c to
blow away the free list via a new call to MemoryContextReset(). This
didn't even remove *any* pfree() operations in the affected codepath,
but was still quite effective.

I'm not sure that that's comparable, but it may be an interesting case
that others are not aware of.

--
Peter Geoghegan

>>>>> "Peter" == Peter Geoghegan <pg@heroku.com> writes:

 >> So I think this is primarily an artifact of doing so much retail
 >> palloc/pfree in a single memory context.

 Peter> I wonder what the effect of commit
 Peter> 25c539233044c235e97fd7c9dc600fb5f08fe065, "Improve performance
 Peter> in freeing memory contexts" might be here. In other words, maybe
 Peter> this is not so bad on 9.6.

It's just as bad on 9.6b2

--
Andrew (irc:RhodiumToad)

On 07/06/2016 11:40 PM, Peter Geoghegan wrote:
> On Wed, Jul 6, 2016 at 2:26 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>>> So I think this is primarily an artifact of doing so much retail
>>> palloc/pfree in a single memory context.
>>
>> As I recall, this is the main reason Andres stuck the slab cache in
>> reorderbuffer.  Maybe it'd be worthwhile to implement a different
>> MemoryContext tailored to this use case, and remove the slab cache
>> stuff.
>
> I withdraw my remarks on that commit -- I don't think that's what it is now.
>
> I had luck adding a new child memory context for tuplesort.c in commit
> 0011c0091e886b874e485a46ff2c94222ffbf550. This allowed tuplesort.c to
> blow away the free list via a new call to MemoryContextReset(). This
> didn't even remove *any* pfree() operations in the affected codepath,
> but was still quite effective.
>
> I'm not sure that that's comparable, but it may be an interesting case
> that others are not aware of.
>

I'm not very familiar with the reorderbuffer.c, but my understanding is
that it's an object with potentially very long lifetime, doing a lot of
interleaved palloc/pfree (or accessing the slab cache). The lifetime of
tuplesort is much shorter with clear boundaries, and probably does not
need palloc/pfree interleaved that much.

The separate memory context would probably fix the pfree issue, but I'm
afraid it'd make all pfree() calls pointless useless as AllocSet keeps
all pfreed chunks in internal freelists and only releases them at the
end. Which means the memory context would allocate memory and then keep
it until it's destroyed, meaning the slab limits would be pointless.

I do have a few rough ideas how this might be improved, but it's a bit
rough at this point. The distinguishing feature of the slab allocator is
that all objects have exactly the same size, which makes the "many
freelist" idea in AllocSet a bit pointless. (Let's ignore that we use
the same context for three different objects for now.)

So I'm thinking about adding a new MemoryContext implementation, e.g.
SlabMemoryContext, which will only allow allocating pieces with a
particular size. Then we can get rid of the AllocSet pieces that deal
with freelists, and also make sure we know which blocks contain free
chunks so that we can reuse chunks from more full blocks, etc. But I
haven't tried coding anything, so maybe it's all stupid.

regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Wed, Jul 13, 2016 at 11:36 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>> 0011c0091e886b874e485a46ff2c94222ffbf550. This allowed tuplesort.c to
>> blow away the free list via a new call to MemoryContextReset(). This
>> didn't even remove *any* pfree() operations in the affected codepath,
>> but was still quite effective.
>>
>> I'm not sure that that's comparable, but it may be an interesting case
>> that others are not aware of.
>>
>
> I'm not very familiar with the reorderbuffer.c, but my understanding is that
> it's an object with potentially very long lifetime, doing a lot of
> interleaved palloc/pfree (or accessing the slab cache). The lifetime of
> tuplesort is much shorter with clear boundaries, and probably does not need
> palloc/pfree interleaved that much.

Actually, the lifetime of a non-final merge step could be tens of
seconds or more, and there are continual palloc() and pfree() calls at
the tuple level, which leads to this fragmentation. (Note that this
does apply to final on-the-fly merges, which were made to use batch
memory by that same commit.)


--
Peter Geoghegan

On 07/13/2016 08:36 PM, Tomas Vondra wrote:
...
>
> I do have a few rough ideas how this might be improved, but it's a bit
> rough at this point. The distinguishing feature of the slab allocator is
> that all objects have exactly the same size, which makes the "many
> freelist" idea in AllocSet a bit pointless. (Let's ignore that we use
> the same context for three different objects for now.)
>
> So I'm thinking about adding a new MemoryContext implementation, e.g.
> SlabMemoryContext, which will only allow allocating pieces with a
> particular size. Then we can get rid of the AllocSet pieces that deal
> with freelists, and also make sure we know which blocks contain free
> chunks so that we can reuse chunks from more full blocks, etc. But I
> haven't tried coding anything, so maybe it's all stupid.

So, I've spent a few hours experimenting with this idea, and I believe
it seems like a good way forward. Attached is an early WIP patch that
does this:

1) adds SlabContext, a new MemoryContext implementation handling
allocations of chunks of equal size. Thanks to that the implementation
is much simpler than AllocSetContext (no multi-size freelists, doubling
etc.) and there's also a bunch of optimization ideas (freelist built
into blocks, ability to free the blocks, which asets can't do).

2) rips out the custom slab from reorderbuffer.c and replaces it with
four memory contexts, one for each type of objects:

   change_context: SlabContext for ReorderBufferChange objects
   txn_context: SlabContext for ReorderBufferChangeTXN
   tup_context: SlabContextCreate for tuples <= MaxHeapTupleSize
   tup_context2: AllocSetContext for tuples > MaxHeapTupleSize

Obviously, all of this is very rough at this point. Nevertheless it
compiles and does not crash when running the test case from the initial
report.

So I've done some simple tests with different number of invocations of
the function, and the comparison looks like this:

        N |   master |  patched
   -----------------------------
    10000 |    100ms |    100ms
    50000 |  15000ms |    350ms
   100000 | 146000ms |    700ms
   200000 |        ? |   1400ms

I got bored waiting for the 200k case on master after ~15 minutes, which
is why I put there the question mark.

I'm not sure whether the SlabContext is needed here - perhaps AllocSets
would do equally well, at least in this case. I however wonder whether
the existing local slab cache can actually improve anything, because
what I see is a long sequence of pallocs() followed by a long sequence
of pfrees(), which makes any reuse impossible. But I guess there are
other cases where the palloc() and pfree() calls are mixed.

The other thing is that splitting the tuple context into two parts seems
a bit wasteful - this mostly follows the MaxHeapTupleSize idea from the
existing slab code, but how many tuples actually are this big? I believe
a single AllocSetContext would work just fine.

regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hi,

On 2016-07-19 07:04:43 +0200, Tomas Vondra wrote:
> So, I've spent a few hours experimenting with this idea, and I believe it
> seems like a good way forward. Attached is an early WIP patch that does
> this:

Cool! Thanks for looking into this.


> I'm not sure whether the SlabContext is needed here - perhaps AllocSets
> would do equally well, at least in this case. I however wonder whether the
> existing local slab cache can actually improve anything, because what I see
> is a long sequence of pallocs() followed by a long sequence of pfrees(),
> which makes any reuse impossible. But I guess there are other cases where
> the palloc() and pfree() calls are mixed.

In something more oltp like you'll see exactly that. IIRC, in pgbench,
we pretty much never allocate for the slab allocated stuff, after some
warmup.


> The other thing is that splitting the tuple context into two parts seems a
> bit wasteful - this mostly follows the MaxHeapTupleSize idea from the
> existing slab code, but how many tuples actually are this big?

The issue is that allocating them differently sized leads to massive
memory fragmentation over time.


Greetings,

Andres Freund

On 07/19/2016 07:27 AM, Andres Freund wrote:
> Hi,
>
> On 2016-07-19 07:04:43 +0200, Tomas Vondra wrote:
>> So, I've spent a few hours experimenting with this idea, and I
>> believe it seems like a good way forward. Attached is an early WIP
>> patch that does this:
>
> Cool! Thanks for looking into this.
>
>
>> I'm not sure whether the SlabContext is needed here - perhaps
>> AllocSets would do equally well, at least in this case. I however
>> wonder whether the existing local slab cache can actually improve
>> anything, because what I see is a long sequence of pallocs()
>> followed by a long sequence of pfrees(), which makes any reuse
>> impossible. But I guess there are other cases where the palloc()
>> and pfree() calls are mixed.
>
> In something more oltp like you'll see exactly that. IIRC, in
> pgbench, we pretty much never allocate for the slab allocated stuff,
> after some warmup.

Yeah. Haven't done much testing at night. I think we'll need a set of
test cases to validate the changes. We certainly don't want to improve
one case and regress two others.

>
>> The other thing is that splitting the tuple context into two parts
>> seems a bit wasteful - this mostly follows the MaxHeapTupleSize
>> idea from the existing slab code, but how many tuples actually are
>> this big?
>
> The issue is that allocating them differently sized leads to massive
> memory fragmentation over time.
>

Sure, in all allocators we basically trade space and time efficiency. If
we could get some apriori some estimate of the tuple size, we could
probably tune this. It's just that ~8kB per tuple seems a bit too high.


regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 07/19/2016 03:35 PM, Tomas Vondra wrote:
> On 07/19/2016 07:27 AM, Andres Freund wrote:
>> Hi,
>>
>> On 2016-07-19 07:04:43 +0200, Tomas Vondra wrote:
>>> So, I've spent a few hours experimenting with this idea, and I
>>> believe it seems like a good way forward. Attached is an early WIP
>>> patch that does this:
>>
>> Cool! Thanks for looking into this.
>>
>>
>>> I'm not sure whether the SlabContext is needed here - perhaps
>>> AllocSets would do equally well, at least in this case. I however
>>> wonder whether the existing local slab cache can actually improve
>>> anything, because what I see is a long sequence of pallocs()
>>> followed by a long sequence of pfrees(), which makes any reuse
>>> impossible. But I guess there are other cases where the palloc()
>>> and pfree() calls are mixed.
>>
>> In something more oltp like you'll see exactly that. IIRC, in
>> pgbench, we pretty much never allocate for the slab allocated stuff,
>> after some warmup.
>
> Yeah. Haven't done much testing at night. I think we'll need a set of
> test cases to validate the changes. We certainly don't want to improve
> one case and regress two others.
>
>>
>>> The other thing is that splitting the tuple context into two parts
>>> seems a bit wasteful - this mostly follows the MaxHeapTupleSize
>>> idea from the existing slab code, but how many tuples actually are
>>> this big?
>>
>> The issue is that allocating them differently sized leads to massive
>> memory fragmentation over time.
>>
>
> Sure, in all allocators we basically trade space and time efficiency. If
> we could get some apriori some estimate of the tuple size, we could
> probably tune this. It's just that ~8kB per tuple seems a bit too high.

[one huge cup of coffee later ...]

So, I think it's actually quite simple to improve this by using a
generational design, i.e. creating a sequence of slab contexts every N
tuples, using the average tuple length from the previous batch as input
for the next one (multiplied by 1.5x to give a bit of slack). Maybe this
heuristics is far too simple, but perhaps we can

Attached is a v2 of the patch doing pretty much this - I wouldn't call
this even half-baked, it's a rather early PoC thingy. But it seems to
work, and for the test case from the initial report, it significantly
reduces memory consumption by a factor of 100x, as the average tuple
length is ~80B, not 8kB. It increases the runtime a bit (say, from 700ms
to 800ms for the 100k test case), but that seems negligible compared to
the master behavior and worth considering the memory consumption reduction.

This relies on one of the SlabContext features - it can easily see which
blocks are free because it has 'free bitmap' in each block, so old
contexts get automatically freed once the tuples are pfreed. I have not
made any attempt to free the context structure itself, which is a memory
leak, but I feel lazy and it's good enough for a PoC.

The AllocSet obviously can't do that, so that part is not part of the
generation design - it would just prevent reuse of the memory, thus
increasing memory consumption. But if we could make the AllocSet capable
of doing AllocSetIsEmpty reasonably (and not just when it's reset, which
is pretty useless I believe), we could discard the whole context (making
the generational design viable).

Another SlabContext detail worth mentioning is that it actually makes
the chunk header larger, by plugging in an additional pointer (to the
block). In the end this is what allows in-block free bitmap etc. and is
more than compensated for by not doing doubling the chunk size etc.

I don't expect to have much time to work on this in the next few weeks,
so if anyone wants to grab this and make it a proper patch, be my guest.

regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 07/19/2016 06:22 PM, Tomas Vondra wrote:
>
> [one huge cup of coffee later ...]
>
> So, I think it's actually quite simple to improve this by using a
> generational design, i.e. creating a sequence of slab contexts every N
> tuples, using the average tuple length from the previous batch as input
> for the next one (multiplied by 1.5x to give a bit of slack). Maybe this
> heuristics is far too simple, but perhaps we can
>
> Attached is a v2 of the patch doing pretty much this - I wouldn't call
> this even half-baked, it's a rather early PoC thingy. But it seems to
> work, and for the test case from the initial report, it significantly
> reduces memory consumption by a factor of 100x, as the average tuple
> length is ~80B, not 8kB. It increases the runtime a bit (say, from 700ms
> to 800ms for the 100k test case), but that seems negligible compared to
> the master behavior and worth considering the memory consumption reduction.
>
> This relies on one of the SlabContext features - it can easily see which
> blocks are free because it has 'free bitmap' in each block, so old
> contexts get automatically freed once the tuples are pfreed. I have not
> made any attempt to free the context structure itself, which is a memory
> leak, but I feel lazy and it's good enough for a PoC.
>
> The AllocSet obviously can't do that, so that part is not part of the
> generation design - it would just prevent reuse of the memory, thus
> increasing memory consumption. But if we could make the AllocSet capable
> of doing AllocSetIsEmpty reasonably (and not just when it's reset, which
> is pretty useless I believe), we could discard the whole context (making
> the generational design viable).
>

Heck, we could even implement this as a special type of memory context,
suitable for cases with mostly the same palloc sizes (but not entirely
constant, as required by the SlabContext as implemented in the patch).

regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 07/19/2016 06:22 PM, Tomas Vondra wrote:
>
> I don't expect to have much time to work on this in the next few weeks,
> so if anyone wants to grab this and make it a proper patch, be my guest.

Meh, who am I kidding. I've spent a bit more time hacking on this
anyway. Attached is v3, fixing various thinkos and optimizations, checks
in the code and especially docs explaining the main ideas. It's far from
perfect but much better than the previous two versions.

And it actually performs even better than the previous versions (about
2x faster on the 50k case).

regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Sorry to pester, but I’m not sure how bugs like this are tracked over longer periods of time.  The discussion around
possiblefixes for this went above my pay grade, and I’m wondering whether a fix was ever released. If not, is there any
wayfor me to know when it does get fixed other than reviewing release notes?   

> On Jul 19, 2016, at 9:18 PM, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
>
> On 07/19/2016 06:22 PM, Tomas Vondra wrote:
>>
>> I don't expect to have much time to work on this in the next few weeks,
>> so if anyone wants to grab this and make it a proper patch, be my guest.
>
> Meh, who am I kidding. I've spent a bit more time hacking on this anyway. Attached is v3, fixing various thinkos and
optimizations,checks in the code and especially docs explaining the main ideas. It's far from perfect but much better
thanthe previous two versions. 
>
> And it actually performs even better than the previous versions (about 2x faster on the 50k case).
>
> regards
>
> --
> Tomas Vondra                  http://www.2ndQuadrant.com
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
> <slab-reorder-buffer-v3.patch>



--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On 2017-02-13 23:01:26 -0500, Michael Day wrote:
> Sorry to pester, but I’m not sure how bugs like this are tracked over longer periods of time.  The discussion around
possiblefixes for this went above my pay grade, and I’m wondering whether a fix was ever released. If not, is there any
wayfor me to know when it does get fixed other than reviewing release notes?  
 

It's still being reviewed (and changes are being made).  I'm
unfortunately doubtful that we can make that into a small enough change
to release in the back-branches, i.e. it'd only be in the next major
version of postgres.

What you could do in your release, is to increase the default limits at
the top of reorderbuffer.c:

static const Size max_cached_changes = 4096 * 2;
static const Size max_cached_tuplebufs = 4096 * 2;        /* ~8MB */
static const Size max_cached_transactions = 512;

using higher limits there will mean memory usage won't go down below
that amount of memory (within one session).

And yes, unfortunately reviewing release notes and/or this thread, is
what you'll have to resort to.

Regards,

Andres


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Andres Freund <andres@anarazel.de> writes:
> On 2017-02-13 23:01:26 -0500, Michael Day wrote:
>> Sorry to pester, but I’m not sure how bugs like this are tracked over longer periods of time.  The discussion around
possiblefixes for this went above my pay grade, and I’m wondering whether a fix was ever released. If not, is there any
wayfor me to know when it does get fixed other than reviewing release notes?   

> It's still being reviewed (and changes are being made).  I'm
> unfortunately doubtful that we can make that into a small enough change
> to release in the back-branches, i.e. it'd only be in the next major
> version of postgres.

Actually, in the context of another discussion we thought of a way to
ameliorate this problem in a localized fashion.  So it will be addressed
in the next point releases (probably in May); or if you're desperate
you could grab the 9.4 branch tip from git.

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8dd5c4171fb2c3b78e713c49f531ad8ff3b245ea

            regards, tom lane


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs