On 02/27/2017 12:17 PM, Andres Freund wrote:
> Hi,
>
> On 2017-02-24 14:10:38 -0800, Andres Freund wrote:
>> I've not yet looked a lot at the next type of context - I want to get
>> this much committed first...
>>
>> I plan to give this another pass sometime this weekend and then push
>> soon.
>
> Before committing I wanted to make sure that
> http://archives.postgresql.org/message-id/32354.1487977458%40sss.pgh.pa.us
> isn't a sufficient fix.
>
> With the test of N=1000000 from this thread I measured both runtime and
> memory usage (note that's peak virtual memory which includes 2GB of
> shared_buffers and such), in assert enabled builds.
>
> master: doesn't finish reasonably
> master+doubly linked list fix: 9390.805 ms VmPeak: 10,969,424 kb
> master+this thread: 6500.293 ms VmPeak: 2,969,528 kB
>
> So the doubly-linked-list fix is great (and much more backpatchable),
> but the patches in this thread are both better runtime *and* peak memory
> usage wise. So that seems like a clear call.
>
Nice, thanks for doing the test.
> I've not yet reviewed the generational allocator yet, but during these
> measurements I get:
> postgres[3970][1]=# select count(*) FROM pg_logical_slot_get_changes('ttt', NULL, NULL);
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d011ef10f0 exceeds 7234
allocated
> LOCATION: GenerationCheck, generation.c:693
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d01023eba0 exceeds 65532
allocated
> LOCATION: GenerationCheck, generation.c:693
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d00d7fb870 exceeds 65532
allocated
> LOCATION: GenerationCheck, generation.c:693
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d00cde17b0 exceeds 65531
allocated
> LOCATION: GenerationCheck, generation.c:693
>
> that seems to occur when there's currently in-progress transactions when
> finishing decoding:
>
...
>
> could it be that the test's condition is inverted?
>
Yeah, that seems like the culprit - the condition seems wrong. I wonder
why I haven't seen it during my tests, though ...
> I'll work on getting slab committed first, and then review / edit /
> commit generation.c later. One first note there is that I'm wondering
> if generation.c is a too generic filename.
>
Naming things is hard.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services