Re: Reducing Memory Consumption (aset and generation)

Поиск
Список
Период
Сортировка
От Ranier Vilela
Тема Re: Reducing Memory Consumption (aset and generation)
Дата
Msg-id CAEudQAqWEXJJU07oqeCGZVJ=2JTK1yapxYgFF=SzmUOkDrFdSA@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Reducing Memory Consumption (aset and generation)  (Ranier Vilela <ranier.vf@gmail.com>)
Ответы Re: Reducing Memory Consumption (aset and generation)  (Matthias van de Meent <boekewurm+postgres@gmail.com>)
Список pgsql-hackers
Em seg., 6 de jun. de 2022 às 21:14, Ranier Vilela <ranier.vf@gmail.com> escreveu:
Em seg., 6 de jun. de 2022 às 20:37, David Rowley <dgrowleyml@gmail.com> escreveu:
On Mon, 6 Jun 2022 at 07:28, Ranier Vilela <ranier.vf@gmail.com> wrote:
> 4) 004-generation-reduces-memory-consumption-BUG.patch
> Same to the (2), but with BUG.
> It only takes a few tweaks to completely remove the field block.

> This fails with make check.
> I couldn't figure out why it doesn't work with 16 bits (struct GenerationChunk).

Hi David, thanks for taking a look at this.
 
I think you're misunderstanding how blocks and chunks work here.  A
block can have many chunks.  You can't find the block that a chunk is
on by subtracting Generation_BLOCKHDRSZ from the pointer given to
GenerationFree(). That would only work if the chunk happened to be the
first chunk on a block. If it's anything apart from that then you'll
be making adjustments to the memory of some prior chunk on the block.
I imagine this is the reason you can't get the tests to pass.
Ok, I am still learning about this.
Can you explain why subtracting Generation_BLOCKHDRSZ from the pointer,
works for sizeof(struct GenerationChunk) = 24 bits,
When all references for the block field have been removed.
This pass check-world.
 

Can you also explain why you think moving code around randomly or
adding unlikely() macros helps reduce the memory consumption overheads
of generation contexts?
Of course, those changes do not reduce memory consumption.
But, IMO, I think those changes improve the access to memory regions,
because of the locality of the data.

About "unlikely macros", this helps the branchs prediction, when most of the time,
malloc and related functions, will not fail.
 
  I imagine you think that's helping to further
improve performance, but you've not offered any evidence of that
separately from the other changes you've made. If you think those are
useful changes then I recommend you run individual benchmarks and
offer those as proof that those changes are worthwhile.
Ok, I can understand, are changes unrelated.
Let's restart this, to simplify the review and commit work.

Regarding the patches now, we have:
1) v1-001-aset-reduces-memory-consumption.patch
Reduces memory used by struct AllocBlockData by minus 8 bits,
reducing the total size to 32 bits, which leads to "fitting" two structs in a 64bit cache.

Remove tests elog(ERROR, "could not find block containing chunk %p" and
elog(ERROR, "could not find block containing chunk %p", moving them to
MEMORY_CONTEXT_CHECKING context.

Since 8.2 versions, nobody complains about these tests.

But if is not acceptable, have the option (3) v1-003-aset-reduces-memory-consumption.patch

2) v1-002-generation-reduces-memory-consumption.patch
Reduces memory used by struct GenerationBlock, by minus 8 bits,
reducing the total size to 32 bits, which leads to "fitting" two structs in a 64bit cache.

3) v1-003-aset-reduces-memory-consumption.patch
Same to the (1), but without remove the tests:
elog(ERROR, "could not find block containing chunk %p" and
elog(ERROR, "could not find block containing chunk %p",
But at the cost of removing a one tiny part of the tests and
moving them to MEMORY_CONTEXT_CHECKING context.

Since 8.2 versions, nobody complains about these tests.

regards,
Ranier Vilela
Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Thomas Munro
Дата:
Сообщение: Re: Collation version tracking for macOS
Следующее
От: Dong Wook Lee
Дата:
Сообщение: Re: pg_buffercache: add sql test