PoC Refactor AM analyse API

Поиск
Список
Период
Сортировка
От Смирнов Денис
Тема PoC Refactor AM analyse API
Дата
Msg-id C7CFE16B-F192-4124-BEBB-7864285E0FF7@arenadata.io
обсуждение исходный текст
Ответы Re: PoC Refactor AM analyse API  (Denis Smirnov <sd@arenadata.io>)
Re: PoC Refactor AM analyse API  (Andrey Borodin <x4mmm@yandex-team.ru>)
Список pgsql-hackers
Hello all!

I suggest a refactoring of analyze AM API as it is too much heap specific at the moment. The problem was inspired by Greenplum’s analyze improvement for append-optimized row and column AM with variable size compressed blocks.
Currently we do analyze in two steps.

1. Sample fix size blocks with algorithm S from Knuth (BlockSampler function)
2. Collect tuples into reservoir with algorithm Z from Vitter.

So this doesn’t work for AMs using variable sized physical blocks for example. They need weight random sampling (WRS) algorithms like A-Chao or logical blocks to follow S-Knuth (and have a problem with RelationGetNumberOfBlocks() estimating a physical number of blocks). Another problem with columns - they are not passed to analyze begin scan and can’t benefit from column storage at ANALYZE TABLE (COL).

The suggestion is to replace table_scan_analyze_next_block() and table_scan_analyze_next_tuple() with a single function: table_acquire_sample_rows(). The AM implementation of table_acquire_sample_rows() can use the BlockSampler functions if it wants to, but if the AM is not block-oriented, it could do something else. This suggestion also passes VacAttrStats to table_acquire_sample_rows() for column-oriented AMs and removes PROGRESS_ANALYZE_BLOCKS_TOTAL and PROGRESS_ANALYZE_BLOCKS_DONE definitions as not all AMs can be block-oriented.




Best regards,
Denis Smirnov | Developer
sd@arenadata.io 
Arenadata | Godovikova 9-17, Moscow 129085 Russia

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Daniel Gustafsson
Дата:
Сообщение: Re: Refactor MD5 implementations and switch to EVP for OpenSSL
Следующее
От: Bharath Rupireddy
Дата:
Сообщение: Re: Parallel Inserts in CREATE TABLE AS