Re: [HACKERS] Custom compression methods

Поиск
Список
Период
Сортировка
Искать
От
Andrey Borodin
Тема
Re: [HACKERS] Custom compression methods
Дата
Msg-id
323B1B01-DA42-419F-A99C-23E2C162D53B@yandex-team.ru
Ответ на
Список
Дерево обсуждения
[HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus K <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Andres Freund <andres@anarazel.de>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Andres Freund <andres@anarazel.de>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods konstantin knizhnik <k.knizhnik@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Oleg Bartunov <obartunov@gmail.com>
Re: [HACKERS] Custom compression methods Michael Paquier <michael.paquier@gmail.com>
Re: [HACKERS] Custom compression methods Ildar Musin <i.musin@postgrespro.ru>
Re: [HACKERS] Custom compression methods Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Craig Ringer <craig@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Oleg Bartunov <obartunov@gmail.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Евгений Шишкин <itparanoia@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Alvaro Herrera <alvherre@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Alvaro Herrera <alvherre@alvh.no-ip.org>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildar Musin <i.musin@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildar Musin <i.musin@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildar Musin <i.musin@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildar Musin <i.musin@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Dmitry Dolgov <9erthalion6@gmail.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Michael Paquier <michael@paquier.xyz>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Re: Re: [HACKERS] Custom compression methods David Steele <david@pgmasters.net>
Re: Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Andres Freund <andres@anarazel.de>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Andres Freund <andres@anarazel.de>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Andrey Borodin <x4mmm@yandex-team.ru>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Andrey Borodin <x4mmm@yandex-team.ru>
Re: [HACKERS] Custom compression methods Andrey Borodin <x4mmm@yandex-team.ru>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Custom compression methods Alvaro Herrera <alvherre@alvh.no-ip.org>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: Re: [HACKERS] Custom compression methods Amit Khandekar <amitdkhan.pg@gmail.com>
Re: Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: [HACKERS] Custom compression methods Mark Dilger <mark.dilger@enterprisedb.com>
Re: [HACKERS] Custom compression methods Dilip Kumar <dilipbalaut@gmail.com>
Re: Re: Re: [HACKERS] Custom compression methods David Steele <david@pgmasters.net>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Chris Travers <chris.travers@adjust.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Chris Travers <chris.travers@adjust.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Thomas Munro <thomas.munro@gmail.com>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Alvaro Herrera <alvherre@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Ildus K <i.kurbangaliev@gmail.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Re: [HACKERS] Custom compression methods Alvaro Herrera <alvherre@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildar Musin <ildar@adjust.com>
Re: Re: Re: [HACKERS] Custom compression methods Chris Travers <chris.travers@adjust.com>
Re: [HACKERS] Custom compression methods Евгений Шишкин <itparanoia@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Adam Brusselback <adambrusselback@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Re: [HACKERS] Custom compression methods Alexander Korotkov <a.korotkov@postgrespro.ru>
Re: [HACKERS] Custom compression methods Oleg Bartunov <obartunov@gmail.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Chapman Flack <chap@anastigmatix.net>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Alvaro Herrera <alvherre@alvh.no-ip.org>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>
Re: [HACKERS] Custom compression methods Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: [HACKERS] Custom compression methods Robert Haas <robertmhaas@gmail.com>


> 28 дек. 2020 г., в 11:14, Andrey Borodin  написал(а):
> 
>> Thanks for the patch.  Maybe we can allow setting custom compression
>> methods for wal compression as well.
> 
> No, unfortunately, we can't use truly custom methods. Custom compression handlers are WAL-logged. So we can use only static set of hardcoded compression methods.

So, I've made some very basic benchmarks on my machine [0].
With pglz after checkpoint I observe 1146 and 1225 tps.
With lz4 I observe 1485 and 1524 tps.
Without wal_compression I see 1529 tps.

These observations can be explained with plain statement: pglz is bottleneck on my machine, lz4 is not.
While this effect can be reached with other means [1] I believe having lz4 for WAL FPIs would be much more CPU efficient.

FPA lz4 for WAL FPI patch v17. Changes: fixed some frontend issues, added some comments.

Best regards, Andrey Borodin.

[0] https://yadi.sk/d/6y5YiROXQRkoEw
[1] https://www.postgresql.org/message-id/flat/25991595-1848-4178-AA57-872B10309DA2%40yandex-team.ru#e7bb0e048358bcff281011dcf115ad42
В списке pgsql-hackers по дате отправления
От: Tom Lane
Дата:
Сообщение: Re: Let's start using setenv()
От: Paul Martinez
Дата:
FAQ