čt 11. 4. 2019 v 18:18 odesílatel Bruce Momjian <bruce@momjian.us> napsal:
On Sun, Mar 31, 2019 at 05:25:51PM +0300, Николай Петров wrote: > Why it should be implemented on Storage Manager level instead of usage > Pluggable storage API [9]? > - From my perspective view Storage Manager level implementation > allows to focus on proper I/O operations and compression. > It allows to write much more simple realization. It's because of > Pluggable storage API force you to implement more complex > interfaces. To be honest, I am really hesitating about this point, > especially because of Pluggable storage API allows to create > extension without core code modification and it potentially allows > to use more perfective compression algorithms (Table Access Manager > allows you to get more information about storing data). > > I would like to implement a proof of concept > and have a couple of questions: > - your opinion about necessity of this feature > (Compressed Storage Manager) > - Is it good idea to implement DB compression on Storage Manager > level? Perhaps it is better to use Pluggable storage API. > - Is there any reason to refuse this proposal? > - Are there any circumstances what didn't allow to implement > Compressed Storage Manager?
Stepping back a bit, there are several levels of compression:
1. single field 2. across all fields in a row 3. across rows on a single page 4. across all rows in a table 5. across tables in a database
We currently do #1 with TOAST, and your approach would allow the first three. #4 feels like it is getting near the features of columnar storage. I think it is unclear if adding #2 and #3 produce enough of a benefit to warrant special storage, given the complexity and overhead of implementing it.
@4 compression over columns on page are probably much more effective. But there can some preprocessing stage, where rows can be transformed to columns.
This doesn't need real column store, and can helps lot of. Real column store has sense when columns are separated to different pages. But for compressions, we can transform rows to columns without real column storage.
Probably 8kB page is too small for this case.
Regards
Pavel
I do think the Pluggable storage API is the right approach, and, if you are going to go that route, adding #4 compression seems very worthwhile.