Got dragged into other things and didn't make any progress on this last week. I'm trying again now.
Good. Thanks for debug and fixing bugs. Can I do anything for this patch now?
I wonder if we're leaving some money on the table, by using varbyte encoding. Googling around, there are many other compression methods out there for compressing integer deltas that compress better, and/or decompress faster.
Ok, I'll try to google around. Probably, there are some better options.
Even if we use varbyte encoding, I wonder if it would be better to treat block + offset number as a single 48-bit integer, rather than encode them separately. That would allow the delta of two items on the same page to be stored as a single byte, rather than two bytes. Naturally it would be a loss on other values, but would be nice to see some kind of an analysis on that. I suspect it might make the code simpler, too.
Yeah, I had that idea, but I thought it's not a better option. Will try to do some analysis.