On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost <sfrost@snowman.net> wrote:
Greetings,
* Paul Ramsey (pramsey@cleverelephant.ca) wrote: > The attached patch adds in a code path to do a partial decompression of the > TOAST entry, when the requested slice is at the start of the object.
There two things that I wonder about in the patch- if it would be of any use to try and allocate on a need basis instead of just allocating the whole chunk up to the toast size,
I'm not sure what I was thinking when I rejected allocating the slice size in favour of the whole uncompressed size... I cannot see why that wouldn't work.
and secondly, why we wouldn't consider handling a non-zero offset. A non-zero offset would, of course, still require decompressing from the start and then just throwing away what we skip over, but we're going to be doing that anyway, aren't we? Why not stop when we get to the end, at least, and save ourselves the trouble of decompressing the rest and then throwing it away.
I was worried about changing the pg_lz code too much because it scared me, but debugging some stuff made me read it more closely so I fear it less now, and doing interior slices seems not unreasonable, so I will give it a try.