On Thu, Jun 20, 2019 at 1:51 AM Binguo Bao <djydewang@gmail.com> wrote: > Hi hackers! > This proposal aims to provide the ability to de-TOAST a fully TOAST'd and compressed field using an iterator and then update the appropriate parts of the code to use the iterator where possible instead of de-TOAST'ing and de-compressing the entire value. Examples where this can be helpful include using position() from the beginning of the value, or doing a pattern or substring match. > > de-TOAST iterator overview: > 1. The caller requests the slice of the attribute value from the de-TOAST iterator. > 2. The de-TOAST iterator checks if there is a slice available in the output buffer, if there is, return the result directly, > otherwise goto the step3. > 3. The de-TOAST iterator checks if there is the slice available in the input buffer, if there is, goto step44. Otherwise, > call fetch_datum_iterator to fetch datums from disk to input buffer. > 4. If the data in the input buffer is compressed, extract some data from the input buffer to the output buffer until the caller's > needs are met. > > I've implemented the prototype and apply it to the position() function to test performance.
Hi Binguo,
Interesting work, and nice performance improvements so far. Just by the way, the patch currently generates warnings: