Re: Toast compression method options

On Fri, Jun 18, 2021 at 12:13 PM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Thu, May 06, 2021 at 07:23:48PM +0530, Dilip Kumar wrote:
> > I have fixed some comments offlist reported by Justin.  Apart from
> > that, I have also improved documentation and test case.  Stil it has a
> > lot of cleanup to be done but I am not planning to do that
> > immediately.
>
> I was testing the various compression algos we touch in core, and I am
> not really convinced that we need more code to control that.  First,
> pglz is living as-is in the backend code for a very long time and no
> one has expressed an interest in controlling the compression strategy
> used AFAIK.  On top of that, LZ4 outclasses it easily, so if there is
> a need to worry about performance with the TOAST data, users could
> just move to use LZ4.
>
> +       if (strcmp(def->defname, "acceleration") == 0)
> +       {
> +           int32 acceleration =
> +               pg_atoi(defGetString(def), sizeof(acceleration), 0);
> +
> +           if (acceleration < INT32_MIN || acceleration > INT32_MAX)
> +               ereport(ERROR,
> +                   (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> +                    errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
> +                               defGetString(def)),
> +                    errhint("expected value between INT32_MIN and
> INT32_MAX")
> +                   ));
>
> Then comes the part with LZ4 and its acceleration.  The default
> compression level used by LZ4 compresses data the most, and it is
> already pretty cheap in CPU.  Do you have cases where this would be
> useful?  Increasing the acceleration reduces the compression to be
> close to zero, but if one cares about the compression cost, he/she
> could fall back to the external storage for basically the same
> effect.  Is there really a use-case for something in-between?

IMHO there is certainly a use case, basically, if we compress the data
so that we can avoid storing it externally.  Now suppose for some
data, with default LZ4 compression, the compression ratio is so high
that you are able to compress to the size which is way under the
limit.  So for such data, the acceleration can be increased such that
compression is fast and compression ratio is good enough that it is
not going to the external storage.  I agree it will be difficult for
the user to make such a decision and select the acceleration value but
based on the data pattern and their compressed length the admin can
make such a decision.  So in short select the acceleration value such
that you can compress it fast and the compression ratio is good enough
to keep it from storing externally.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com