Re: [HACKERS] generic LONG VARLENA

wieck@debis.com (Jan Wieck) writes:
>     first  I  want  to  summarize  some details, to see if we all
>     agree so far in the discussion.

I snipped everything I agreed with ;-)

>     - The implementation should be generic for all variable  size
>       types, but can be enabled/disabled per type.              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Per-type control doesn't strike me as interesting or useful.  If there
needs to be a control at all, which I doubt, per-table would be the
way to go.  But how many users will really say, "Oh yes, I *want* the
thing to fail if my tuple's too big!"?  I say: make it automatically
apply whenever needed, don't force users to think about it.

>     - The type specific functions call a central support function
>       to  get the usual VARLENA format, which is taken from a LRU
>       cache or fetched from  the  extension  relation.  They  are
>       responsible  for freeing the memory after they're done with
>       the value.

If we are going to do this, we ought also think about solving the
generic memory-leakage problem at the same time.  No point in having
to revisit all the same code later to deal with that issue.

>     I  don't  think  it  is  a  good idea to create the expansion
>     relation all the time. Some keyword in CREATE  TABLE,  and/or
>     another ALTER TABLE should do it instead, so the DB admin can
>     activate the LONG feature on a per table base as  needed.

I don't believe it.  See above: people will complain that it's a bug
that the system doesn't handle their long data values.  Saying "oh, you
have to turn it on" will not appease them.  My objection is really the
same as for the specialized LONG datatype: I do *not* want people to
have to put nonstandard junk into their database schema declarations
in order to activate this feature.  I think it should Just Work and
stay out of users' faces.

Creating the expansion relation isn't that big a deal, but if you
don't want to do it always, why not do it on first use?

>     Also  I  would  like to say that system relations cannot have
>     expansion relations.  At  least  not  until  we  have  enough
>     experience with this stuff.

I'd really, really, really like to have this work for rules, though.
Why shouldn't we allow it for system relations?  Most of the critical
ones have fixed-width tuples anyway, so it won't matter for them.

BTW, it strikes me we should drop the "lztext" special datatype, and
instead have compression automatically applied to any varlena that
we are contemplating putting out-of-line.  (If we're really lucky,
that saves us having to put the value out-of-line!)

>     Is  that  now  what we initially want to give a try? If so, I
>     would like to start soon to get the generic part ready  ASAP.
>     Others  could  then  join  in  and  contribute by adding LONG
>     support for all the VARLENA data types we have.

Yes, if we don't do it inside fastgetattr then there's a lot of code
that will have to change.

>     Would really be a big leap if we can get this finished for  a
>     reasonable number of VARLENA types by February 1.

The more I think about this the more I think that it's a bad, bad idea
to try to have it ready by Feb 1.  There's not really enough time to
get it right and test it.  I don't want to be putting out an unstable
release, and that's what I'm afraid we'll have if we try to rush in
such a major change as this.  Particularly when we have nontrivial
amounts of unfinished business elsewhere that we shouldn't neglect.
(Jan, do you really think you can make this happen *and* bring foreign
keys to a finished status before February?  If you are going to leave
stuff undone in foreign keys, I think you are making the wrong choice.)

Furthermore, we can save ourselves some time if we tackle this change
in combination with the fmgr revision and the memory-leak-elimination
issue.  We will be touching all the same per-data-type code for each
of these issues, so why not touch it once instead of several times?

In short, I like this design but I think we should plan it for 7.1.
        regards, tom lane