Re: [HACKERS] LONG varsize - how to go on

> How I would like to go on:
> 
>     What I've done so far is to prepare the HeapTuple handling so
>     it should be possible to  reallocate  the  t_data  part  from
>     inside the heap_insert() and heap_update().

OK, I assume this is so you can change the tuple size on the fly when
inserting the tuple.

> 
>     Next  would  be to define how the runtime configuration is to
>     be stored and configured. So I can get at least  the  minimum
>     required  system  catalog  changes made soon. If we leave out
>     auto-compression for now, a rellongrelid  (type  Oid  default
>     Invalid)  in  pg_class,  and  a attcanlong (type bool default
>     true) in pg_attribute would do it. This would give  space  to
>     enable  a  single relation for this feature, and then disable
>     single columns again. New utility commands will finally  gain
>     access  to  the  features.  Some  scripts  will  do it during
>     development, so the feature will not show up  -  thus  beeing
>     silently  unavailable to the user - until we want to ship it.

Got it.  You want to store the oid of the long table for the base table
inside the pg_class/Relation structure.  Good idea.

> 
>     I think these are the best places to  put  the  configuration
>     into,  because  the information would be already available at
>     no cost inside  heap  access  (rel->rd_rel->rellongrelid  and
>     rel->rd_att...).
> 

Yes, I recommand a macro so access is clear.  See
RelationGetRelationName for an example.

>     What  I  want  to  implement  for  now,  is to store extended
>     VARLENA  attributes  into  a  regular  (but  other   relkind)
>     relation  with  the  Oid  key  as discussed.  That will cause
>     minimum changes to VACUUM. If storage/retrieval of attributes
>     is  encapsulated  right,  it  could get replaced by something
>     different at any time in  the  future  when  we  have  enough
>     experience with this new technique.

Good.  I recommend calling it pg_* so it is automatically excluded from
client table lists.

> 
>     Christof Petig and me then could start implementing it, using
>     lztext with locally  disabled  compression  feature  for  the

I would recommand having compression disabled by default, and enabled by
the user.

>     basics.   I'll  not  commit  any  changes until after feature
>     freeze of the upcoming release. During the last changes  (for
>     HeapTuple  handling)  I've seen many places in the code, that
>     deal themself on VARSIZE/VARDATA with text  type  attributes,
>     which  then  must  use textout() instead (what IMHO is better
>     anyway). So implementing an ALL-varsize move off, instead  of
>     a  special LONG datatype, will take longer than February 1st.

OK, so we are going to see this after 7.0 is released, which is fine.  I
understand the concern about all the accesses to VARDATA() inside the
code.  That will clearly be difficult.

> 
>     This plan means in summary:
> 
>     1.  Full  possible  configurability  with   minimum   catalog
>         changes.
> 
>     2.  Hidden  without any side effects until we agree to enable
>         it.
> 
>     3.  Later optimizable storage of extended attributes.
> 
>     I can't see any reason to way much longer. Can we please have
>     a consensus to get started?

Sounds good.

--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026