Re: [GENERAL] Bug and/or feature? Complex data types in tables...

[ moved to pg-hackers, since it's *way* off topic for -general ]

Michael Glaesemann <grzm@myrealbox.com> writes:
> On Jan 3, 2004, at 2:31 PM, Tom Lane wrote:
>> The thing we are missing (i.e., what makes it crash) is an internal
>> representation that allows a tuple to be embedded as a field of a 
>> larger
>> tuple.  I've looked at this a couple of times, and each time concluded
>> that it was more work than I could afford to spend at the moment.  The
>> support-such-as-it-is for tuple return values uses a structure that has
>> embedded pointers, and it doesn't make any effort to get rid of
>> out-of-line TOAST pointers within the tuple.  Neither one of those
>> things are acceptable for a tuple that's trying to act like a Datum.

> Would you mind explaining this a little more, or pointing me to where I 
> can learn more about this?

Well, to make composite data types into real first-class citizens, we
have to be able to represent their values as ordinary Datums that don't
act differently from run-of-the-mill Datums, except when some operation
that actually wants to understand the contents of the value is invoked.
I think the only workable representation is as a variable-length datum
along the lines of
    int32 length word (overall length of datum)    OID type indicator (OID of the composite type)    header fields
similarto a normal on-disk tuple    null bitmap if needed    values of fields (themselves also Datums)
 

It's possible we could leave out the type OID, but given that we found
it useful to include an element type OID in array headers, I'm betting
we want one for composite-type values too.  Without it, we must always
know the exact composite type makeup from context.  (But see below.)

Now, this structure could be TOASTed as a whole, since it's just a
varlena data type.  But we cannot expect the toasting routines to look
inside it --- that would imply that it's not like other varlena data
types after all.  That means that the contained fields had better not be
out-of-line TOAST value references, because there's no way to keep track
of them and keep from deleting the referenced value too soon.  (It would
be workable for them to be compressed inline, but I suspect we don't
really want that either, since it'd interfere with attempts to compress
the overall datum.)  So somehow we'd need to expand out any toasted
component fields, at least before attempting to store such a datum on
disk.  Not sure where to do that cleanly.

The other point was that what's actually returned at the moment from a
function-returning-tuple is a Datum that contains a pointer to a
TupleTableSlot, not a pointer to a datum of this kind.  (Look in
executor/functions.c and executor/execQual.c to see the related code.)
We'd need to change that API, which would be a good thing to do, but
it'd no doubt break some user-written functions.

In particular I do not know how we'd handle functions declared to return
RECORD --- in general, they may need to return composite types that are
defined on-the-fly and don't have any associated type OID.  The
TupleTableSlot convention works for this since it can include a
tupledescriptor that was built on the fly.  We can't have tupdescs
embedded in datums stored on disk, however.
        regards, tom lane