Re: [HACKERS] Binary cursor header changed from 20 to 16 Bytes?

"G. Anthony Reina" <reina@nsi.edu> writes:
> Tom Lane wrote:
>> There is *no* header overhead for binary data as far as libpq or the
>> FE/BE protocol is concerned; what you get from PQgetvalue() is just
>> a pointer to whatever the backend's internal representation of the
>> data type is.  It's certainly possible for particular data types to
>> change representation from time to time, though I didn't recall anyone
>> planning such a thing for 6.5.  What data type is the column you're
>> retrieving, anyway?  (I'm guessing float4 array, perhaps?)  What kind
>> of platform is the backend running on?
>> 
>> regards, tom lane

> Right on the money. The column being retrieved is a float4 array. I am
> running the backend on a Red Hat Linux 6.0 machine (Pentium II / 400
> MHz / 512 Meg RAM / 128 Meg Shared buffers). The clients are all SGI
> machines (O2, Impact, and Indy).

OK, I think I see what is going on here.  If you look at the
declarations for arrays in src/include/utils/array.h, the overhead for
a one-dimensional array is 5 * sizeof(int) (total object size, #dims,
flags word, lo bound, hi bound) rounded up to the next MAXALIGN()
boundary to ensure that the array element data is aligned safely.

I was not quite right about the libpq presentation of a binary cursor
being the same as the backend's internal representation.  Actually,
it seems that the length word is stripped off --- all variable-size
datatypes are required to start with a word that is the total object
size, and what you get from libpq is a pointer to the word after that.

So, what you should have been measuring was 4*sizeof(int) plus
the array alignment padding, if any.

6.4 had some hardwired assumptions about compiler alignment behavior,
which were giving us grief on platforms that didn't conform, so as of
6.5 we determine the actual alignment properties of the
compiler/platform during configure.  The old code used to be aligning
the array overhead to a multiple of 8 whether that was appropriate or
not, but the new code is only aligning to the largest alignment multiple
actually observed on the target platform.  Evidently that's just 4 on
your system.

Bottom line: yes, the change from 20 to 16 is likely to persist.

I think there is actually a bug in the way libpq is doing this, because
it is allocating space for the stored varlena object minus the total-
size word.  This means that any internal alignment assumptions are *not*
being respected --- for example, in a machine that does need MAXALIGN
of 8, the client-side representation of an array object will fail to
have its array members aligned at a multiple-of-8 address.  libpq ought
to allocate space for and store the whole varlena object including
length word, the same as it is in the backend, so that internal fields
of the varlena will have the same alignment as in the backend.  Will put
this on my todo list.

> The clients are all SGI machines (O2, Impact, and Indy).

You realize, of course, that using a binary cursor in a cross-platform
environment is a fairly dangerous thing to do.  Any cross-machine
discrepancies in data format or alignment become your problem...
        regards, tom lane