Re: Re: COPY BINARY file format proposal

Philip Warner <pjw@rhyme.com.au> writes:
> I don't want to continue being picky, but you could just use 4 bytes for a
> maj-min-rev-patch version number (in that order), and avoid the endian
> issues by reading and writing each byte. No big deal, though.

Well, the thing is that we need to protect the contents of
datatype-specific structures.  If it were just a matter of byte-flipping
the counts and lengths defined by the (proposed) file format, I'd have
specified that we write 'em all in network byte order and be done with
it.  But knowing the internal structure of every datatype in the system
is a very different game, and I don't want to try to play that game ...
at least not yet.  So the proposal is just to identify the endianness
that the file is being written with.  Recovering the data on a machine
of different endianness is a project for future data archeologists.

>> This allows for both backwards-compatible header additions (extend the
>> header without changing the version number) and non-backwards-compatible
>> changes (bump the version number).

> That's where the rev & patch levels help if you adopt the above version
> numbering - 1.0-** should should all be compatibile, 1.1 should be able to
> read <= 1.1-**, 1.0-** should not be expected to read 1.1-** etc.

Tell you the truth, I don't believe in file-format version numbers at
all.  My experience with such things is that they defeat portability
rather than promote it, because readers tend to reject files that they
could have actually have read as a result of insignificant version number
issues.  You can read all about my view of this issue in the PNG spec
(RFC 2083, esp section 12.13) --- the versioning philosophy described
there is largely yours truly's.

I will not complain about sticking a "version 1.0" field into a format
when there is no real intention of changing it in the future ... but
assigning deep significance to major/minor numbers, or something like
that, is wrongheaded.  You need a much finer-grained view of
compatibility issues than that if you want to achieve anything much
in cross-version compatibility.  Feature-based versioning, like PNG's
notion of critical vs. ancillary chunks, is the thing you need for
that.  I didn't bring up the issue in this morning's proposal --- but
if we ever do add stuff to the proposed extensible header, I will hold
out for self-identifying feature-related items much like PNG chunks.
        regards, tom lane