Re: [HACKERS] cidr

Replies to 5 messages contained below.

> Date: Tue, 21 Jul 1998 10:00:03 -0400 (EDT)
> From: "Matthew N. Dodd" <winter@jurai.net>
>
> I don't see a problem with having a separate type for /32's.  It doesn't
> hurt anything, and it takes up less room that a CIDR.  When you've got
> several million records this becomes an issue.  (Not from a perspective of
> space, but more data requires more time to muck through during queries.)
>
> Plus, it would enable me to use my existing data without reloading.
> (ignoring the fact that 6.4 will probably require this.)

It's a tradeoff.  If one byte of prefix-length which adds 2MB of storage to
a 2-million record table (which probably takes 3GB to store anyway due to
the other fields and the metadata) is too much, then let's make a separate
type for hosts as you suggest.  But we're headed down an icky sticky path,
which is separate types for IPv4 hosts, IPv4 CIDR blocks which can be hosts,
IPv6 hosts, and IPv6 CIDR blocks which can be hosts.  This seems too sticky
and too icky for me -- I prefer polymorphism in 4GL's since I want to talk
about what I mean and let the computer figure out how to represent/store it.

In that sense I would argue for a variable width "int" type rather than a
bunch of different "int2", "int4" etc types.  (Too late, I know.)  Though
in the case of IPv6 I don't think enough is yet known about address formats
(RFC 1884 was for example just rewritten, and I'm not sure the IETF is done
messing with that stuff given what I know about the DNAME plans) and so I'd
argue that putting the address family into the internal representation and
then not supporting anything but IPv4 at this time -- basically what the
implementation I posted here two days ago does -- is the practical short
term thing to do.

> Date: Tue, 21 Jul 1998 10:35:21 -0400
> From: Nick Bastin <nbastin@rbbsystems.com>
>
> Making it IPv4 only just means we'll have to do it again later, and having
> IPv6 functionality now would be good for those of us who are currently
> working with IPv6 networks...

That's either an argument for implementing an IPv6 type immediately, or an
argument for polymorphism in a single AF-independent type.  Can you be more
specific?  My view of IPv6, as expressed above, is "let's leave room in the
type's internal representation but otherwise not worry about IPv6 right now."

> From: Bruce Momjian <maillist@candle.pha.pa.us>
> Date: Tue, 21 Jul 1998 10:40:58 -0400 (EDT)
>
> I say stick with IPv4 at this point.  We can always change it in future
> upgrades.  dump/reload will handle any changes in the internal format.

As expressed above, I agree with this viewpoint.

> We can call it INET now, and change it to INET4/INET6 if we decide we
> want separate types for the two address types.

I can live with this approach, choking up no hairballs at all.

> > That means "192.7.34.21/24" is either (a) a syntax error or (b) equivilent
> > to "192.7.34/24".
>
> How do we store the netmask?  Is that a separate field?

There is no netmask.  In CIDR notation the "/nn" suffix just tells you how
to interpret the mantissa if it does not fall on an octet boundary.  Therefore
"204.152.184/21" is three bits shorter than "204.152.184".  There is no
provision in the CIDR universe for expressing a "netmask" since that would
be a mantissa longer than its "prefix".  CIDR is all about prefixes, it's
not just a shorthand for aggregating an <address,netmask> pair.  I can see
why you'd like to be able to use it as an aggregated <address,netmask> pair
but (a) that's not what it is and (b) this is not the time or place to invent
something new in the CIDR field -- that sort of work would and should begin
with an Internet Draft in the appropriate working group.

> > Can we wait and see if someone misses / asks for these before we make them?
>
> Suppose I want to retrieve only 'host' addresses.  How do we do that?

There's no way to do that with the type I posted here the other day.  There'd
be no problem adding the function you proposed, like LENGTH(cidr), and then
doing a SELECT...WHERE using that function -- but it would be an iterative
search, there's no way I can think of for the query optimizer to build up the
implicit trie you'd need to go directly to all prefixes of a certain length.

> > Perhaps we ought to make new type insertion easier since it's so cool?
>
> Yep, it is cool.  When the code is installed as a standard part of the
> backend, you have more facilities to install types.  There are examples
> of many other types in the include/catalog/*.h files, so you just pick
> one and duplicate the proper partsTrying to do that with an SQL
> statement is really messy, particularly because the standard types DON'T
> use SQL to install themselves.  You also must specify unique OIDs for
> these new entries.  Also, the terminology is not something that many
> people are familiar with, so a lot of it is having the user understand
> what they need to do.  The manuals do a pretty good job.  If you have
> any specific ideas, or things that got you confused that we should
> clearify, please let us know.

From a marketing standpoint, if user defined types are one of PostgreSQL's
unique features, then they ought to be so easy to add that we have hundreds
of them in ./contrib at any given time.  This means that most of the standard
ones should be installed using whatever technology is used to install new
contributed ones: because it will force that installation process to become
easier.  There's no reason to avoid new syntax for this since it's a new
thing -- if we can do CREATE FUNCTION and CREATE TYPE and CREATE OPERATOR
then why not CREATE BINDING to just associate various functions, by name
rather than by OID, with the magic index operator slots for that type?  Or
even extending the CREATE TYPE syntax to bind the indexing functions to the
type at the time of its creation?

> From: Bruce Momjian <maillist@candle.pha.pa.us>
> Date: Tue, 21 Jul 1998 10:45:45 -0400 (EDT)
>
> It is already being worked on by one of the ip_and_mac developers.  He
> is merging the types.  We need him to work on it because he understands
> PostgreSQL better.

Sounds great, I'll wait to hear from that person (if my help is needed.)

> Again, it is being worked on.  I don't think Paul wants to get into
> installing in into the main tree.  It is quite a job.  We may need to
> increase the max system oid to get us more available oids.

Actually I would have dived into this and thought it was fun, but doubtless
I would not have done as good or as quick a job as the ip_and_mac guys.

> >     And the type is to be a 'CIDR', which is the appropriate
> > terminology for what it is...those that need it, will know what it is
> > *shrug*
>
> I use IP addresses and didn't know.  I am also hoping we can allow
> storage of old and cidr types in the same type, at least superficially.

Sounds like conclusive evidence for calling this the INET type rather than
the CIDR type.  And if someone wants to make an INET32 type to account for
the case of millions of host-only (no prefix length needed) fields, so be it.

> From: Bruce Momjian <maillist@candle.pha.pa.us>
> Date: Tue, 21 Jul 1998 11:01:10 -0400 (EDT)
>
> > I don't see a problem with having a separate type for /32's.  It doesn't
> > hurt anything, and it takes up less room that a CIDR.  When you've got
> > several million records this becomes an issue.  (Not from a perspective of
> > space, but more data requires more time to muck through during queries.)
>
> I would like one type, and we can specifiy a way of pulling out just
> hosts or class addresses.

I also lean significantly in the direction of a single type for all of IPv4
rather than a separate INET32 type.

> > Plus, it would enable me to use my existing data without reloading.
> > (ignoring the fact that 6.4 will probably require this.)

I think there's no way to justify permanent engineering decisions on the basis
of a single reload operation or the avoidance of one.

> Yep.

> From: Bruce Momjian <maillist@candle.pha.pa.us>
> Date: Tue, 21 Jul 1998 11:02:11 -0400 (EDT)
>
> > Making it IPv4 only just means we'll have to do it again later, and having
> > IPv6 functionality now would be good for those of us who are currently
> > working with IPv6 networks...
>
> Oh.  OK.  We do have variable-length types.

My question is, do those types have an internal framing format with an outer
length and an inner opaque structure, or does each one have a "length"
accessor to which an opaque structure is passed?  In the former case, we'll
be burning more space on a length indicator even though the address family
and prefix length are in the opaque part of the structure.  In the latter
case, there's no big deal at all since given the address family and prefix
length, an accessor can tell the type system the size of a particular datum.