Обсуждение: [HACKERS] Max size of data types and tuples. (fwd)
Here's a blast from the past. Shows how I keep those open issues in my mailbox. Forwarded message: > From scrappy@postgreSQL.org Wed Jan 29 14:49:26 1997 > X-Authentication-Warning: hub.org: pgsql set sender to owner-pg95-dev@postgreSQL.org using -f > Date: Wed, 29 Jan 1997 13:38:10 -0500 > From: aixssd!darrenk@abs.net (Darren King) > Message-Id: <9701291838.AA22296@ceodev> > To: hackers@postgreSQL.org > Subject: [HACKERS] Max size of data types and tuples. > Mime-Version: 1.0 > Content-Type: text/plain; charset=US-ASCII > Content-Transfer-Encoding: 7bit > Content-Md5: 9bBKVeeTrq97EhvkS6qT3A== > Sender: owner-pg95-dev@hub.org > Reply-To: hackers@hub.org, aixssd!darrenk@abs.net (Darren King) > > > Does anyone know of a valid reason for the 4096 byte limit > on the textual fields (char, varchar, text, etc...)? > > Could it be bumped to (MAXTUPLEN - sizeof(int)) in the parser > and in utils/adt/varchar.c? MAXTUPLEN would have to be > calculated more accurately though as noted in the comment > around its definition. > > > The following are just some of my observations and comments > on things that I think should be cleaned up a little in > the source (and I'll volunteer to do them). > > 1. Clean up the #defines for the max block size. Currently, > there are at least four references to 8192... > > include/config.h > include/storage/bufmgr.h > include/optimizer/internal.h > backend/rewrite/rewriteDefine.c > > The _PAGE_SIZE_ in internal.h is _not_ used anywhere in the > optimizer. This define should be linked to BLCKSZ, but would > be better to remove it and just use BLCKSZ. > > optimizer/costsize.c includes storage/bufpage.h for BLCKSZ, > but in fact this is defined in config.h!. Also included in > > executor/nodeHash.c > executor/nodeHashjoin.c > utils/sort/psort.c > > __These includes of storage/bufpage.h can be removed.__ > > > There should be #define MAX_BLOCK_SIZE 8192 > #define CURRENT_BLOCK_SIZE 8192 > > The MAX_BLOCK_SIZE is a hard and fast limit since only 13 bits > can be used in offsets and the like. I believe that in the > future, PostgreSql should support block sizes other than 8k, > like 2k and 4k. Long-term goal, but the code should be done > to allow this at a future time. The comments in storage/bufpage.h > suggest to me that the original designers of Postgres had this > in mind. > > > 2. Once the block size issue is taken care of, calculate the > maximum tuple size more accurately. > > > 3. When #1 & #2 are resolved, let the textual fields have a max > of (MAX_TUPLE_SIZE - sizeof(int)). > > > 4. Since only 13 bits are needed for storing the size of these > textual fields in a tuple, could PostgreSql use a 16-bit int to > store it? Currently, the size is padded to four bytes in the > tuple and this eats space if you have many textual fields. > Without further digging, I'm assuming that the size is double-word > aligned so that the actual text starts on a double-word boundary. > > Thanks for reading this far. :) > > Comments, suggestions most welcome... > > > Darren darrenk@insightdist.com > -- Bruce Momjian maillist@candle.pha.pa.us
Re: [HACKERS] Max size of data types and tuples. (fwd)
От
darrenk@insightdist.com (Darren King)
Дата:
> > Here's a blast from the past. Shows how I keep those open issues in my > mailbox. > > Forwarded message: > > Date: Wed, 29 Jan 1997 13:38:10 -0500 > > From: aixssd!darrenk@abs.net (Darren King) > > To: hackers@postgreSQL.org > > Subject: [HACKERS] Max size of data types and tuples. Still buried in my 'received' box here too. Can't imagine all the bugs and/or issues you have kept in yours. > > 1. Clean up the #defines for the max block size. Currently, > > there are at least four references to 8192... Think I found and fixed all of these up. > > __These includes of storage/bufpage.h can be removed.__ Still _quite_ a few #includes that can be removed throughout. First, "utils/elog.h" and "util/palloc.h" are include in "postgres.h", so are unnecessary to include by themselves since "postgres.h" is include in _every_ .c file, correct? Also numerous #includes of "storage/bufpage.h" and "storage/fd.h" that are unnecessary since the things they were included for (BLCKSZ and SEEK_*) are now either in "config.h" or found in a system include file. > > 2. Once the block size issue is taken care of, calculate the > > maximum tuple size more accurately. ... > > 3. When #1 & #2 are resolved, let the textual fields have a max > > of (MAX_TUPLE_SIZE - sizeof(int)). This could be done as soon as I come up with a way of defining the packet size for the interfaces since this is the newest limiting factor. Peter's suggestion of backend functions for getting info might be the way to go. It would let the various interfaces get the info they need and would be a step towards JDBC and ODBC compliance. darrenk
> > > > > Here's a blast from the past. Shows how I keep those open issues in my > > mailbox. > > > > Forwarded message: > > > Date: Wed, 29 Jan 1997 13:38:10 -0500 > > > From: aixssd!darrenk@abs.net (Darren King) > > > To: hackers@postgreSQL.org > > > Subject: [HACKERS] Max size of data types and tuples. > > Still buried in my 'received' box here too. Can't imagine all the bugs > and/or issues you have kept in yours. Not too bad now. > > > > > 1. Clean up the #defines for the max block size. Currently, > > > there are at least four references to 8192... > > Think I found and fixed all of these up. > > > > > __These includes of storage/bufpage.h can be removed.__ > > Still _quite_ a few #includes that can be removed throughout. First, > "utils/elog.h" and "util/palloc.h" are include in "postgres.h", so are > unnecessary to include by themselves since "postgres.h" is include in > _every_ .c file, correct? Yes, must be included. Period. Even 3rd party apps. > > Also numerous #includes of "storage/bufpage.h" and "storage/fd.h" that are > unnecessary since the things they were included for (BLCKSZ and SEEK_*) are > now either in "config.h" or found in a system include file. > > > > > 2. Once the block size issue is taken care of, calculate the > > > maximum tuple size more accurately. > ... > > > 3. When #1 & #2 are resolved, let the textual fields have a max > > > of (MAX_TUPLE_SIZE - sizeof(int)). > > This could be done as soon as I come up with a way of defining the packet > size for the interfaces since this is the newest limiting factor. > > Peter's suggestion of backend functions for getting info might be the way to > go. It would let the various interfaces get the info they need and would be > a step towards JDBC and ODBC compliance. Again, we could just set 3rd party apps to be the maximum tuple size we will ever have to support. -- Bruce Momjian maillist@candle.pha.pa.us
On Wed, 14 Jan 1998, Bruce Momjian wrote: > > This could be done as soon as I come up with a way of defining the packet > > size for the interfaces since this is the newest limiting factor. > > > > Peter's suggestion of backend functions for getting info might be the way to > > go. It would let the various interfaces get the info they need and would be > > a step towards JDBC and ODBC compliance. > > Again, we could just set 3rd party apps to be the maximum tuple size we > will ever have to support. Currently, were returning some defaults based on the 8K block size. Probably for these, we may be able to get away with the values we are setting. However, there are a few things that I think we will still need to implement as functions. -- Peter T Mount petermount@earthling.net or pmount@maidast.demon.co.uk Main Homepage: http://www.demon.co.uk/finder Work Homepage: http://www.maidstone.gov.uk Work EMail: peter@maidstone.gov.uk
> > On Wed, 14 Jan 1998, Bruce Momjian wrote: > > > > This could be done as soon as I come up with a way of defining the packet > > > size for the interfaces since this is the newest limiting factor. > > > > > > Peter's suggestion of backend functions for getting info might be the way to > > > go. It would let the various interfaces get the info they need and would be > > > a step towards JDBC and ODBC compliance. > > > > Again, we could just set 3rd party apps to be the maximum tuple size we > > will ever have to support. > > Currently, were returning some defaults based on the 8K block size. > > Probably for these, we may be able to get away with the values we are > setting. However, there are a few things that I think we will still need > to implement as functions. OK, let's decide soon, so people can be ready for Feb 1. -- Bruce Momjian maillist@candle.pha.pa.us
On Wed, 14 Jan 1998, Bruce Momjian wrote: > > On Wed, 14 Jan 1998, Bruce Momjian wrote: > > > > > > This could be done as soon as I come up with a way of defining the packet > > > > size for the interfaces since this is the newest limiting factor. > > > > > > > > Peter's suggestion of backend functions for getting info might be the way to > > > > go. It would let the various interfaces get the info they need and would be > > > > a step towards JDBC and ODBC compliance. > > > > > > Again, we could just set 3rd party apps to be the maximum tuple size we > > > will ever have to support. > > > > Currently, were returning some defaults based on the 8K block size. > > > > Probably for these, we may be able to get away with the values we are > > setting. However, there are a few things that I think we will still need > > to implement as functions. > > OK, let's decide soon, so people can be ready for Feb 1. I'm going to sort out what needs to be done to get us as close to compliance as possible over the next couple of days. Hopefully, we can decide on some of them then. -- Peter T Mount petermount@earthling.net or pmount@maidast.demon.co.uk Main Homepage: http://www.demon.co.uk/finder Work Homepage: http://www.maidstone.gov.uk Work EMail: peter@maidstone.gov.uk
Are we done with these issues, or are you still working on them, or is Peter working on this? > > > > > > > > > Here's a blast from the past. Shows how I keep those open issues in my > > > mailbox. > > > > > > Forwarded message: > > > > Date: Wed, 29 Jan 1997 13:38:10 -0500 > > > > From: aixssd!darrenk@abs.net (Darren King) > > > > To: hackers@postgreSQL.org > > > > Subject: [HACKERS] Max size of data types and tuples. > > > > Still buried in my 'received' box here too. Can't imagine all the bugs > > and/or issues you have kept in yours. > > Not too bad now. > > > > > > > > > 1. Clean up the #defines for the max block size. Currently, > > > > there are at least four references to 8192... > > > > Think I found and fixed all of these up. > > > > > > > > __These includes of storage/bufpage.h can be removed.__ > > > > Still _quite_ a few #includes that can be removed throughout. First, > > "utils/elog.h" and "util/palloc.h" are include in "postgres.h", so are > > unnecessary to include by themselves since "postgres.h" is include in > > _every_ .c file, correct? > > Yes, must be included. Period. Even 3rd party apps. > > > > > Also numerous #includes of "storage/bufpage.h" and "storage/fd.h" that are > > unnecessary since the things they were included for (BLCKSZ and SEEK_*) are > > now either in "config.h" or found in a system include file. > > > > > > > > 2. Once the block size issue is taken care of, calculate the > > > > maximum tuple size more accurately. > > ... > > > > 3. When #1 & #2 are resolved, let the textual fields have a max > > > > of (MAX_TUPLE_SIZE - sizeof(int)). > > > > This could be done as soon as I come up with a way of defining the packet > > size for the interfaces since this is the newest limiting factor. > > > > Peter's suggestion of backend functions for getting info might be the way to > > go. It would let the various interfaces get the info they need and would be > > a step towards JDBC and ODBC compliance. > > Again, we could just set 3rd party apps to be the maximum tuple size we > will ever have to support. > > > -- > Bruce Momjian > maillist@candle.pha.pa.us > > -- Bruce Momjian maillist@candle.pha.pa.us