Re: Changing the query string max length

> Hi, Massimo,
> 
> You might have noticed my posting a couple of days back about the max query
> string length problem.  I was referred to you by Tom Lane, as he says that
> you've done some work in this area.  I'd like to sort this out, mainly
> because I'd like to get involved in the development, and this is about as
> good a place to start as any.  So, some question, if you have some time:
> 
> Why is MAX_QUERY_SIZE dependent on BLCKSZ?

The implicit assumption is that the maximum query size is correlated to the
tuple size which is slightly less than block size. A query size twice the
tuple size seems reasonable in most cases, unless you insert very complex
data with a lot of escapes, function calls or constants with large ascii
output formats like floats.

> Are there dependencies that I should be aware of when trying to 
>     adjust this?  Not dependencies on MAX_QUERY_SIZE, 
>     but rather dependencies _of_ MAX_QUERY_SIZE.

You should check the buffer size in the parser and egcp. I made also a change
into some Makefile to set the YY_BUF_SIZE used by lex to 65536. This is
really a kludge but I didn't find a better way to do it. It seems that the
libpq buffer size is not dependent on BLCKSZ.

> Which areas of the system, that you are aware of, will require 
>     changing, or at least checking, to ensure that they work?

Except the above YY_BUF_SIZE you shouldn't need to change anything.
I turned all references to the original `8192' to references to BLCKSZ,
so changing BLCKSZ should automatically adjust all the other constants.
Try to grep '#define.*BLCKSZ' in the sources and see what is depending on
BLCKSZ. I'm currently working with a backend compiled with BLCKSZ=32768
and it works fine for me.

> 
> Any other hints, tips, and otherwise would be much appreciated.

A better way would be to allocate and grow query buffers dynamically while
reading the query but you will anyway have troubles with lex and yacc which
use statically allocated buffers whose size is hardwired in the program.
This is why I had to make those ugly changes in the Makefiles.

> 
> MikeA
> 
> --------------------------------------------------------------------
> Science is the game we play with God to find out what his rules are.
> --------------------------------------------------------------------
> 
> [(LI)U]NIX IS user friendly; it's just picky about who its friends are.
> 

-- 
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto               email: dz@cs.unitn.it               |
|  Via Marconi, 141                phone: ++39-0461534251              |
|  38057 Pergine Valsugana (TN)      www: http://www.cs.unitn.it/~dz/  |
|  Italy                             pgp: finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+