Обсуждение: still Query Limits to 8K ?

hi!

I just checked postgres 6.4 and I have to tell you it was a great work!!!
it's 3-4 times faster than 6.3.2 was. more stable. 
I have a little question:
the TEXT fieldtype how many data can store?
8192 bytes?

I also wonder what's the reason to limit a query to 8192bytes maximum..
is there any serious problem to have longer queries?

it could be good to have more than 8K fields (without lo_import and so on)
for example 64K...

this is would be good sometimes to store texts...
regards,        redax

.----------------------------------------------------------.
|Zsolt Varga            | tel/fax:   +36 36 422811         |
| AgriaComputer LTD     | email:     redax@agria.hu        |
| System Administrator  | URL:       http://www.agria.hu/  |
`----------------------------------------------------------'

> I also wonder what's the reason to limit a query to 8192bytes 
> maximum..
> is there any serious problem to have longer queries?

I would think you could find and increase the size of string buffers
used in psql, libpq, and the backend.

> it could be good to have more than 8K fields (without lo_import and so 
> on) for example 64K...

And Darren K (?) worked on parameterizing the maximum allowed tuple
size. You may just be able to increase it, rebuild, and reinstall.
Perhaps you could check the mail archives to see where it was left.
                    - Tom

Zsolt Varga <redax@agria.hu> writes:
> the TEXT fieldtype how many data can store?
> 8192 bytes?

Yup, same as it ever was.  (Actually a bit less than 8K, since you have
to subtract the size of the tuple overhead, not to mention any other
fields that might be in the same tuple.)  In reality you probably don't
want to design a database in which tuples are bigger than a K or so.
If your tuples run 4K or more then you can only fit one to a disk page,
which means you waste a lot of disk space.

There has been talk of allowing tuples to span multiple disk pages,
but it's not a simple change.

> I also wonder what's the reason to limit a query to 8192bytes maximum..
> is there any serious problem to have longer queries?

That's a completely separate issue from the maximum tuple size on disk.
It's driven by the size of the text buffers used to hold the current
query.

Offhand I think it would not be hard to fix libpq and the backend to
not have any particular limit on the textual length of a query.  They'd
still need to have buffers holding the query, but they could realloc()
the buffers bigger as needed.  (So, for example, libpq's buffer size
would be proportional to the longest query you'd asked in a given
session.  But the initial buffer size could be much smaller than 8K,
so for the average frontend app this would actually save memory.)

One thing I do not know is whether we'd start to run into any hard-wired
limits in the parser/planner/executor with very large (complex) queries
... Thomas, would the parser go down if you handed it a SELECT with a
few thousand OR clauses?

I'd be willing to fix libpq and the backend as far as the parser input
buffer, if someone else will take responsibility for looking at any
hard-wired limits that might exist downstream of that.
        regards, tom lane

> One thing I do not know is whether we'd start to run into any 
> hard-wired limits in the parser/planner/executor with very large 
> (complex) queries ... Thomas, would the parser go down if you handed 
> it a SELECT with a few thousand OR clauses?

I'm not recalling any specific limitation here. And if there is one,
then we should parameterize it so it is easy to find and easy to pump
up.

> I'd be willing to fix libpq and the backend as far as the parser input
> buffer, if someone else will take responsibility for looking at any
> hard-wired limits that might exist downstream of that.

Sure, parameterize everything (if it isn't done already) and then we can
try pumping it up and see what breaks. I'll push on any problems we find
deeper inside the backend.
                     - Tom

Tom Lane wrote:
> One thing I do not know is whether we'd start to run into any hard-wired
> limits in the parser/planner/executor with very large (complex) queries
> ... Thomas, would the parser go down if you handed it a SELECT with a
> few thousand OR clauses?

Yep, I know one:

include/utils/elog.h:32:#define ELOG_MAXLEN 4096

I made a patch before 6.4, which was backed out because
I used vsnprintf which wasn't available everywhere.

This is still to short, suggests we do this for 6.4.1:

#define ELOG_MAXLEN 8192

And use my patch (with fixes) for 6.5.
regards,
-- 
-----------------
Göran Thyni
http://kirra.net/