Обсуждение: SELECT with row>32k hangs over SSL-Connection
hi, i have a table with test_field of type "TEXT". when i do: select test_field from test where id=1; the connection (with psql) hangs and no output is received, so i have to kill psql. this occurs, at first sight, if length(test_field)>32748. i think, in more general, the problem occurs, when the (or one) returned row is >32k. i can do the same without any problems, if i connect without ssl or via local socket. the problem also occurs with DBI. so, i think it's not a psql issue. i'm using Postgres 7.3.4 on slackware 9.0 with openssl 0.9.7c the workaround to disable ssl is not really sufficient :-( i will take a deeper look at it, if it's not a known problem. best regards, Christian Wetzig
On Tue, Oct 28, 2003 at 12:37:29 +0100, Christian Wetzig <postgres@wetzig.de> wrote: > > i can do the same without any problems, if i connect without ssl or via > local socket. the problem also occurs with DBI. so, i think it's not a > psql issue. > > i'm using Postgres 7.3.4 on slackware 9.0 with openssl 0.9.7c > > the workaround to disable ssl is not really sufficient :-( > > i will take a deeper look at it, if it's not a known problem. I am pretty sure it is a known problem. You could try getting post 7.3.4 updates to 7.3 from CVS or try 7.4beta5 to see if the problem is fixed.
On Tue, Oct 28, 2003 at 15:23:51 +0100, Christian Wetzig <postgres@wetzig.de> wrote: > Bruno Wolff III wrote: > >On Tue, Oct 28, 2003 at 12:37:29 +0100, > > Christian Wetzig <postgres@wetzig.de> wrote: > >>i will take a deeper look at it, if it's not a known problem. > > > > > >I am pretty sure it is a known problem. You could try getting post 7.3.4 > >updates to 7.3 from CVS or try 7.4beta5 to see if the problem is fixed. > > you're right, problem seems to be gone in 7.4beta5, which will hopefully > enter release status soon ;-) > > though a patch for 7.3 or a 7.3.5 would be nice, so there is no need for > dump/restore I think that a 7.3.5 release should be done a bit after 7.4 is released, but my opinion doesn't count for much.
Bruno Wolff III <bruno@wolff.to> writes: > I think that a 7.3.5 release should be done a bit after 7.4 is released, > but my opinion doesn't count for much. Yeah, I think we have accumulated enough changes in the 7.3 branch to justify a 7.3.5, but I'm not sure when we'll get around to it. The SSL fixes are already in that branch, so what I'd recommend to Christian right at the moment is to fetch the tip of the REL7_3_STABLE branch from our CVS server and use that. regards, tom lane
Tom Lane wrote: > Bruno Wolff III <bruno@wolff.to> writes: >> I think that a 7.3.5 release should be done a bit after 7.4 is released, >> but my opinion doesn't count for much. > > Yeah, I think we have accumulated enough changes in the 7.3 branch to > justify a 7.3.5, but I'm not sure when we'll get around to it. On 10/03/2003 Bruce was the only one responding to my question if the namespace fix I had for PL/Tcl should be backpatched into 7.3.5. He claimed that we'll probably not release any 7.3.X any more and we dropped the issue. Guess the question is open again then. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Jan Wieck <JanWieck@Yahoo.com> writes: > Tom Lane wrote: >> Yeah, I think we have accumulated enough changes in the 7.3 branch to >> justify a 7.3.5, but I'm not sure when we'll get around to it. > On 10/03/2003 Bruce was the only one responding to my question if the > namespace fix I had for PL/Tcl should be backpatched into 7.3.5. He > claimed that we'll probably not release any 7.3.X any more and we > dropped the issue. > Guess the question is open again then. I'm on the fence right now, but one or two more fixes in the 7.3 branch will be enough to make me feel we should put out 7.3.5. If you are confident of that namespace fix, then I'd say by all means commit it into the 7.3 branch so it will be there when 7.3.5 happens. Attached are the current CVS log entries for post-7.3.4 changes in REL7_3_STABLE. What do you think, is it time yet? regards, tom lane 2003-10-20 16:01 tgl * src/backend/rewrite/: rewriteManip.c (REL7_3_STABLE), rewriteManip.c: It is possible for ResolveNew to be used to insert a sublink into a subquery that didn't previously have one. We have traditionally made the caller of ResolveNew responsible for updating the hasSubLinks flag of the outermost query, but this fails to account for hasSubLinks in subqueries. Fix ResolveNew to handle this. We might later want to change the calling convention of ResolveNew so that it can fix the outer query too, simplifying callers. But I went with the localized fix for now. Per bug report from J Smith, 20-Oct-03. 2003-10-02 18:25 tgl * src/backend/utils/adt/ruleutils.c (REL7_3_STABLE): When dumping CREATE INDEX, must show opclass name if the opclass isn't in the schema search path. Otherwise pg_dump doesn't correctly dump scenarios where a custom opclass is created in 'public' and then used by indexes in other schemas. 2003-09-29 14:53 momjian * src/bin/scripts/clusterdb (REL7_3_STABLE): [ Patch applied only to 7.3.X.] Hi There's a bug in the clusterdb script where it looks like the arguments to the psql command are being passed in the wrong order, so it fails when you run it on a database that is not on localhost. Here's the output from the command: 133 anands-Computer:bin/scripts> clusterdb -h wooster -U rr granada psql: warning: extra option wooster ignored psql: warning: extra option -U ignored psql: warning: extra option rr ignored psql: warning: extra option -F: ignored psql: warning: extra option -P ignored psql: warning: extra option format=unaligned ignored psql: warning: extra option -t ignored psql: warning: extra option -c ignored psql: warning: extra option SELECT nspname, pg_class.relname, pg_class_2.relname FROM pg_class, pg_class AS pg_class_2 JOIN pg_namespace ON (pg_namespace.oid=relnamespace), pg_index WHERE pg_class.oid=pg_index.indrelid AND pg_class_2.oid=pg_index.indexrelid AND pg_index.indisclustered AND pg_class.relowner=(SELECT usesysid FROM pg_user WHERE usename=current_user) ignored psql: FATAL: user "-h" does not exist I'm attaching a patch that fixes the problem. The diff was run on postgresql 7.3.4 Thanks a lot. Anand Ranganathan 2003-09-28 13:46 wieck * src/bin/pg_dump/pg_dump.c (REL7_3_STABLE): Backpatched changes for rules when casts are dumped according to discussion on hackers. Jan 2003-09-23 11:11 tgl * src/backend/executor/spi.c (REL7_3_STABLE): _SPI_cursor_operation forgot to check for failure return from _SPI_begin_call. Per gripe from Tomasz Myrta. 2003-09-17 14:40 tgl * src/pl/plpython/plpython.c (REL7_3_STABLE): Back-patch fix for plpython problems with dropped table columns; per bug report from Arthur Ward, who also tested this patch. 2003-09-03 15:01 tgl * src/backend/utils/adt/formatting.c (REL7_3_STABLE): Back-patch the other part of Karel's formatting bug fix. 2003-09-03 11:00 tgl * src/backend/utils/adt/formatting.c (REL7_3_STABLE): Repair problems with to_char() overrunning its input string. From Karel Zak. 2003-08-24 17:26 petere * src/bin/psql/po/de.po (REL7_3_STABLE): Fix translation mistake. 2003-08-24 01:13 ishii * src/backend/utils/mb/Unicode/gb18030_to_utf8.map (REL7_3_STABLE): Fix GB18030 to UTF-8 mapping table 2003-08-24 01:00 ishii * src/backend/utils/mb/Unicode/UCS_to_GB18030.pl (REL7_3_STABLE): Fix bug in GB18030 conversion script 2003-08-22 17:57 tgl * src/interfaces/libpq/fe-secure.c (REL7_3_STABLE): Sigh, I'm an idiot ... SSL_ERROR_WANT_READ isn't an error condition at all, it just means 'no data available yet'. 2003-08-08 11:49 tgl * src/backend/utils/mb/conversion_procs/Makefile (REL7_3_STABLE): Conversion functions must be STRICT to prevent them from getting null inputs. 2003-08-07 13:56 barry * src/interfaces/jdbc/org/postgresql/: Driver.java.in, jdbc1/AbstractJdbc1Statement.java (REL7_3_STABLE): Backport to 7.3. Third try to fix the sql injection vulnerability. This fix completely removes the ability (hack) of being able to bind a list of values in an in clause. It was demonstrated that by allowing that functionality you open up the possibility for certain types of sql injection attacks. The previous fix attempts all focused on preventing the insertion of additional sql statements (the semi-colon problem: xxx; any new sql statement here). But that still left the ability to change the where clause on the current statement or perform a subselect which can circumvent applicaiton security logic and/or allow you to call any stored function. Modified Files: Tag: REL7_3_STABLE jdbc/org/postgresql/Driver.java.in jdbc/org/postgresql/jdbc1/AbstractJdbc1Statement.java 2003-08-05 13:39 tgl * src/backend/utils/adt/datetime.c (REL7_3_STABLE): Fix several places where fractional-second inputs were misprocessed in HAVE_INT64_TIMESTAMP cases, including two potential stack smashes when more than six fractional digits were supplied. Per bug report from Philipp Reisner. 2003-08-04 13:58 tgl * src/backend/libpq/be-secure.c (REL7_3_STABLE): SSL_read/SSL_write do not approximate the return conventions of recv() and send() very well at all; and in any case we can't use retval==0 for EOF due to race conditions. Make the same fixes in the backend as are required in libpq. 2003-08-04 13:25 tgl * src/interfaces/libpq/: fe-misc.c, fe-secure.c (REL7_3_STABLE): Fix some more problems with testing error returns from SSL. 2003-07-29 18:18 tgl * src/backend/access/nbtree/nbtsearch.c (REL7_3_STABLE): Fix longstanding error in _bt_search(): should moveright at top of loop not bottom. Otherwise we fail to moveright when the root page was split while we were "in flight" to it. This is not a significant problem when the root is above the leaf level, but if the root was also a leaf (ie, a single-page index just got split) we may return the wrong leaf page to the caller, resulting in failure to find a key that is in fact present. Bug has existed at least since 7.1, probably forever. 2003-07-24 00:38 tgl * src/backend/utils/adt/date.c (REL7_3_STABLE): Fix timestamp_date for HAVE_INT64_TIMESTAMP case.
I'd say yes based on the SSL and pg_dump fixes that were back patched ... On Wed, 29 Oct 2003, Tom Lane wrote: > Jan Wieck <JanWieck@Yahoo.com> writes: > > Tom Lane wrote: > >> Yeah, I think we have accumulated enough changes in the 7.3 branch to > >> justify a 7.3.5, but I'm not sure when we'll get around to it. > > > On 10/03/2003 Bruce was the only one responding to my question if the > > namespace fix I had for PL/Tcl should be backpatched into 7.3.5. He > > claimed that we'll probably not release any 7.3.X any more and we > > dropped the issue. > > Guess the question is open again then. > > I'm on the fence right now, but one or two more fixes in the 7.3 branch > will be enough to make me feel we should put out 7.3.5. If you are > confident of that namespace fix, then I'd say by all means commit it > into the 7.3 branch so it will be there when 7.3.5 happens. > > Attached are the current CVS log entries for post-7.3.4 changes in > REL7_3_STABLE. What do you think, is it time yet? > > regards, tom lane >
Perhaps I am not the appropriate one to mention this (and if you already have a QA group,GREAT!!!) since I am not a Postgres developer but... Is there any QA group that looks at the design specs and does independent desk checking and test writting according to those specs and then tests the code before it is released? Just a thought... Lynn Quoting Tom Lane <tgl@sss.pgh.pa.us>: > Jan Wieck <JanWieck@Yahoo.com> writes: > > Tom Lane wrote: > >> Yeah, I think we have accumulated enough changes in the 7.3 branch > to > >> justify a 7.3.5, but I'm not sure when we'll get around to it. > > > On 10/03/2003 Bruce was the only one responding to my question if the > > > namespace fix I had for PL/Tcl should be backpatched into 7.3.5. He > > claimed that we'll probably not release any 7.3.X any more and we > > dropped the issue. > > Guess the question is open again then. > > I'm on the fence right now, but one or two more fixes in the 7.3 > branch > will be enough to make me feel we should put out 7.3.5. If you are > confident of that namespace fix, then I'd say by all means commit it > into the 7.3 branch so it will be there when 7.3.5 happens. > > Attached are the current CVS log entries for post-7.3.4 changes in > REL7_3_STABLE. What do you think, is it time yet? > > regards, tom lane > > 2003-10-20 16:01 tgl > > * src/backend/rewrite/: rewriteManip.c (REL7_3_STABLE), > rewriteManip.c: It is possible for ResolveNew to be used to insert > a sublink into a subquery that didn't previously have one. We have > traditionally made the caller of ResolveNew responsible for > updating the hasSubLinks flag of the outermost query, but this > fails to account for hasSubLinks in subqueries. Fix ResolveNew to > handle this. We might later want to change the calling convention > of ResolveNew so that it can fix the outer query too, simplifying > callers. But I went with the localized fix for now. Per bug > report from J Smith, 20-Oct-03. > > 2003-10-02 18:25 tgl > > * src/backend/utils/adt/ruleutils.c (REL7_3_STABLE): When dumping > CREATE INDEX, must show opclass name if the opclass isn't in the > schema search path. Otherwise pg_dump doesn't correctly dump > scenarios where a custom opclass is created in 'public' and then > used by indexes in other schemas. > > 2003-09-29 14:53 momjian > > * src/bin/scripts/clusterdb (REL7_3_STABLE): > [ Patch applied only to 7.3.X.] > > Hi There's a bug in the clusterdb script where it looks like the > arguments to the psql command are being passed in the wrong order, > so it fails when you run it on a database that is not on localhost. > Here's the output from the command: > > 133 anands-Computer:bin/scripts> clusterdb -h wooster -U rr granada > psql: warning: extra option wooster ignored psql: warning: extra > option -U ignored psql: warning: extra option rr ignored psql: > warning: extra option -F: ignored psql: warning: extra option -P > ignored psql: warning: extra option format=unaligned ignored psql: > warning: extra option -t ignored psql: warning: extra option -c > ignored psql: warning: extra option SELECT nspname, > pg_class.relname, pg_class_2.relname FROM pg_class, pg_class AS > pg_class_2 JOIN pg_namespace ON (pg_namespace.oid=relnamespace), > pg_index WHERE pg_class.oid=pg_index.indrelid AND > pg_class_2.oid=pg_index.indexrelid AND pg_index.indisclustered AND > pg_class.relowner=(SELECT usesysid FROM pg_user WHERE > usename=current_user) ignored psql: FATAL: user "-h" does not > exist > > I'm attaching a patch that fixes the problem. The diff was run on > postgresql 7.3.4 > > Thanks a lot. Anand Ranganathan > > 2003-09-28 13:46 wieck > > * src/bin/pg_dump/pg_dump.c (REL7_3_STABLE): Backpatched changes > for rules when casts are dumped according to discussion on hackers. > > Jan > > 2003-09-23 11:11 tgl > > * src/backend/executor/spi.c (REL7_3_STABLE): _SPI_cursor_operation > forgot to check for failure return from _SPI_begin_call. Per gripe > from Tomasz Myrta. > > 2003-09-17 14:40 tgl > > * src/pl/plpython/plpython.c (REL7_3_STABLE): Back-patch fix for > plpython problems with dropped table columns; per bug report from > Arthur Ward, who also tested this patch. > > 2003-09-03 15:01 tgl > > * src/backend/utils/adt/formatting.c (REL7_3_STABLE): Back-patch > the other part of Karel's formatting bug fix. > > 2003-09-03 11:00 tgl > > * src/backend/utils/adt/formatting.c (REL7_3_STABLE): Repair > problems with to_char() overrunning its input string. From Karel > Zak. > > 2003-08-24 17:26 petere > > * src/bin/psql/po/de.po (REL7_3_STABLE): Fix translation mistake. > > 2003-08-24 01:13 ishii > > * src/backend/utils/mb/Unicode/gb18030_to_utf8.map (REL7_3_STABLE): > Fix GB18030 to UTF-8 mapping table > > 2003-08-24 01:00 ishii > > * src/backend/utils/mb/Unicode/UCS_to_GB18030.pl (REL7_3_STABLE): > Fix bug in GB18030 conversion script > > 2003-08-22 17:57 tgl > > * src/interfaces/libpq/fe-secure.c (REL7_3_STABLE): Sigh, I'm an > idiot ... SSL_ERROR_WANT_READ isn't an error condition at all, it > just means 'no data available yet'. > > 2003-08-08 11:49 tgl > > * src/backend/utils/mb/conversion_procs/Makefile (REL7_3_STABLE): > Conversion functions must be STRICT to prevent them from getting > null inputs. > > 2003-08-07 13:56 barry > > * src/interfaces/jdbc/org/postgresql/: Driver.java.in, > jdbc1/AbstractJdbc1Statement.java (REL7_3_STABLE): Backport to 7.3. > Third try to fix the sql injection vulnerability. This fix > completely removes the ability (hack) of being able to bind a list > of values in an in clause. It was demonstrated that by allowing > that functionality you open up the possibility for certain types of > sql injection attacks. The previous fix attempts all focused on > preventing the insertion of additional sql statements (the > semi-colon problem: xxx; any new sql statement here). But that > still left the ability to change the where clause on the current > statement or perform a subselect which can circumvent applicaiton > security logic and/or allow you to call any stored function. > > Modified Files: > Tag: REL7_3_STABLE > jdbc/org/postgresql/Driver.java.in > jdbc/org/postgresql/jdbc1/AbstractJdbc1Statement.java > > 2003-08-05 13:39 tgl > > * src/backend/utils/adt/datetime.c (REL7_3_STABLE): Fix several > places where fractional-second inputs were misprocessed in > HAVE_INT64_TIMESTAMP cases, including two potential stack smashes > when more than six fractional digits were supplied. Per bug report > from Philipp Reisner. > > 2003-08-04 13:58 tgl > > * src/backend/libpq/be-secure.c (REL7_3_STABLE): SSL_read/SSL_write > do not approximate the return conventions of recv() and send() very > well at all; and in any case we can't use retval==0 for EOF due to > race conditions. Make the same fixes in the backend as are > required in libpq. > > 2003-08-04 13:25 tgl > > * src/interfaces/libpq/: fe-misc.c, fe-secure.c (REL7_3_STABLE): > Fix some more problems with testing error returns from SSL. > > 2003-07-29 18:18 tgl > > * src/backend/access/nbtree/nbtsearch.c (REL7_3_STABLE): Fix > longstanding error in _bt_search(): should moveright at top of loop > not bottom. Otherwise we fail to moveright when the root page was > split while we were "in flight" to it. This is not a significant > problem when the root is above the leaf level, but if the root was > also a leaf (ie, a single-page index just got split) we may return > the wrong leaf page to the caller, resulting in failure to find a > key that is in fact present. Bug has existed at least since 7.1, > probably forever. > > 2003-07-24 00:38 tgl > > * src/backend/utils/adt/date.c (REL7_3_STABLE): Fix timestamp_date > for HAVE_INT64_TIMESTAMP case. > > ---------------------------(end of > broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to > majordomo@postgresql.org) >
Lynn.Tilby@asu.edu wrote: > > Perhaps I am not the appropriate one to mention this (and if you already > have a QA group,GREAT!!!) since I am not a > Postgres developer but... Is there any QA group that looks at the > design specs and does independent desk checking and test writting > according to those specs and then tests the code before it is released? We have an informal test process that occurs on the hackers list. It isn't formal, but as you can tell from the quality of the releases, it is quite effective. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Quoting Bruce Momjian <pgman@candle.pha.pa.us>: > Lynn.Tilby@asu.edu wrote: > > > > Perhaps I am not the appropriate one to mention this (and if you > already > > have a QA group,GREAT!!!) since I am not a > > Postgres developer but... Is there any QA group that looks at the > > design specs and does independent desk checking and test writting > > according to those specs and then tests the code before it is > released? > > We have an informal test process that occurs on the hackers list. It > isn't formal, but as you can tell from the quality of the releases, it > is quite effective. > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 359-1001 > + If your life is a hard drive, | 13 Roberts Road > + Christ can be your backup. | Newtown Square, Pennsylvania > 19073 > EXCELLENT!!!!! Lynn