Обсуждение: BUG #4281: some types of errors do not log statements
The following bug has been logged online: Bug reference: 4281 Logged by: thomas Email address: me@alternize.com PostgreSQL version: 8.3.3 Operating system: Windows 2003 Description: some types of errors do not log statements Details: this isn't really a bug but rather a request for an improvement. i've noticed that in some cases of errornous sql statements, the statement itself is logged to the pg_log, in other cases it isn't: logged: 2008-07-05 01:53:02 CEST 3528 486eade5.dc8 10.1.1.71(53082)ERROR: column "xyz" does not exist at character 294 2008-07-05 01:53:02 CEST 3528 486eade5.dc8 10.1.1.71(53082)STATEMENT: SELECT xyz FROM test not logged: 2008-07-05 02:16:15 CEST 6644 486ebab4.19f4 127.0.0.1(1616)ERROR: invalid byte sequence for encoding "UTF8": 0xc474 2008-07-05 02:16:15 CEST 6644 486ebab4.19f4 127.0.0.1(1616)HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding". it would be usefull to always see the sql statement that provoked the error. especially in the case of wrong utf byte sequences it can get very difficult to find the point of failure without more information. thanks, thomas
thomas wrote: > > The following bug has been logged online: > > Bug reference: 4281 > Logged by: thomas > Email address: me@alternize.com > PostgreSQL version: 8.3.3 > Operating system: Windows 2003 > Description: some types of errors do not log statements > Details: > > this isn't really a bug but rather a request for an improvement. > > i've noticed that in some cases of errornous sql statements, the statement > itself is logged to the pg_log, in other cases it isn't: > > logged: > > 2008-07-05 01:53:02 CEST 3528 486eade5.dc8 10.1.1.71(53082)ERROR: column > "xyz" does not exist at character 294 > 2008-07-05 01:53:02 CEST 3528 486eade5.dc8 10.1.1.71(53082)STATEMENT: > SELECT xyz FROM test > > > not logged: > > 2008-07-05 02:16:15 CEST 6644 486ebab4.19f4 127.0.0.1(1616)ERROR: invalid > byte sequence for encoding "UTF8": 0xc474 > 2008-07-05 02:16:15 CEST 6644 486ebab4.19f4 127.0.0.1(1616)HINT: This error > can also happen if the byte sequence does not match the encoding expected by > the server, which is controlled by "client_encoding". > > > it would be usefull to always see the sql statement that provoked the error. > especially in the case of wrong utf byte sequences it can get very difficult > to find the point of failure without more information. I am unclear what would cause this. Is the STATEMENT: line coming from log_statement? What is the query that is not showing? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> thomas wrote:
>> 2008-07-05 02:16:15 CEST 6644 486ebab4.19f4 127.0.0.1(1616)ERROR: invalid
>> byte sequence for encoding "UTF8": 0xc474
> I am unclear what would cause this.
An encoding violation in an incoming SQL command would cause it, because
we won't have set debug_query_string yet. And there is no easy fix for
that, because we certainly don't want to have illegally encoded data
inside the backend. The best possible scenario would be that you get
a different error while trying to translate the string back out for the
client, and the worst would be one of those recursive encoding-error-
during-error-reporting crashes.
I think the short answer is that the OP needs to fix his client code to
not generate queries that are illegally encoded according to what he has
set client_encoding to be.
regards, tom lane
> thomas wrote: >> The following bug has been logged online: >> >> Bug reference: 4281 >> Logged by: thomas >> Email address: me@alternize.com >> PostgreSQL version: 8.3.3 >> Operating system: Windows 2003 >> Description: some types of errors do not log statements >> Details: >> >> this isn't really a bug but rather a request for an improvement. >> >> i've noticed that in some cases of errornous sql statements, the statement >> itself is logged to the pg_log, in other cases it isn't: >> >> logged: >> >> 2008-07-05 01:53:02 CEST 3528 486eade5.dc8 10.1.1.71(53082)ERROR: column >> "xyz" does not exist at character 294 >> 2008-07-05 01:53:02 CEST 3528 486eade5.dc8 10.1.1.71(53082)STATEMENT: >> SELECT xyz FROM test >> >> >> not logged: >> >> 2008-07-05 02:16:15 CEST 6644 486ebab4.19f4 127.0.0.1(1616)ERROR: invalid >> byte sequence for encoding "UTF8": 0xc474 >> 2008-07-05 02:16:15 CEST 6644 486ebab4.19f4 127.0.0.1(1616)HINT: This error >> can also happen if the byte sequence does not match the encoding expected by >> the server, which is controlled by "client_encoding". >> >> >> it would be usefull to always see the sql statement that provoked the error. >> especially in the case of wrong utf byte sequences it can get very difficult >> to find the point of failure without more information. > > I am unclear what would cause this. Is the STATEMENT: line coming from > log_statement? What is the query that is not showing? > pretty much any query that has invalid utf8 byte code in it won't be shown in the logs. the particular problem was with an adserver tool (openX) that tried to insert iso-encoded data with umlauts (äüö) into an utf8 database. for example it tried to insert the geolocation "Zürich" during a ad request logging which failed with: " ERROR: invalid byte sequence for encoding "UTF8": 0xFC" without showing the actual query. maybe its by design (to not insert badly encoded characters into the utf8 encoded logs)? nevertheless to debug those faulty programm/codes, it would help to see what query provokes the error... > Is the STATEMENT: line coming from log_statement? humm... don't know. pretty much standard out of the box pgsql installation, the postresql.conf settings are defaults. thanks thomas
Thomas H. wrote: > maybe its by design (to not insert badly encoded characters into the > utf8 encoded logs)? nevertheless to debug those faulty programm/codes, > it would help to see what query provokes the error... Well, the problem is mainly that there is no query, because the bytes arriving are garbage. A human observer could make sense of it in some cases, but not a computer in the general case.
"Peter Eisentraut" <peter_e@gmx.net> writes: > Thomas H. wrote: >> maybe its by design (to not insert badly encoded characters into the >> utf8 encoded logs)? nevertheless to debug those faulty programm/codes, >> it would help to see what query provokes the error... > > Well, the problem is mainly that there is no query, because the bytes arriving > are garbage. A human observer could make sense of it in some cases, but not > a computer in the general case. How is that different from any other syntax error? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!
Gregory Stark <stark@enterprisedb.com> writes:
> "Peter Eisentraut" <peter_e@gmx.net> writes:
>> Well, the problem is mainly that there is no query, because the bytes arriving
>> are garbage. A human observer could make sense of it in some cases, but not
>> a computer in the general case.
> How is that different from any other syntax error?
It's not a syntax error, it's an encoding error, and we daren't try to
spit the data back out for fear of getting more encoding errors and
ending up in an infinite recursion in elog.c.
regards, tom lane