Обсуждение: Re: [HACKERS] JDBC connections to 9.1
Bernd Helmle <mailings@oopsware.de> writes:
> If i am reading it correct, it reads "UTF8" from the backend, while
> expecting "UNICODE" only. Not sure what change has caused this,
> though.
I am --- when I redid the GUC assign_hook logic a few weeks ago,
I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the client
sent over. For instance, previous versions:
regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
client_encoding
-----------------
UnIcOdE
(1 row)
versus HEAD:
regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
client_encoding
-----------------
UTF8
(1 row)
I wasn't aware that JDBC would fail on that. It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?
regards, tom lane
On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Bernd Helmle <mailings@oopsware.de> writes: >> If i am reading it correct, it reads "UTF8" from the backend, while >> expecting "UNICODE" only. Not sure what change has caused this, >> though. > > I am --- when I redid the GUC assign_hook logic a few weeks ago, > I changed the client_encoding code so that it shows the normalized > (official) name of the encoding, not whatever random string the client > sent over. For instance, previous versions: > > regression=# set client_encoding = 'UnIcOdE'; > SET > regression=# show client_encoding ; > client_encoding > ----------------- > UnIcOdE > (1 row) > > versus HEAD: > > regression=# set client_encoding = 'UnIcOdE'; > SET > regression=# show client_encoding ; > client_encoding > ----------------- > UTF8 > (1 row) > > I wasn't aware that JDBC would fail on that. It's pretty annoying that > it does, but maybe we should grin and bear it, ie revert the change to > canonicalize the GUC's value? Ouch. I hate to revert that, since it seems like a clear improvement. But I also hate to break JDBC. Ouch. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Bernd Helmle <mailings@oopsware.de> writes: >> If i am reading it correct, it reads "UTF8" from the backend, while >> expecting "UNICODE" only. Not sure what change has caused this, >> though. > > I am --- when I redid the GUC assign_hook logic a few weeks ago, > I changed the client_encoding code so that it shows the normalized > (official) name of the encoding, not whatever random string the client > sent over. For instance, previous versions: > > regression=# set client_encoding = 'UnIcOdE'; > SET > regression=# show client_encoding ; > client_encoding > ----------------- > UnIcOdE > (1 row) > > versus HEAD: > > regression=# set client_encoding = 'UnIcOdE'; > SET > regression=# show client_encoding ; > client_encoding > ----------------- > UTF8 > (1 row) > > I wasn't aware that JDBC would fail on that. It's pretty annoying that > it does, but maybe we should grin and bear it, ie revert the change to > canonicalize the GUC's value? > > regards, tom lane > Older drivers will fail for sure. We can fix newer drivers, but if we leave it we will see a slew of bug reports. The reason the driver does this is to guarantee the client encoding is unicode. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Tom Lane <tgl@sss.pgh.pa.us> wrote: > I changed the client_encoding code so that it shows the normalized > (official) name of the encoding, not whatever random string the > client sent over. For instance, previous versions: > > regression=# set client_encoding = 'UnIcOdE'; > SET The whole area of character sets and encoding schemes is confusing enough without accepting a character set name as an encoding scheme specification. I'll bet that in five or ten years we'll be accepting more than one encoding scheme for the Unicode character set. > I wasn't aware that JDBC would fail on that. It's pretty annoying > that it does, but maybe we should grin and bear it, ie revert the > change to canonicalize the GUC's value? Can we fix the JDBC driver rather than reverting this? Long run, I'd be in favor of just rejecting a character set name as a client encoding specification. I think inferring one is being generous. -Kevin
On 18/04/11 15:57, Tom Lane wrote: > Bernd Helmle<mailings@oopsware.de> writes: >> If i am reading it correct, it reads "UTF8" from the backend, while >> expecting "UNICODE" only. Not sure what change has caused this, >> though. > I am --- when I redid the GUC assign_hook logic a few weeks ago, > I changed the client_encoding code so that it shows the normalized > (official) name of the encoding, not whatever random string the client > sent over. For instance, previous versions: > > regression=# set client_encoding = 'UnIcOdE'; > SET > regression=# show client_encoding ; > client_encoding > ----------------- > UnIcOdE > (1 row) > > versus HEAD: > > regression=# set client_encoding = 'UnIcOdE'; > SET > regression=# show client_encoding ; > client_encoding > ----------------- > UTF8 > (1 row) > > I wasn't aware that JDBC would fail on that. It's pretty annoying that > it does, but maybe we should grin and bear it, ie revert the change to > canonicalize the GUC's value? > > regards, tom lane > Am I right in thinking that would be that change committed on the 7th (http://archives.postgresql.org/pgsql-committers/2011-04/msg00039.php) ? I've just run the JDBC test build on my machine and it fails dismally with this very message repeated over and over again. What concerns me most is that (assuming my dates are right) the JDBC driver has been broken for 11 days and no one noticed. This would lead me to believe that there is no JDBC build server. What would it take to set one up? If someone can point me to a test machine I'd happily assist in setting one up. As for the breakage itself I'm OK with a new driver version for a new database version and from my experience people expect that. I recall a number of people asking me if an 8.4 driver would be OK to use against 9 before the 9 version was stable. Regards, -- Mike Fowler Registered Linux user: 379787
On Mon, Apr 18, 2011 at 11:14 AM, Mike Fowler <mike@mlfowler.com> wrote: > On 18/04/11 15:57, Tom Lane wrote: >> >> Bernd Helmle<mailings@oopsware.de> writes: >>> >>> If i am reading it correct, it reads "UTF8" from the backend, while >>> expecting "UNICODE" only. Not sure what change has caused this, >>> though. >> >> I am --- when I redid the GUC assign_hook logic a few weeks ago, >> I changed the client_encoding code so that it shows the normalized >> (official) name of the encoding, not whatever random string the client >> sent over. For instance, previous versions: >> >> regression=# set client_encoding = 'UnIcOdE'; >> SET >> regression=# show client_encoding ; >> client_encoding >> ----------------- >> UnIcOdE >> (1 row) >> >> versus HEAD: >> >> regression=# set client_encoding = 'UnIcOdE'; >> SET >> regression=# show client_encoding ; >> client_encoding >> ----------------- >> UTF8 >> (1 row) >> >> I wasn't aware that JDBC would fail on that. It's pretty annoying that >> it does, but maybe we should grin and bear it, ie revert the change to >> canonicalize the GUC's value? >> >> regards, tom lane >> > Am I right in thinking that would be that change committed on the 7th > (http://archives.postgresql.org/pgsql-committers/2011-04/msg00039.php) ? > I've just run the JDBC test build on my machine and it fails dismally with > this very message repeated over and over again. What concerns me most is > that (assuming my dates are right) the JDBC driver has been broken for 11 > days and no one noticed. This would lead me to believe that there is no JDBC > build server. What would it take to set one up? If someone can point me to a > test machine I'd happily assist in setting one up. > > As for the breakage itself I'm OK with a new driver version for a new > database version and from my experience people expect that. I recall a > number of people asking me if an 8.4 driver would be OK to use against 9 > before the 9 version was stable. > > Regards, > One would need a machine which supports java 1.4, 1.5, and 1.6 since the driver builds all 3 versions. There's actually a 4th between 1.4 and 1.5 but I don't recall the specifics Dave > -- > Mike Fowler > Registered Linux user: 379787 > > > -- > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-jdbc >
Dave Cramer <pg@fastcrypt.com> writes:
> On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I wasn't aware that JDBC would fail on that. �It's pretty annoying that
>> it does, but maybe we should grin and bear it, ie revert the change to
>> canonicalize the GUC's value?
> Older drivers will fail for sure. We can fix newer drivers, but if we
> leave it we will see a slew of bug reports.
Yeah. I'm thinking what we should do here is revert the change, with a
note in the source about why, and also change the JDBC driver to send
and expect "UTF8" not "UNICODE" (which as Kevin says is more correct
anyway). Then in a few releases' time we can un-revert the server
change.
regards, tom lane
Mike Fowler <mike@mlfowler.com> writes: > On 18/04/11 15:57, Tom Lane wrote: >> I am --- when I redid the GUC assign_hook logic a few weeks ago, >> I changed the client_encoding code so that it shows the normalized >> (official) name of the encoding, not whatever random string the client >> sent over. For instance, previous versions: > Am I right in thinking that would be that change committed on the 7th > (http://archives.postgresql.org/pgsql-committers/2011-04/msg00039.php) ? Yes, that one. > What concerns me > most is that (assuming my dates are right) the JDBC driver has been > broken for 11 days and no one noticed. This would lead me to believe > that there is no JDBC build server. What would it take to set one up? +1 for doing something along that line. regards, tom lane
On Mon, Apr 18, 2011 at 11:24 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Dave Cramer <pg@fastcrypt.com> writes: >> On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> I wasn't aware that JDBC would fail on that. It's pretty annoying that >>> it does, but maybe we should grin and bear it, ie revert the change to >>> canonicalize the GUC's value? > >> Older drivers will fail for sure. We can fix newer drivers, but if we >> leave it we will see a slew of bug reports. > > Yeah. I'm thinking what we should do here is revert the change, with a > note in the source about why, and also change the JDBC driver to send > and expect "UTF8" not "UNICODE" (which as Kevin says is more correct > anyway). Then in a few releases' time we can un-revert the server > change. > Well initially my concern was that people would have a challenge in the case where they had to re-certify their application if we made this change, however I realize they will have to do this anyway since upgrading to 9.1 is what necessitates it. So I'm less concerned with bug reports since people can just upgrade both Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Dave Cramer <pg@fastcrypt.com> writes:
> On Mon, Apr 18, 2011 at 11:24 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Yeah. I'm thinking what we should do here is revert the change, with a
>> note in the source about why, and also change the JDBC driver to send
>> and expect "UTF8" not "UNICODE" (which as Kevin says is more correct
>> anyway). Then in a few releases' time we can un-revert the server
>> change.
> Well initially my concern was that people would have a challenge in
> the case where they had to re-certify their application if we made
> this change, however I realize they will have to do this anyway since
> upgrading to 9.1 is what necessitates it.
I don't see any backwards compatibility risk, if that's what you mean.
Every backend release since 7.3 has treated client_encoding 'UTF8' and
'UNICODE' the same, and earlier releases didn't accept either one.
regards, tom lane
On 04/18/2011 11:25 AM, Tom Lane wrote: > >> What concerns me most is that (assuming my dates are right) the JDBC driver has been >> broken for 11 days and no one noticed. This would lead me to believe >> that there is no JDBC build server. What would it take to set one up? > +1 for doing something along that line. > > All you'd need to do is write a step for a buildfarm animal to fetch the JDBC driver and run some tests, and run it in a buildfarm client somewhere. The server code is quite agnostic about the steps that are reported on. IOW in addition to a running buildfarm member you need to write a small amount (< 100 lines, possibly much less) of perl code. cheers andrew
On 18/04/11 17:35, Andrew Dunstan wrote: > > > On 04/18/2011 11:25 AM, Tom Lane wrote: >> >>> What concerns me most is that (assuming my dates are right) the >>> JDBC driver has been >>> broken for 11 days and no one noticed. This would lead me to believe >>> that there is no JDBC build server. What would it take to set one up? >> +1 for doing something along that line. >> >> > > All you'd need to do is write a step for a buildfarm animal to fetch > the JDBC driver and run some tests, and run it in a buildfarm client > somewhere. The server code is quite agnostic about the steps that are > reported on. > > IOW in addition to a running buildfarm member you need to write a > small amount (< 100 lines, possibly much less) of perl code. > > cheers > > andrew I've found the entry on the Developer Wiki (http://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto). What I'll do is set-up three "farms" on my machine - one for 1.4, one for 1.5 and one for 1.6. It's been a while since I've had an excuse to write some Perl! I can't guarantee when I'll have it done as I'm away for a little over a week from Wednesday and I'm not allowed internet access! Regards, -- Mike Fowler Registered Linux user: 379787
On 18/04/11 17:12, Tom Lane wrote: > Dave Cramer<pg@fastcrypt.com> writes: >> Well initially my concern was that people would have a challenge in >> the case where they had to re-certify their application if we made >> this change, however I realize they will have to do this anyway since >> upgrading to 9.1 is what necessitates it. > I don't see any backwards compatibility risk, if that's what you mean. > Every backend release since 7.3 has treated client_encoding 'UTF8' and > 'UNICODE' the same, and earlier releases didn't accept either one. > > regards, tom lane > As there seems to be a consensus forming for fixing the JDBC driver, I've taken the liberty do so at the risk of being shot down. The patch is fairly straightforward, just changing UNICODE to UTF8 in a number of files including the translation files. I've tested this against 9.1devel (HEAD) and 8.4.7. For each database version I build and the tests running JDKs 1.4.2_19, 1.5.0_22 and 1.6.0_2. All on 32-bit. Regards, -- Mike Fowler Registered Linux user: 379787
Вложения
On Mon, 18 Apr 2011, Mike Fowler wrote: > On 18/04/11 17:12, Tom Lane wrote: >> Dave Cramer<pg@fastcrypt.com> writes: >>> Well initially my concern was that people would have a challenge in >>> the case where they had to re-certify their application if we made >>> this change, however I realize they will have to do this anyway since >>> upgrading to 9.1 is what necessitates it. >> I don't see any backwards compatibility risk, if that's what you mean. >> Every backend release since 7.3 has treated client_encoding 'UTF8' and >> 'UNICODE' the same, and earlier releases didn't accept either one. >> >> regards, tom lane >> > > As there seems to be a consensus forming for fixing the JDBC driver, I've > taken the liberty do so at the risk of being shot down. The patch is fairly > straightforward, just changing UNICODE to UTF8 in a number of files including > the translation files. I've tested this against 9.1devel (HEAD) and 8.4.7. > For each database version I build and the tests running JDKs 1.4.2_19, > 1.5.0_22 and 1.6.0_2. All on 32-bit. > Thanks, applied, mostly. It's great to have a patch for a problem before you even know it exists. I didn't modify the .po files. I doubt this will change the adjacent translation wording, but directly patching .po files is only something to do in more dire circumstances (like needing to make a backpatch to an old branch that won't get translators to look at it before the next release.) I also discarded your changes to AbstractJdbc3Statement. Those Unicode mentions are from the interface Javadoc, so I left them alone. Kris Jurka
Kris Jurka <books@ejurka.com> writes:
> On Mon, 18 Apr 2011, Mike Fowler wrote:
>> As there seems to be a consensus forming for fixing the JDBC driver, I've
>> taken the liberty do so at the risk of being shot down. The patch is fairly
>> straightforward, just changing UNICODE to UTF8 in a number of files including
>> the translation files. I've tested this against 9.1devel (HEAD) and 8.4.7.
>> For each database version I build and the tests running JDKs 1.4.2_19,
>> 1.5.0_22 and 1.6.0_2. All on 32-bit.
> Thanks, applied, mostly. It's great to have a patch for a problem before
> you even know it exists.
For purposes of the notes in the server-side fix, could you state which
JDBC driver versions these changes will first appear in?
regards, tom lane
On Tue, 19 Apr 2011, Tom Lane wrote: > Kris Jurka <books@ejurka.com> writes: >> On Mon, 18 Apr 2011, Mike Fowler wrote: >>> As there seems to be a consensus forming for fixing the JDBC driver, I've >>> taken the liberty do so at the risk of being shot down. The patch is fairly >>> straightforward, just changing UNICODE to UTF8 in a number of files including >>> the translation files. I've tested this against 9.1devel (HEAD) and 8.4.7. >>> For each database version I build and the tests running JDKs 1.4.2_19, >>> 1.5.0_22 and 1.6.0_2. All on 32-bit. > >> Thanks, applied, mostly. It's great to have a patch for a problem before >> you even know it exists. > > For purposes of the notes in the server-side fix, could you state which > JDBC driver versions these changes will first appear in? > This is in 9.1dev-900 and won't be backpatched. http://jdbc.postgresql.org/download.html#development Kris Jurka
Kris Jurka <books@ejurka.com> writes:
> On Tue, 19 Apr 2011, Tom Lane wrote:
>> For purposes of the notes in the server-side fix, could you state which
>> JDBC driver versions these changes will first appear in?
> This is in 9.1dev-900 and won't be backpatched.
OK, thanks. I've committed a patch to keep the server from
canonicalizing a setting of "UNICODE" (but not any other variations).
That should keep older drivers from breaking, and in a few years we
can remove the kluge, if anyone bothers ...
regards, tom lane