Обсуждение: driver initialization and connection separation
Hello All, I've got a Java based application that started life way back in Java 1.0. It supports about 6 RDBMS engines, presently, including Postgres, of course... When it needs to have multiple RDBMS connections, as it occasionally does, it merely creates an instance of itself and calls itself. This lets it connect to as many database instances as it likes and never confuse or otherwise conflate them as the instances connect to various RDBMSes. (For the curious, the API the application is coded against does sql dialect translation on-the-fly, sophisticated error handling, such as losing a network connection to an RDBMS, client-side, RDBMS independent journaling, etc. and this is why new instances, though seemingly a bit 'heavy-weight' is in fact a good, clean solution.) However, I haven't ever previously asked it to use the embeded SSL features before and this seems to present a problem - maybe more than one problem! I plan on eventually connecting a Postgres instance to the wilds of the internet. There is no need to validate users via certificates, though that's a cool thing to do that we'll eventually get to, and in fact forcing client authentication would be a real show-stopper at this point. So I was pleased to see the NonValidating Factory feature. I got the connectivity working just fine, thank you, but when I then asked the application to switch between encrypted and non-encrypted connections, it "broke." A modest amount of testing showed that when the app. attempted to reload the driver, the Postgres JDBC driver wasn't initializing as outlined here: http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Class.html#forName%28java.lang.String%29 I would have taken "initialization" to mean removing components (like NonValidatingFactory and WrappedFactory). Please note that the applciation did not call deregisterDriver as outlined here: http://java.sun.com/j2se/1.4.2/docs/api/java/sql/DriverManager.html#deregisterDriver%28java.sql.Driver%29 Rather, the app. just calls Class.forName() as it typically would in making a connection. ...Driver initialization has never before been an issue, so I'd never even considered that anything else was necessary. My supposition is that because the driver already found itself loaded, it just left itself in place. I'm considering what to do about this; On the one hand, to my mind this would be considered a bug, but on the other, it might be deemed that the proper thing to do to deregister the driver - deregistering wouldn't be a work-around, per se. (I'm assuming that de-registering the driver works to solve this problem!) Still, what, then, would "initialize" mean, as describe above, if not to clear out issues like this? I was also thinking about the same issue for the "WrappedFactory", which I'm not presently using but well might. Because the application creates multiple instances for multiple connections, I presume existing connections are safe from disruption by loading an incompatible driver setup - if anyone knows otherwise, please comment! I'm thinking that the driver instances will be unique to each instance of my class that calls them, but at least I know I am making an assumption. Meanwhile, I'm off to see if in fact deregistering the driver works... Thanks for your thoughts. Regards, Richard -- Richard Troy, Chief Scientist Science Tools Corporation 510-717-6942 rtroy@ScienceTools.com, http://ScienceTools.com/
Richard Troy wrote: > I got the > connectivity working just fine, thank you, but when I then asked the > application to switch between encrypted and non-encrypted connections, it > "broke." What were you trying to do exactly, and how did it break? > A modest amount of testing showed that when the app. attempted to > reload the driver, the Postgres JDBC driver wasn't initializing as > outlined here: > > http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Class.html#forName%28java.lang.String%29 Well, classes only ever get initialized once. -O
On Sun, 31 Jan 2010, Oliver Jowett wrote: > > Richard Troy wrote: > > > I got the > > connectivity working just fine, thank you, but when I then asked the > > application to switch between encrypted and non-encrypted connections, it > > "broke." > > What were you trying to do exactly, and how did it break? Sorry Oliver, I meant to be clear. The application can connect to any RDBMS, Postgres, Oracle, Ingres, whatever, and reconnect to any other, whenever the user wants. But once it has connected to Postgres using the SSL feature, all future connections using Postgres will use SSL. The only solution thus far is to kill the application and then start it again - there is no switching between using SSL and not using SSL, even though the driver is officially reloaded using Class.forName(driverClassName). Note that the official Java documentation from Sun says that doing so is equivalent to Class.forName(className, true, currentLoader) which is using the method for.Name(name, initialize, ClassLoader) - I cited a URL to their site to support this. To me this says that subsequent executions of Class.forName("driver") should yield identical results as with the first time it's executed, but that's demonstrably not true with Postgres - it "remembers" that it was asked to use SSL in the past and it continues to want to do so in the future. It seems to me that there's something missing from the initialization code, namely to remove the SSL features if the driver was already loaded. It should only be loaded when a URL asking for it comes along with the option ssl=true (for example). However, one could easily argue that it's not the initialization code that needs help but the code that constructs a new connection; it makes just as much sense that the request for a new connection give the appropriate type of connection, with or without SSL depending on whether the option in the URL was specified. ...I readily admit I haven't tried making multiple, simultaneous connections from the same instance of the driver; I presume that's possible but I've never needed. It seems to me this is much more a question of each individual connection than the driver itself. ...Your follow-on comment (below) about classes being initialized only once isn't helpful here because it isn't clear to me whether this is a case of initializing an object or loading _code_ that instantiates objects. In other words, even though I've been using Java since it's first release, I've never before had to worry about the code that instantiates objects changing while my code is running, so I haven't thought about it much. To my mind, each instance of my objects which then instantiate other objects - like JDBC connection objects - should get clean, separate and distinct objects, so in effect the driver code is loaded multiple times. But as I said, I could be wrong - I don't know much about how the JVM manages code that's loaded, but my suspicion is that it'll manage the individual instances of loading the same driver separately. Meanwhile, I just added this bit of code prior to the call to Class.forName(): for (Enumeration fu = DriverManager.getDrivers(); fu.hasMoreElements();) { DriverManager.deregisterDriver((java.sql.Driver)fu.nextElement()); } As I said, I don't have to worry about multiple connections within a single instance of this class, so deregistering all the drivers is fine - there should only ever be one connection and we're about to replace it. If there's an unacceptable performance hit for unloading a driver when it isn't needed, then I'll think about adding to this. But if it works, it's staying! -smile- I'll probably test it in the next few minutes... Regards, Richard > > > A modest amount of testing showed that when the app. attempted to > > reload the driver, the Postgres JDBC driver wasn't initializing as > > outlined here: > > > > http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Class.html#forName%28java.lang.String%29 > > Well, classes only ever get initialized once. > > -O > -- Richard Troy, Chief Scientist Science Tools Corporation 510-717-6942 rtroy@ScienceTools.com, http://ScienceTools.com/
Richard Troy wrote: > It seems to me that there's something missing from the initialization > code, namely to remove the SSL features if the driver was already loaded. > It should only be loaded when a URL asking for it comes along with the > option ssl=true (for example). However, one could easily argue that it's > not the initialization code that needs help but the code that constructs a > new connection; it makes just as much sense that the request for a new > connection give the appropriate type of connection, with or without SSL > depending on whether the option in the URL was specified. Driver initialization has nothing to do with this; you are making assumptions about Class.forName() that are not correct. It does not "initialize" the driver in any way other than what the JVM requires for any other class (see below) If you get a different sort of SSL-vs-not-SSL connection for the same connection URL depending on what connections you got in the past - that *is* a bug. Do you have a testcase showing the problem? > ...Your follow-on comment (below) about classes being initialized only > once isn't helpful here because it isn't clear to me whether this is a > case of initializing an object or loading _code_ that instantiates > objects. In other words, even though I've been using Java since it's first > release, I've never before had to worry about the code that instantiates > objects changing while my code is running, so I haven't thought about it > much. To my mind, each instance of my objects which then instantiate other > objects - like JDBC connection objects - should get clean, separate and > distinct objects, so in effect the driver code is loaded multiple times. > But as I said, I could be wrong - I don't know much about how the JVM > manages code that's loaded, but my suspicion is that it'll manage the > individual instances of loading the same driver separately. Class.forName()'s comment about initialization is talking about class initialization as described in the JVM spec - it involves (for example) running static initializer blocks in the class. This is guaranteed to happen before certain uses of the class (e.g. calling any method), and can be forced early via Class.forName(). But it is only ever done once per class. If you load the same class in a different classloader, it's effectively a different class, so will be separately initialized. (But I don't recommend doing that with the JDBC driver, because it'll mess with driver registration) -O
On Sun, 31 Jan 2010, Oliver Jowett wrote: > > It seems to me that there's something missing from the initialization > > code, namely to remove the SSL features if the driver was already loaded. > > It should only be loaded when a URL asking for it comes along with the > > option ssl=true (for example). However, one could easily argue that it's > > not the initialization code that needs help but the code that constructs a > > new connection; it makes just as much sense that the request for a new > > connection give the appropriate type of connection, with or without SSL > > depending on whether the option in the URL was specified. > > Driver initialization has nothing to do with this; you are making > assumptions about Class.forName() that are not correct. It does not > "initialize" the driver in any way other than what the JVM requires for > any other class (see below) I'm happy to conceed to your points on "initialization." > If you get a different sort of SSL-vs-not-SSL connection for the same > connection URL depending on what connections you got in the past - that > *is* a bug. Do you have a testcase showing the problem? Well, that's basically it: Connect with any Postgres JDBC valid URL you like as many times as you like and it always works perfectly so long as that URL doesn't ask for SSL, but once you have connected using SSL, the driver will thereafter ONLY connect with SSL whether the URL asks for it or not. IF the URL happens to want an SSL connection, it'll work, even if it's to a different server. Or, at least, so showed my testing last night. However, I think I just figured it out: Even though it's been in production for literally a dozen years, I think I just found a bug with our URL processing code that no one had reported. We're using URL templates so we can handle everybody's URL format with ease. The ones for Postgres look like this: jdbc:postgresql://%h:%p/%d jdbc:postgresql://%h:%p/%d?%o jdbc:postgresql://%h:%p/%d?user=%u&password=%pw jdbc:postgresql://%h:%p/%d?user=%u&password=%pw&%o where %h is the host %p is the port %u is the username %pw is the password and %o is the optional keyword=value[&keyword=value] string. ...I think what's happening is that the value that gets substituted for %o isn't being cleared when it evaluates the new connection data; if there was no "optional" information to add to the URL for the new connection, it's retaining its old value. This would explain what I saw and not be an issue with Postgres at all, obviously! I've got some work to do to prove the point, and I've got the flu, so I'll probably leave it until tomorrow. However, it fits the observed behavior... I should have proved the point before opening my big yap! I'll blame that social gaff on the flu! -wink- Thanks for your patience, Richard
On Sat, 30 Jan 2010, Richard Troy wrote: > > jdbc:postgresql://%h:%p/%d > jdbc:postgresql://%h:%p/%d?%o > jdbc:postgresql://%h:%p/%d?user=%u&password=%pw > jdbc:postgresql://%h:%p/%d?user=%u&password=%pw&%o > > where %h is the host > %p is the port > %u is the username > %pw is the password > and %o is the optional keyword=value[&keyword=value] string. > > ...I think what's happening is that the value that gets substituted for %o > isn't being cleared when it evaluates the new connection data; if there > was no "optional" information to add to the URL for the new connection, > it's retaining its old value. This would explain what I saw and not be an > issue with Postgres at all, obviously! > > I've got some work to do to prove the point, and I've got the flu, so I'll > probably leave it until tomorrow. However, it fits the observed > behavior... I should have proved the point before opening my big yap! I'll > blame that social gaff on the flu! -wink- > ...That was it alright: our code wasn't clearing the options - which in this case included: ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory Sorry to have troubled anyone, Richard
Richard Troy wrote: > The application can connect to any RDBMS, Postgres, Oracle, Ingres, > whatever, and reconnect to any other, whenever the user wants. But once it > has connected to Postgres using the SSL feature, all future connections > using Postgres will use SSL. The only solution thus far is to kill the > application and then start it again - there is no switching between using > SSL and not using SSL, even though the driver is officially reloaded using > Class.forName(driverClassName). As Oliver pointed out, that does not reload the driver. > Note that the official Java documentation from Sun says that doing so is > equivalent to Class.forName(className, true, currentLoader) which is using > the method for.Name(name, initialize, ClassLoader) - I cited a URL to > their site to support this. Once a class loader has loaded the class, it does not reload it again but determines that it already has done so. > To me this says that subsequent executions of Class.forName("driver") > should yield identical results as with the first time it's executed, but To you, maybe, but not in reality. Your perception will match reality once you've read the documentation. <http://java.sun.com/javase/6/docs/api/java/lang/ClassLoader.html> If you use a different class loader you can load the class again, but that poses other gotchas. > that's demonstrably not true with Postgres - it "remembers" that it was > asked to use SSL in the past and it continues to want to do so in the > future. Remembering that it used SSL is not curable by failing to reload the class. > It seems to me that there's something missing from the initialization > code, namely to remove the SSL features if the driver was already loaded. The class is not re-initialized since you're using the same class loader and it was already initialized once. Ergo, nothing is missing from the initialization. > It should only be loaded when a URL asking for it comes along with the > option ssl=true (for example). However, one could easily argue that it's > not the initialization code that needs help but the code that constructs a > new connection; it makes just as much sense that the request for a new > connection give the appropriate type of connection, with or without SSL > depending on whether the option in the URL was specified. According to how I read <http://jdbc.postgresql.org/documentation/84/connect.html> you should be able to get a non-SSL connection by not specifying the "ssl" property in the version of 'DriverManager.getConnection()' that takes a 'Properties' argument. > ...I readily admit I haven't tried making multiple, simultaneous > connections from the same instance of the driver; I presume that's > possible but I've never needed. It seems to me this is > much more a question of each individual connection than the driver itself. You use the driver with static methods, such as 'DriverManager.getConnection()'. AFAICS there is no access directly to driver instances. > ...Your follow-on comment (below) about classes being initialized only > once isn't helpful here because it isn't clear to me whether this is a > case of initializing an object or loading _code_ that instantiates He wasn't talking about initializing objects. He was talking about initialization of the driver *class*, and it's entirely relevant. Whether it's helpful to you or not depends on whether you assimilate and act upon the information. > objects. In other words, even though I've been using Java since it's first > release, I've never before had to worry about the code that instantiates > objects changing while my code is running, so I haven't thought about it Again, this is not about instantiation. You really should think about it much. > much. To my mind, each instance of my objects which then instantiate other > objects - like JDBC connection objects - should get clean, separate and > distinct objects, so in effect the driver code is loaded multiple times. Not how it works. Your mind needs to change on this. > But as I said, I could be wrong - I don't know much about how the JVM > manages code that's loaded, but my suspicion is that it'll manage the > individual instances of loading the same driver separately. Nope. It's not about instances. > Meanwhile, I just added this bit of code prior to the call to > Class.forName(): > > for (Enumeration fu = DriverManager.getDrivers(); fu.hasMoreElements();) > { > DriverManager.deregisterDriver((java.sql.Driver)fu.nextElement()); > } > > As I said, I don't have to worry about multiple connections within a > single instance of this class, so deregistering all the drivers is fine - I am not so sure about that. It seems like an awful lot of work, and how will it reregister? Once loaded and initialized, a class will not reinitialize unless it's actually garbage collected first. Since registration is part of intialization, and you don't get to reinitialize, I don't think you'll get the class to re-register. Deregistration is not idiomatic and I expect it will cause heartache. Better go with a conventional approach. > there should only ever be one connection and we're about to replace it. If > there's an unacceptable performance hit for unloading a driver when it Deregistration != unloading. > isn't needed, then I'll think about adding to this. But if it works, it's > staying! -smile- I'll probably test it in the next few minutes... Better would be to use the correct approach, which might be the 'DriverManager.getConnection()' call to which I just alluded. I haven't tested it yet - why don't you and let us know how it works? -- Lew