Обсуждение: PSQLException: The column name
postgresql-9.0-801.jdbc3.jar
PostgreSQL 8.3.7 on amd64-portbld-freebsd7.2, compiled by GCC cc (GCC) 4.2.1
20070719 [FreeBSD]
Caused by: org.postgresql.util.PSQLException: The column name mdn was not found
in this ResultSet.
at
org.postgresql.jdbc2.AbstractJdbc2ResultSet.findColumn(AbstractJdbc2ResultSet.java:2562)
at
org.postgresql.jdbc2.AbstractJdbc2ResultSet.getString(AbstractJdbc2ResultSet.java:2405)
at
org.apache.commons.dbcp.DelegatingResultSet.getString(DelegatingResultSet.java:225)
I receive the above error message when accessing the JDBC resultset from
multiple threads. It does not occur consistently, but roughly every 20 or 30
runs and almost immediately upon invocation. Looking
at org/postgresql/jdbc2/AbstractJdbc2ResultSet.java it appears there is a race
condition around building columnNameIndexMap
if (columnNameIndexMap == null) { columnNameIndexMap
= new HashMap(fields.length * 2); // The JDBC spec says when you
have duplicate columns names, // the first one should be returned.
So load the map in // reverse order so the first ones will overwrite
later ones. for (int i = fields.length - 1; i >= 0; i--)
{
columnNameIndexMap.put(fields[i].getColumnLabel().toLowerCase(Locale.US), new
Integer(i + 1)); } }
I cannot confirm this, but I would guess that on thread enters, has the NULL map
and creates the new HashMap. The second thread enters, sees the non-NULL
columnNameIndexMap, and gets to the HashMap.get call prior to the first thread
fully populating the map. If this is the problem, would wrapping the IF block
in a synchronized(this) block fix it?
Thanks.
Michael
On Mon, 8 Nov 2010, Michael Fork wrote: > Caused by: org.postgresql.util.PSQLException: The column name mdn was not found > in this ResultSet. > > I receive the above error message when accessing the JDBC resultset from > multiple threads. It does not occur consistently, but roughly every 20 > or 30 runs and almost immediately upon invocation. Looking at > org/postgresql/jdbc2/AbstractJdbc2ResultSet.java it appears there is a > race condition around building columnNameIndexMap > > I cannot confirm this, but I would guess that on thread enters, has the > NULL map and creates the new HashMap. The second thread enters, sees > the non-NULL columnNameIndexMap, and gets to the HashMap.get call prior > to the first thread fully populating the map. If this is the problem, > would wrapping the IF block in a synchronized(this) block fix it? > That looks like the correct diagnosis and fix to me. Kris Jurka
On Tue, Nov 9, 2010 at 4:10 PM, Kris Jurka <books@ejurka.com> wrote:
On Mon, 8 Nov 2010, Michael Fork wrote:Caused by: org.postgresql.util.PSQLException: The column name mdn was not found
in this ResultSet.I receive the above error message when accessing the JDBC resultset from multiple threads. It does not occur consistently, but roughly every 20 or 30 runs and almost immediately upon invocation. Looking at org/postgresql/jdbc2/AbstractJdbc2ResultSet.java it appears there is a race condition around building columnNameIndexMapI cannot confirm this, but I would guess that on thread enters, has the NULL map and creates the new HashMap. The second thread enters, sees the non-NULL columnNameIndexMap, and gets to the HashMap.get call prior to the first thread fully populating the map. If this is the problem, would wrapping the IF block in a synchronized(this) block fix it?
That looks like the correct diagnosis and fix to me.
That would only be a valid fix if ResultSet is intended to be inherently threadsafe. I wouldn't think that is the case. There is no mention of thread safety at all in the javadocs, which I would always take to imply that there is no implicit thread safety. Code which utilizes a ResultSet should do so in a thread safe manner, rather than attempting to make the resultset implementation thread-safe - unless my assumptions about the intent of the interface designers is incorrect.
Looking into this further, I think the initial fix was insufficient. I think there are two ways to go. One is to synchronize the entire findColumnIndex. Since we are using a standard HashMap and not a synchronized one, the additional put could cause issues when doing a get if a resize was underway. The alternative would be to use Collections.synchronizedMap at instantiation of the HashMap and wrap the initial load of it in the synchronized(this) block as proposed below. Once loaded, the get/puts would be synchronized by the map. Thanks. Michael ----- Original Message ---- From: Kris Jurka <books@ejurka.com> To: Michael Fork <mfork00@yahoo.com> Cc: pgsql-jdbc@postgresql.org Sent: Tue, November 9, 2010 7:10:18 PM Subject: Re: [JDBC] PSQLException: The column name <col> was not found in this ResultSet. On Mon, 8 Nov 2010, Michael Fork wrote: > Caused by: org.postgresql.util.PSQLException: The column name mdn was not found > in this ResultSet. > > I receive the above error message when accessing the JDBC resultset from >multiple threads. It does not occur consistently, but roughly every 20 or 30 >runs and almost immediately upon invocation. Looking at >org/postgresql/jdbc2/AbstractJdbc2ResultSet.java it appears there is a race >condition around building columnNameIndexMap > > I cannot confirm this, but I would guess that on thread enters, has the NULL >map and creates the new HashMap. The second thread enters, sees the non-NULL >columnNameIndexMap, and gets to the HashMap.get call prior to the first thread >fully populating the map. If this is the problem, would wrapping the IF block >in a synchronized(this) block fix it? > That looks like the correct diagnosis and fix to me. Kris Jurka -- Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-jdbc
On 11/10/2010 12:27 AM, Samuel Gendler wrote: > On Tue, Nov 9, 2010 at 4:10 PM, Kris Jurka <books@ejurka.com > <mailto:books@ejurka.com>> wrote: > > > > On Mon, 8 Nov 2010, Michael Fork wrote: > > Caused by: org.postgresql.util.PSQLException: The column name > mdn was not found > in this ResultSet. > > I receive the above error message when accessing the JDBC > resultset from multiple threads. It does not occur > consistently, but roughly every 20 or 30 runs and almost > immediately upon invocation. Looking at > org/postgresql/jdbc2/AbstractJdbc2ResultSet.java it appears > there is a race condition around building columnNameIndexMap > > I cannot confirm this, but I would guess that on thread enters, > has the NULL map and creates the new HashMap. The second thread > enters, sees the non-NULL columnNameIndexMap, and gets to the > HashMap.get call prior to the first thread fully populating the > map. If this is the problem, would wrapping the IF block in a > synchronized(this) block fix it? IT is possible without proper synchronization for thread A to instantiate and fully population a shared Map reference, and for thread B subsequently to see a 'null', empty or partially populated view from that reference. > That looks like the correct diagnosis and fix to me. > > > That would only be a valid fix if ResultSet is intended to be inherently > threadsafe. I wouldn't think that is the case. There is no mention of > thread safety at all in the javadocs, which I would always take to imply > that there is no implicit thread safety. Code which utilizes a > ResultSet should do so in a thread safe manner, rather than attempting > to make the resultset implementation thread-safe - unless my assumptions > about the intent of the interface designers is incorrect. It is meaningless to talk about thread safety with respect to interfaces. -- Lew
>> Code which utilizes a >> ResultSet should do so in a thread safe manner, rather than attempting >> to make the resultset implementation thread-safe - unless my assumptions >> about the intent of the interface designers is incorrect. > > It is meaningless to talk about thread safety with respect to interfaces. I think Samuel was talking about the thread-safety guarantees of the interface contract. E.g., the List interface does not guarantee thread-safety, so if you want to call add() on multiple Lists from multiple threads, synchronization is your responsibility. If the interface contract of ResultSet does not guarantee thread safety, I don't see why the driver should bother to offer it there. Hashtable, Vector, and StringBuffer took that route, and look at them now. --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-3500 Main www.truviso.com
> ...you want to call add() on multiple Lists from multiple threads... That should, of course, be "on the same List from multiple threads". --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-3500 Main www.truviso.com
Samuel Gendler wrote: >>> Code which utilizes a >>> ResultSet should do so in a thread safe manner, rather than attempting >>> to make the resultset implementation thread-safe - unless my assumptions >>> about the intent of the interface designers is incorrect. Lew wrote: >> It is meaningless to talk about thread safety with respect to interfaces. Maciek Sakrejda wrote: > I think Samuel was talking about the thread-safety guarantees of the > interface contract. E.g., the List interface does not guarantee It is meaningless to talk about thread safety with respect to interfaces. Thread safety is an implementation guarantee (or lack thereof), not discernible nor enforceable by an interface. > thread-safety, so if you want to call add() on multiple Lists from > multiple threads, synchronization is your responsibility. If the If you want to call 'add()' on multiple 'List's from multiple threads, even if you are using thread-safe implementations, if the actions among the several list instances needs to be coordinated then synchronization is your responsibility. > interface contract of ResultSet does not guarantee thread safety, I Because interfaces cannot guarantee thread safety. > don't see why the driver should bother to offer it there. Hashtable, > Vector, and StringBuffer took that route, and look at them now. Those are not interfaces. It is impossible for ResultSet to guarantee thread safety. It is also impossible for List to guarantee thread safety, yet 'java.util.concurrent.CopyOnWriteArrayList<E>' is a "thread-safe variant of ArrayList". Thread safety lives or dies only in the implementation. -- Lew
Maciek Sakrejda wrote: >> ...you want to call add() on multiple Lists from multiple threads... > > That should, of course, be "on the same List from multiple threads". To put that in context with the correction: > if you want to call add() on the same List from > multiple threads, synchronization is your responsibility. Not if you use a thread-safe 'List'. Then the implementation handles thread safety for you. -- Lew
We're getting off topic--perhaps I didn't express myself clearly. Getting back to Samuel's original point, if the ResultSet interface contract does not require a given implementation to provide thread safety for these operations--and it appears that it does not--then maybe pgjdbc should not try to provide it. If a user needs to use a ResultSet from multiple threads, it can be synchronized externally. --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-3500 Main www.truviso.com
On 2010-11-12 07:18, Maciek Sakrejda wrote: > We're getting off topic--perhaps I didn't express myself clearly. > Getting back to Samuel's original point, if the ResultSet interface > contract does not require a given implementation to provide thread > safety for these operations--and it appears that it does not--then > maybe pgjdbc should not try to provide it. If a user needs to use a > ResultSet from multiple threads, it can be synchronized externally. The pgjdbc driver is supposed to be thread safe: http://jdbc.postgresql.org/documentation/pgjdbc.html#Using+the+Driver+in+a+Multithreaded+or+a+Servlet+Environment Whether this is good or bad doesn't matter. Has been this way forever, can't change that now. Till -- Kyon, Till Toenges, tt@kyon.de, http://kyon.de Blumenstr. 39, 47798 Krefeld, +49-2151-3292595
Maciek Sakrejda wrote: > We're getting off topic--perhaps I didn't express myself clearly. > Getting back to Samuel's original point, if the ResultSet interface > contract does not require a given implementation to provide thread > safety for these operations--and it appears that it does not--then I do not know of any interface that requires its implementations to be thread safe, anywhere in Java. The closest I can think of is the JPA javax.persistence.EntityManagerFactory, all extant implementations of which are thread-safe, but of course the interface cannot require that. Can you think of one that does? -- Lew
* Till Toenges: > The pgjdbc driver is supposed to be thread safe: > > http://jdbc.postgresql.org/documentation/pgjdbc.html#Using+the+Driver+in+a+Multithreaded+or+a+Servlet+Environment > > Whether this is good or bad doesn't matter. Has been this way forever, > can't change that now. To me, it only says that Connection objects are thread-safe. ResultSets are a different story. To implement them in a thread-safe manner, you need a thread-local variables, otherwise you cannot get wasNull() right. -- Florian Weimer <fweimer@bfk.de> BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99
* Lew: > I do not know of any interface that requires its implementations to be > thread safe, anywhere in Java. The closest I can think of is the JPA > javax.persistence.EntityManagerFactory, all extant implementations of > which are thread-safe, but of course the interface cannot require > that. > > Can you think of one that does? java.util.concurrent.BlockingQueue and many others from the java.util.concurrent package. This is not enforced by the language, of course. -- Florian Weimer <fweimer@bfk.de> BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99
On 11/12/2010 12:00 PM, Florian Weimer wrote: Lew: >> I do not know of any interface that requires its implementations to be >> thread safe, anywhere in Java. The closest I can think of is the JPA >> javax.persistence.EntityManagerFactory, all extant implementations of >> which are thread-safe, but of course the interface cannot require >> that. >> >> Can you think of one that does? > java.util.concurrent.BlockingQueue and many others from the > java.util.concurrent package. This is not enforced by the language, > of course. And therefore does not require implementations to be thread safe, as I stated. Even the documentation for 'BlockingQueue' makes that point: "BlockingQueue implementations are thread-safe." Not the interface, only its implementations are thread safe, and implicitly only the ones known at the time the Javadoc comments were written. -- Lew
> Even the documentation for 'BlockingQueue' makes that point: "BlockingQueue > implementations are thread-safe." Not the interface, only its > implementations are thread safe, and implicitly only the ones known at the > time the Javadoc comments were written. Of course BlockingQueue does not *provide* thread safety, but it *requires* it of its implementations. Not through any enforceable means, but this is part of the interface contract, i.e., this is part of what it means to be a BlockingQueue. Following the thread safety guarantees (or lack thereof) specified in the interface contract is as important as having put() actually put an item on the queue instead of removing one (note that this is also not "required" by the interface). Implementing an interface is more than just adding "implements Foo" and writing whatever code you like in all the methods required to be there by the compiler. But we're getting really off-topic. Samuel suggested that the ResultSet interface does not guarantee thread safety (i.e., it does not promise that any implementation will be synchronized). As Till pointed out, the PostgreSQL implementation JDBC API *does* in fact promise thread safety (which is fine--an implementation can offer additional guarantees here, as per Hashtable and Map). This would suggest that we do need the fix that Michael originally suggested. However--and I think this is the only real open issue--Florian pointed out that the guarantees in the PostgreSQL JDBC docs could be interpreted to mean that the PostgreSQL Connection is guaranteed to be thread-safe while ResultSet is not (especially given the dicey wasNull() issue). So, any thoughts on Florian's suggestion? --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-3500 Main www.truviso.com
On Sun, 14 Nov 2010, Maciek Sakrejda wrote: > But we're getting really off-topic. Samuel suggested that the > ResultSet interface does not guarantee thread safety (i.e., it does > not promise that any implementation will be synchronized). As Till > pointed out, the PostgreSQL implementation JDBC API *does* in fact > promise thread safety (which is fine--an implementation can offer > additional guarantees here, as per Hashtable and Map). This would > suggest that we do need the fix that Michael originally suggested. > > However--and I think this is the only real open issue--Florian pointed > out that the guarantees in the PostgreSQL JDBC docs could be > interpreted to mean that the PostgreSQL Connection is guaranteed to be > thread-safe while ResultSet is not (especially given the dicey > wasNull() issue). > The real question is how realistic/useful is the coding pattern in question. There are other known thread safety problems in the driver that have not been dealt with. For example two threads sharing a PreparedStatement, one executing it while the other is simultaneously calling setXXX on it. The driver could synchonize this, but you'd random behavior behavior depending on which call happened first. So no driver changes were made to help support this because it doesn't seem like a reasonable coding pattern. Perhaps Michael could provide some additional context regarding the application and why multiple threads are processing the same ResultSet. You can avoid wasNull problems by only using getObject, but you'd still need to synchonize the next() calls. And how much gain can you really get by parallel ResultSet reading? Kris Jurka
I am using the Spring Batch project (http://static.springsource.org/spring-batch/) to complete a large, parallel job configured to run 10 threads. A org.springframework.batch.item.database.JdbcCursorItemReader<T> object is used to open the recordset using TYPE_FORWARD_ONLY and CONCUR_READ_ONLY. I tried following the code but was unable to figure out exactly how the recordset object was accessed and where. It does sound like the root of the problem lies in Spring Batch not properly synchronizing access to a recordset object and they should address the issue there if possible. While I would like to see the issue in PostgreSQL JDBC driver addressed as it was the only problem I have encountered to date, I understand if it is not since there is no explicit thread safe guarantee around a recordset. Thanks. Michael ----- Original Message ---- From: Kris Jurka <books@ejurka.com> To: Maciek Sakrejda <msakrejda@truviso.com> Cc: Michael Fork <mfork00@yahoo.com>; pgsql-jdbc@postgresql.org Sent: Sun, November 14, 2010 5:44:33 PM Subject: Re: [JDBC] PSQLException: The column name <col> was not found in this ResultSet. On Sun, 14 Nov 2010, Maciek Sakrejda wrote: > But we're getting really off-topic. Samuel suggested that the > ResultSet interface does not guarantee thread safety (i.e., it does > not promise that any implementation will be synchronized). As Till > pointed out, the PostgreSQL implementation JDBC API *does* in fact > promise thread safety (which is fine--an implementation can offer > additional guarantees here, as per Hashtable and Map). This would > suggest that we do need the fix that Michael originally suggested. > > However--and I think this is the only real open issue--Florian pointed > out that the guarantees in the PostgreSQL JDBC docs could be > interpreted to mean that the PostgreSQL Connection is guaranteed to be > thread-safe while ResultSet is not (especially given the dicey > wasNull() issue). > The real question is how realistic/useful is the coding pattern in question. There are other known thread safety problems in the driver that have not been dealt with. For example two threads sharing a PreparedStatement, one executing it while the other is simultaneously calling setXXX on it. The driver could synchonize this, but you'd random behavior behavior depending on which call happened first. So no driver changes were made to help support this because it doesn't seem like a reasonable coding pattern. Perhaps Michael could provide some additional context regarding the application and why multiple threads are processing the same ResultSet. You can avoid wasNull problems by only using getObject, but you'd still need to synchonize the next() calls. And how much gain can you really get by parallel ResultSet reading? Kris Jurka -- Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-jdbc