Re: PostgresDataSource Question

(I responded to the mailing list after joe asked to keep this thread
here.)

--- Kov�cs_P�ter <peter.kovacs@sysdata.siemens.hu> wrote:
> I am afraid that (with my effective contribution :-)) things have got
mixed
> up a bit here. So I will try to sort it  out.

I feel the same way as you.  Also, as I go over your letter, keep in mind
that like most people here, I work on the driver in my spare time.
I'm currently working on two open source projects on my spare time;
submitting code for the pgsql JDBC driver and as a committer for the
castor JDO project.  I also have to make a living with my stuff at
work. :-) (I'm just lucky enough that my work also uses both Castor
and PostgreSQL, but I cannot work on either open-source project during
work.)

> First of all: the XADataSourceImpl does some kind of pooling, but it is
> probably not intended to implement the functionality you want. I will

Actually, it just about does.  The only thing it doesn't do is limit
and block waiting for connections to be free. (But I'm just arguing
semantics. Your assessment is correct.)

> explain what this sorta pooling does, but before doing so I'd like to
make
> another statement that has not yet been made: support for connection
pooling
> and support for distributed transactions can be implemented separately.
So
> you do not need to bother with the xa package at all if you want to
support
> connection pooling or even if you want _implement_ connection pooling
(in my

I never disagreed with this.  I had thought from your previous
postings that you were afraid I was re-doing work that is already
available in the XA package.  I see now that this is not what you
meant.

> view there is a difference between supporting connection pooling and
> implementing connection pooling). The reason why I was kind of pushing
the
> xa package is that, people on this list keep talking about
incompleteness
> regarding the Exoffice's xa package -- and they are right from their
> perspective. But if you look at it in another way, this is good stuff --
and
> this is clearly my fault that I did not fully explain my perspective.

The code base in the xa package is good, just not complete.  We may
differ on what we think is incomplete though... (And I'm not saying
that the lack of a pooling implementation makes the XA package incomplete)

> I think that first I should explain my view. What I really need is a way
to
> handle transactions in an environment where there are only local

I understand the JTA layer, and I can see how you would find it useful
from 'local' transaction even when you can't do distributed
transactions.

[..]

> What I said above may be obvious, but a brief explanation may be
helpful. I
> am using a simple model here: I partition my application into two
layers:
> business logic layer (BLL) and data access layer (DAL). When I was
talking

Just so you know where I"m coming from, I use either Castor, TopLink
or EJB CMP entity beans, in my code-base to acheive this.  JBoss's EJB
layer works nicely with the existing JDBC driver, in my view, even
without distributed transaction availablity.  However, I prefer Castor
since its a) its easy to switch databases as needed, without changes
to the BLL and b) everything works as more of a java object. (Same is
true with Toplink, cocobase, and other non-opensource mappers)

With that said, I see the biggest 'bang-for-the-buck' usage of JTA is
inside of tools like these, rather than directly.  So when we work on the
XA package, it benefits the users of WebLogic, TopLink, etc the most.
Do you agree with this? (It benfits you here, since you are writing
directly to the jdbc2.0 optional spec)

[ .. ]

> attached to and detached from a thread.) So why should not I use the
JTA,
> if it does the job (well)???

In truth, you are correct.  I would only argue that its overkill.

> So I was setting out to create my own implementation of JTA. But how to
do
> it? The JTA defines its versions of XA interface to integrate the
resource
> manager(s). Why should I invent another interface? Most of the RDMBSs
> already provide implementations of these interfaces anyway. Does it do
any
> harm, if the architecture is also capable of handling distributed
> transactions. (My strong belief is that: it does not, and it is even
good
> that I can use the same infrastructure for distributed transactions, as
my
> needs and tools advance -- but you may disagree.)

Actually, no. I do not disagree.  Don't re-build what is already
available.  (That's why I point you to JBoss, since it already does
the stuff you want.)

> ((I actually implemented a mechanism for handling local transactions in
an
> abstract way using a custom API. It was not easy, but you learn a lot
from
> doing this kind of stuff, because you have to go through and find
solutions
> to problems which arise in such an environment. After I thought the
> implementation was complete and worked nicely, another potential problem
> came to my mind. The problem was the following. I used ThreadLocal to
attach
> transaction objects to threads. Also, I used CORBA for IPC. Now, every
> decent CORBA implementation uses thread pools to process incoming
requests.
> What happens --I asked myself--, if the user-programmer forgets to end
> [commit or rollback] the transaction??? It may take some time before the
> timer for the transaction expires and will be cleaned up from the thread
it
> has been attached to. During this time, the thread can be reused by the
ORB
> to process another incoming CORBA request, and the implementation that
> executes in the reused thread will be confused, because it will find
that it
> is part of an ongoing transaction. The clean solution to this is to use
a
> mechanism which is integrated in the CORBA infrastructure. CORBA
provides
> the local interface Current to move around thread specific information.
But,
> I asked myself, if I am already bogged down so deeply in this mess, why
> should not I use OMG's Object Transaction Service -- and why should not
I go
> the standard way accross the board. And that made it. [Just one small
> differentiation between standards: some of the standards are open in the
> sense that you can freely make AND distribute complying implementations,
and
> some of the standards are open in the sense that you can make clean-room
> implementations, but you cannot distribute complying implementation
without
> further arrangement with the standard's owner/author/...the lawyers know
> better what. My understanding is that JNDI, JDBC and JTA falls in the
former
> category and EJB and Servlets in the later, but I may be completely
> wrong.]))

[I left the above paragraph in tact for clarity]

To be honest, it sounds like your going through alot of work for very
little benefit.  You're saying that if your stuff encounters bad code
(where the conenction isn't committed/closed/rollback) and don't want
to wait for the timer, you want to still use the connection?  I point
to your statment "During this time, the thread can be reused by the
ORB to process another request" How much benefit is there with this?
(I'm not trying to say don't work on this problem, I'm trying to
understand the real issue you have with it.)

I don't think the EJB fails in this regard you mention.  Servlets do
simply because servlets aren't particular with what you do in it. EJB
is very particular.

(Side note, how familiar are you with EJB applications?  I'm asking
because I think EJB containers solve alot of what your trying to write
on the application server level.  I view our support of JTA, etc using
the XA classes vital for gaining acceptance with more EJB containers.
I want WebLogic to offer out-of-the-box a CMP persistance layer that
maps (at 100%) with postgresql one day.  This is my ultimate goal.
Course, if JBoss has their wish, WebLogic will be a non-product by
then anwaysy. ;-)

[ .. ]

> Summary: I need Exoffice's xa package, because it can be used to
integrate
> PostgreSQL into a JTA implementation. I am not interested in how it
> impelements 2pc, whether it fakes or not or whether it implements 2pc at
> all. You cannot use PosgtreSQL for 2pc anyway (as we already repeated it
too
> many times).

This is why I say our goals may be different, but they are not
othognal.  I too want to see us fully enabled in a JTA implementation,
though I am interested in a two-phase commit since we get the most
'bang-for-the-buck' that way.

> So what kind of pooling is done in XADataSourceImpl? The best way to
> describe it is going through a scenario.
>
> We have the following components:
> -- DataSource: implemented in the middleware;
> -- Pool: a pool of connections implemented in the middleware;
> -- PostgresqlXADataSource: implemented by the jdbc driver. Itself
implements
> org.postgresql.xa.XADataSourceImpl.

Here is the initial issue... When you say 'we have the following, you
say DataSource in the middleware.  I disagree. The datasource is part
of the JDBC driver.  The pool is 'extra' from the needs of the jdbc
driver.  You have clearly convinced me of that.  We do not have a
functioning org.postgresql.xa.XADataSourceImpl only because its not
complete.  Some of the issues are the two-phase-commits, which you
clearly do not need.  But for me, that is the primary issue.

[ .. I won't reproduce the following section here to save space, but let
  others review it in the archive..]

You have a good set of information about the XA package, and thanks
for helping to clarify it for me.  It helped me decide to continue
working on the pool in my original format, really for one reason; the
tools that the XA package is meant to provide are not necessary for my
pool.  They solve two different problems, though share the same base
concept.

[ .. ]

I realize that I'm quoting out of context, but please bear with me...

> committed (and the commit() has successfully been called on the physical
> connection). It is clear that when a connection is requested from
> XADataSourceImpl, it will first look for a free one in its internal pool
> before creating a new one, but this pooling mechanism does not (and in
fact,
> based on the spec, is not supposed to) do anything along lines of
meeting
> pooling requriement (b). I can imagine for example an RDMBS-JDBC driver

Well, it comes across in the source code that its trying to do this,
regardless that it may not be to spec.  This seems like one area that
may be wrong in the XA package.

> combination, where physical connections can be effectively detached from
and
> attached to transactions. In such a case, the JDBC driver does not need
to
> implement any internal pooling. The XADataSourceImpl in our case needs
to

I agree 100%.  The requirements of the jdbc2.0 optional api spec do
not require that the jdbc driver implement any internal pooling.
However, I want to provide it for folks so that they can use it
out-of-the-box anyways.

I can still claim that I'm working on getting the driver to compliance
to jdbc2.0 optional for when I work on rowsets and two-phase commits.
Also, not that I'm trying to see what I can and cannot reuse of the
current xa implementation.  Example: I can still user the
XAConnectionImpl.java instead of my wrapped connection because of how
it uses event listener. (My submittal had my own version, but I'm
working on a new submittal that reuses that implementation.)  Looking
at the spec, my implementation can make use of XAConnections and
XAResource objects easily.  (As long as these XA classes (and the
PostgresDataSource are up to spec.)

Before we go any further, I've laid out my plans of what I'm working
on the project.  Again, they are
 --pooling implementation
 --rowset implementation
 --two-phase commits

From our discussions, I don't think you believe I shouldn't be working
on these items, just that they are not what you would think is the most
important to work on.  Perhaps you can layout (in this simple format)
what you are going to work on? (Or, a request of what you would like)
so that a) others can see if they want to help and b) people can see
if they want it in the TODO file.

=====
Virtually,        |                   "Must you shout too?"
Ned Wolpert       |                                  -Dante
wolpert@yahoo.com |
_________________/              "Who watches the watchmen?"
4e75                                       -Juvenal, 120 AD

-- Place your commercial here --                      fnord

__________________________________________________
Do You Yahoo!?
Send your FREE holiday greetings online!
http://greetings.yahoo.com