Обсуждение: Working toward a JTA 1.0.1 Compliant XADataSource

I'd like to start this thread by thanking everyone for their input on the questions I had regarding the existing XA
implementationthis week. You have all been very supportive and very encouraging. Special thanks to Heikki and the rest
ofthe community for providing the existing XA support and helping us understand it. 


For our application and middleware stack we've not had any luck making minor adjustments to improve the issues we've
beenhaving. My early panic about serialized methods did help to eliminate one class of errors in our test environment
(oncewe were finally able to duplicate the issue), but culminated in exposing the lack of interleaving or shareable
XAResourcesfor 2pc operations with the GlassFish transaction manager as the ultimate culprit for our issues. It seems
thatwhen GlassFish gets busy (and your resource pool gets busy) it starts expecting some aspects of interleaving -- in
theform of commit(xid1, false) after start(xid2, NOFLAGS) has been issued to an XAResource. 


As has been discussed on this list earlier in the week an implementation that truly implements JTA 1.0.1, or at least
supportsinterleaving will bring a new set of trade-offs. Such an implementation would also make more complex JEE
deploymentsatop PostgreSQL possible (and more reliable, as in our case) for a wider set of containers and transaction
managers.We love PostgreSQL. It's done very well for us in the past and we'd like to give back by making it a better
productfor everyone to use. This looks like a pretty good place to start doing that. 

It is my intent to contribute to an effort to expand and improve the XA support for PostgreSQL. I believe this can be
done.I believe a good balance of configurable trade-offs to meet more (if not all) of the JTA 1.0.1 specification can
befound. I believe a more compliant XA implementation can be robust and preform respectably. 

I also like to think this can happen fairly fast.

On behalf of the development team I've been working with on this the last few days, I'd like to invite any interested
partyto take a peek at what we're doing on github. 
https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA

I've waited to announce our work on this until I was convinced we had the ability to pull it off and that it was in a
stateworth asking for input on. 

Right now, the XADataSource implementation in our POLARIS_XA branch passes the existing XADataSource test suite -- even
theinterleaving tests that were commented out for the old implementation. 
The new implementation already has code to support:
  * resource sharing
  * interleaving
  * suspend / resume / join
  * Achieves functional parity (according to the junit tests) with the existing driver.

There are several of us still working on filling in the implementation. We know we have quite a ways to go before this
is'production ready'. 
That said our situation is such that we are working to get our immediate needs met while trying hard to do so in a
mannerthat will not degrade or break existing uses of the XADataSource. We're going to need help in making sure we
don'thave horrible regressions. 

We'd like to invite critiques, comments, or pull requests from the rest of the community. Especially those of you
who'vehelped us reach this point (weather you knew it or not)! What can we do better? What things have we done wrong?
Whatproblems do you foresee? 
Now that there's what I'd consider a functional prototype let's get the conversation started.

What's the next step to getting this pushed upstream? What barriers do you foresee, and what can I (or anyone else) do
inthe meantime? Is this the appropriate communication channel for such things? 

Regards,
-Bryan Varner

Senior Developer
POLARIS Laboratories

The statements, views, and opinions expressed in this email are my own personal remarks, and do no represent or imply
anyobligation on behalf of my employer, POLARIS Laboratories. 

P.S. I'm scheduled to be on vacation next week, but I have a sneaking suspicion I'll be keeping tabs on this mailing
list.The other developers from our team who subscribe to this group, (Kirk Winters : kwinters at polarislabs.com, and
AdamGray : agray at polarislabs.com) will also be keeping tabs. 

On 16.02.2013 09:16, Bryan Viarner wrote:
> For our application and middleware stack we've not had any luck making minor adjustments to improve the issues we've
beenhaving. My early panic about serialized methods did help to eliminate one class of errors in our test environment
(oncewe were finally able to duplicate the issue), but culminated in exposing the lack of interleaving or shareable
XAResourcesfor 2pc operations with the GlassFish transaction manager as the ultimate culprit for our issues. It seems
thatwhen GlassFish gets busy (and your resource pool gets busy) it starts expecting some aspects of interleaving -- in
theform of commit(xid1, false) after start(xid2, NOFLAGS) has been issued to an XAResource. 

Sorry to hear that. I'd be interested to have an isolated, stand-alone
test application that reproduces the problem, so that I could also play
with it.

> As has been discussed on this list earlier in the week an implementation that truly implements JTA 1.0.1, or at least
supportsinterleaving will bring a new set of trade-offs. Such an implementation would also make more complex JEE
deploymentsatop PostgreSQL possible (and more reliable, as in our case) for a wider set of containers and transaction
managers.

Frankly, I don't think this is worth the trouble, or worth complicating
the driver for. It would be nice to be fully compliant, just so we could
say that we are, but in practice, meh.

The PostgreSQL driver is not alone with the limitations. I believe
Oracle has similar limitations, as does MySQL, and DB2. Which is why all
application servers I've bumped into this far have options to work
around them. I'm very surprised if Glassfish doesn't. In practice, a
transaction manager that insists on working support for transaction
interleaving and suspend/resume, is pretty useless.

For example, in JBoss the option is called "track-connection-by-tx".
Since JBoss 5, the default is "true", so you don't need to remember to
set it manually anymore. I'm guessing that the JBoss developers decided
that it's better to set it by default, because in practice most
connector implementations require it to be set.

On Atomikos, the setting is called "com.atomikos.icatch.threaded_2pc",
and they also seem to have switched the default to false (ie. the right
option for PostgreSQL") in version 3.7.

IMHO it was a serious mistake from the JTA spec authors to require those
extra features. They are not required for robust two-phase commit, and
are difficult to implement in many JDBC drivers, including PostgreSQL's.

> We love PostgreSQL. It's done very well for us in the past and we'd like to give back by making it a better product
foreveryone to use. This looks like a pretty good place to start doing that. 

I think the best action here would be to expand the FAQ with specific
information on how to set up the data source on various app servers
(http://jdbc.postgresql.org/documentation/faq.html). And if Glassfish
doesn't yet provide a suitable work-around option, petition them to
implement one.

> On behalf of the development team I've been working with on this the last few days, I'd like to invite any interested
partyto take a peek at what we're doing on github. 
> https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA

There seems to be some unrelated refactoring in this branch, moving
files around, so it's a bit hard to see what the real changes are.

One random thought just occurred to me: while I don't think this is
worth the complexity in the PostgreSQL driver, would it be possible to
implement this as a generic wrapper that could be used with any
XADataSource implementation? The concept of implementing the transaction
interleaving and suspend/resume by adding yet another layer of pooling
seems pretty universal. It could then be used with any application
server which doesn't already provide a work-around option, and with any
JDBC implementation that has these limitations.

- Heikki

> Sorry to hear that. I'd be interested to have an isolated, stand-alone
> test application that reproduces the problem, so that I could also play
> with it.

Now that we've been able to reproduce it in our test environments, it's
far more likely we can do that. I'd be happy to send you some debug logs
from the pgjdbc driver, where it documented what was going on.

> Frankly, I don't think this is worth the trouble, or worth complicating
> the driver for. It would be nice to be fully compliant, just so we could
> say that we are, but in practice, meh.

If the existing implementation actually worked for my case, I'd agree
with you. It's not working for me, so at this point, I disagree.

> The PostgreSQL driver is not alone with the limitations. I believe
> Oracle has similar limitations, as does MySQL, and DB2. Which is why all
> application servers I've bumped into this far have options to work
> around them. I'm very surprised if Glassfish doesn't. In practice, a
> transaction manager that insists on working support for transaction
> interleaving and suspend/resume, is pretty useless.

While we are not seeing join / suspend / resume, we are seeing the
expectation that you can commit a previously prepared TX while already
working on a new TX. I've read the JTA specification through several
times at this point, and I'm not sure if that should be considered
interleaving, or properly implementing the resource sharing aspect of
XAResource.

Here's the configuration options for the GlassFish version we're using.
http://docs.oracle.com/cd/E18930_01/html/821-2416/beanp.html#scrolltoc

The closest thing to interleaving I see is
ALLOW_MULTIPLE_ENLISTS_DELISTS, which is disabled (false) by default.

> IMHO it was a serious mistake from the JTA spec authors to require those
> extra features. They are not required for robust two-phase commit, and
> are difficult to implement in many JDBC drivers, including PostgreSQL's.

I too, hope this gets fixed in the 2.0 spec.

>> We love PostgreSQL. It's done very well for us in the past and we'd
>> like to give back by making it a better product for everyone to use.
>> This looks like a pretty good place to start doing that.
>
> I think the best action here would be to expand the FAQ with specific
> information on how to set up the data source on various app servers
> (http://jdbc.postgresql.org/documentation/faq.html). And if Glassfish
> doesn't yet provide a suitable work-around option, petition them to
> implement one.

I have been in contact with a few of the GlassFish developers. I will
try to see if there is something they could do in a future version.

GlassFish is a bigger, slower moving project at this point, and I don't
have the time to wait for them to change their JTA implementation.

>> https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA
>
> There seems to be some unrelated refactoring in this branch, moving
> files around, so it's a bit hard to see what the real changes are.

To get better support for code completion in a few IDE's we moved things
into a 'src' directory, and updated the build.xml to handle it.

We added a protected method to the PGPoolingConnection so that
subclasses can obtain the underlying Connection without having to cache
it as a private member variable.

The XADataSource now implements most of the XA control invocation, and
extends AbstractXADataSource, which is the file filtered to extend the
proper spec version implementation by the ant build.

> One random thought just occurred to me: while I don't think this is
> worth the complexity in the PostgreSQL driver, would it be possible to
> implement this as a generic wrapper that could be used with any
> XADataSource implementation?

That's essentially what we're doing with this implementation... except
that we're tied to the pgjdbc build system to make sure we build for the
right version of the jdbc spec, and the PostgreSQL dialect.

It certainly would be 'possible' to wrap a normal DataSource into an XA
compliant pool and provide different implementations for different spec
versions and dialects.

> The concept of implementing the transaction
> interleaving and suspend/resume by adding yet another layer of pooling
> seems pretty universal. It could then be used with any application
> server which doesn't already provide a work-around option, and with any
> JDBC implementation that has these limitations.

Having it work for 'any' database is a lofty goal, would take a lot more
time, and won't help address the problem I have any faster. From a
practical engineering standpoint, what you're proposing is a very large
undertaking, and one that won't help me resolve my situation any faster
than sticking to one database, one driver, and two jdbc version specs.

Regards,
-Bryan Varner

For what it's worth Bryan I thing what you're doing is a good thing,
this makes the implementation more robust and I'll be testing what you
have as soon as I find the time.

Thanks,
Florent

On Fri, Feb 15, 2013 at 11:16 PM, Bryan Varner <bvarner@polarislabs.com> wrote:
> I'd like to start this thread by thanking everyone for their input on the questions I had regarding the existing XA
implementationthis week. You have all been very supportive and very encouraging. Special thanks to Heikki and the rest
ofthe community for providing the existing XA support and helping us understand it. 
>
>
> For our application and middleware stack we've not had any luck making minor adjustments to improve the issues we've
beenhaving. My early panic about serialized methods did help to eliminate one class of errors in our test environment
(oncewe were finally able to duplicate the issue), but culminated in exposing the lack of interleaving or shareable
XAResourcesfor 2pc operations with the GlassFish transaction manager as the ultimate culprit for our issues. It seems
thatwhen GlassFish gets busy (and your resource pool gets busy) it starts expecting some aspects of interleaving -- in
theform of commit(xid1, false) after start(xid2, NOFLAGS) has been issued to an XAResource. 
>
>
> As has been discussed on this list earlier in the week an implementation that truly implements JTA 1.0.1, or at least
supportsinterleaving will bring a new set of trade-offs. Such an implementation would also make more complex JEE
deploymentsatop PostgreSQL possible (and more reliable, as in our case) for a wider set of containers and transaction
managers.We love PostgreSQL. It's done very well for us in the past and we'd like to give back by making it a better
productfor everyone to use. This looks like a pretty good place to start doing that. 
>
> It is my intent to contribute to an effort to expand and improve the XA support for PostgreSQL. I believe this can be
done.I believe a good balance of configurable trade-offs to meet more (if not all) of the JTA 1.0.1 specification can
befound. I believe a more compliant XA implementation can be robust and preform respectably. 
>
> I also like to think this can happen fairly fast.
>
> On behalf of the development team I've been working with on this the last few days, I'd like to invite any interested
partyto take a peek at what we're doing on github. 
> https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA
>
> I've waited to announce our work on this until I was convinced we had the ability to pull it off and that it was in a
stateworth asking for input on. 
>
> Right now, the XADataSource implementation in our POLARIS_XA branch passes the existing XADataSource test suite --
eventhe interleaving tests that were commented out for the old implementation. 
> The new implementation already has code to support:
>   * resource sharing
>   * interleaving
>   * suspend / resume / join
>   * Achieves functional parity (according to the junit tests) with the existing driver.
>
> There are several of us still working on filling in the implementation. We know we have quite a ways to go before
thisis 'production ready'. 
> That said our situation is such that we are working to get our immediate needs met while trying hard to do so in a
mannerthat will not degrade or break existing uses of the XADataSource. We're going to need help in making sure we
don'thave horrible regressions. 
>
> We'd like to invite critiques, comments, or pull requests from the rest of the community. Especially those of you
who'vehelped us reach this point (weather you knew it or not)! What can we do better? What things have we done wrong?
Whatproblems do you foresee? 
> Now that there's what I'd consider a functional prototype let's get the conversation started.
>
> What's the next step to getting this pushed upstream? What barriers do you foresee, and what can I (or anyone else)
doin the meantime? Is this the appropriate communication channel for such things? 
>
> Regards,
> -Bryan Varner
>
> Senior Developer
> POLARIS Laboratories
>
> The statements, views, and opinions expressed in this email are my own personal remarks, and do no represent or imply
anyobligation on behalf of my employer, POLARIS Laboratories. 
>
> P.S. I'm scheduled to be on vacation next week, but I have a sneaking suspicion I'll be keeping tabs on this mailing
list.The other developers from our team who subscribe to this group, (Kirk Winters : kwinters at polarislabs.com, and
AdamGray : agray at polarislabs.com) will also be keeping tabs. 



--
Florent Guillaume, Director of R&D, Nuxeo
Open Source, Java EE based, Enterprise Content Management (ECM)
http://www.nuxeo.com   http://www.nuxeo.org   +33 1 40 33 79 87

Since it's been nearly two weeks since my first message, I thought I'd
send an update on where we are with our XA reimplementation.

Our production environment (GlassFish) is standing on it's own two feet
with our XA implementation.
https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA

We're seeing decent performance and (mostly) reliable operation.

We're hammering out a few obscure race condition bugs with regards to
threaded invocations of shared XAResources & the connection handle
usage. I expect we'll have these fixed and in our github repo by Monday.

We are passing all the existing XA pgjdbc tests, as well as the expanded
ones we've produced. The driver operates with and without interleaving
(you can disable the extra physical connection allocation with a property).

Our XA implementation also passes the JBoss TM (version 4.17) JDBC (XA
mode!) test suite.
https://github.com/jbosstm/narayana/tree/4.17


I know there were some complaints earlier in the thread that we moved
all the source code into a 'src' directory. We did this to support IDE
environments which don't like having the source folder be the same as
the root of the project (NetBeans comes to mind). We also fixed the
build.xml to _completely_ respect the property it already had for the
source directory location.

If the src directory and build changes are going to be an obstacle to
getting this work accepted upstream, now would be a good time to tell me.

If there's no hope of having this accepted upstream, now would be a good
time to tell me.

Again, I invite anyone to suggest changes, test it out, and report back
what you find.

Regards,
-Bryan Varner


----------------
Senior Developer
POLARIS Laboratories


On 02/20/2013 05:44 PM, Florent Guillaume wrote:
> For what it's worth Bryan I thing what you're doing is a good thing,
> this makes the implementation more robust and I'll be testing what you
> have as soon as I find the time.
>
> Thanks,
> Florent

Yes, moving everything to src will not work for the project.

Dave

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On Fri, Mar 1, 2013 at 3:48 PM, Bryan Varner <bvarner@polarislabs.com> wrote:

Since it's been nearly two weeks since my first message, I thought I'd send an update on where we are with our XA reimplementation.

Our production environment (GlassFish) is standing on it's own two feet with our XA implementation.
https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA

We're seeing decent performance and (mostly) reliable operation.

We're hammering out a few obscure race condition bugs with regards to threaded invocations of shared XAResources & the connection handle usage. I expect we'll have these fixed and in our github repo by Monday.

We are passing all the existing XA pgjdbc tests, as well as the expanded ones we've produced. The driver operates with and without interleaving (you can disable the extra physical connection allocation with a property).

Our XA implementation also passes the JBoss TM (version 4.17) JDBC (XA mode!) test suite.
https://github.com/jbosstm/narayana/tree/4.17


I know there were some complaints earlier in the thread that we moved all the source code into a 'src' directory. We did this to support IDE environments which don't like having the source folder be the same as the root of the project (NetBeans comes to mind). We also fixed the build.xml to _completely_ respect the property it already had for the source directory location.

If the src directory and build changes are going to be an obstacle to getting this work accepted upstream, now would be a good time to tell me.

If there's no hope of having this accepted upstream, now would be a good time to tell me.

Again, I invite anyone to suggest changes, test it out, and report back what you find.

Regards,
-Bryan Varner


----------------
Senior Developer
POLARIS Laboratories

On 02/20/2013 05:44 PM, Florent Guillaume wrote:

For what it's worth Bryan I thing what you're doing is a good thing,
this makes the implementation more robust and I'll be testing what you
have as soon as I find the time.

Thanks,
Florent

--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Dave Cramer <pg@fastcrypt.com> wrote:
> Hannu Krosing <hannu@2ndquadrant.com> wrote:
>> Dave Cramer wrote:
>>> Bryan Varner <bvarner@polarislabs.com> wrote:

>>>> If the src directory and build changes are going to be an
>>>> obstacle to getting this work accepted upstream, now would be
>>>> a good time to tell me.

>>> Yes, moving everything to src will not work for the project.

>> Could you also tell _why_ it would not work ?

> There's no technical reason as to why it would not work.
>
> The project has used this layout for as long as I can remember.
> If we were to change it for this patch why not change it for
> every patch ?

My opinions, FWIW:

(1)  A change like this should be done as a separate patch, so that
it doesn't make it harder to find substantive logical changes to
the code.  I recommend that you offer the improvements to
XADataSource without the source file relocation, and if you want to
convince people to reorganize the directory structure work on that
at a different time on a new thread.

(2)  I strongly prefer having a src directory below the top level.

(3)  I don't think anyone is advocating that each patch be allowed
to reorganize the directory structure; it's a question of which is
the best way to organize it long-term.

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

For what it's worth, there isn't any belligerence intended.  We're just merely asking in order to determine how much
morewe have  left to do before it can be considered.  Reverting the src directory change at this point is rather minor
comparedto the work that's been put into it so far.  There's still a little work to be done chasing down the corner
cases,but it's good to have the conversation now so the patch can be in an acceptable form when submitted rather than
gothrough a more extensive revision process after-the-fact. 

-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Kevin Grittner
Sent: Saturday, March 02, 2013 9:20 AM
To: Dave Cramer; Hannu Krosing
Cc: Bryan Varner; Florent Guillaume; pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] Working toward a JTA 1.0.1 Compliant XADataSource

Dave Cramer <pg@fastcrypt.com> wrote:
> Hannu Krosing <hannu@2ndquadrant.com> wrote:
>> Dave Cramer wrote:
>>> Bryan Varner <bvarner@polarislabs.com> wrote:

>>>> If the src directory and build changes are going to be an obstacle
>>>> to getting this work accepted upstream, now would be a good time to
>>>> tell me.

>>> Yes, moving everything to src will not work for the project.

>> Could you also tell _why_ it would not work ?

> There's no technical reason as to why it would not work.
>
> The project has used this layout for as long as I can remember.
> If we were to change it for this patch why not change it for every
> patch ?

My opinions, FWIW:

(1)  A change like this should be done as a separate patch, so that it doesn't make it harder to find substantive
logicalchanges to the code.  I recommend that you offer the improvements to XADataSource without the source file
relocation,and if you want to convince people to reorganize the directory structure work on that at a different time on
anew thread. 

(2)  I strongly prefer having a src directory below the top level.

(3)  I don't think anyone is advocating that each patch be allowed to reorganize the directory structure; it's a
questionof which is the best way to organize it long-term. 

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

+1 on using the src/main/java and src/main/test as this is now used by maven, gradle, and is widely understood
convention. 

I think it might even help in making it easier to create a maven build script for the driver, which might make it
easierto get the driver into maven central.  



On 2013-03-03, at 3:52 PM, Stephen Nelson <stephen@eccostudio.com> wrote:

> I think it would be better to adopt the Maven-style directory layout e.g src/main/java and src/test/java. It would
allowthe flexibility to move to other build systems in the future, if desired. Plus it seems quite a logical
arrangement.
>
> I also agree, that such a big change should form a separate patch that only performs the directory modifications, to
confirmall permutations of the existing build creates the same artifact after the re-arrangement. 
>
> Thanks
>
> Stephen

> I also agree, that such a big change should form a separate patch that
> only performs the directory modifications, to confirm all permutations
> of the existing build creates the same artifact after the re-arrangement.
> Thanks
> Stephen

I too, agree with this approach.

The current layout 'works' in netbeans, it just looks weird. There's no
way (that we can find) to filter out the 'build' or non-source directories.

In the future, I'd really like to see the build system moved to
something that allows us to easily push maven artifacts. I can't think
of any reason maven central wouldn't accept this project for inclusion.

Our POLARIS_XA branch currently has everything moved back to '.'.
However, I've retained the changes to build.xml which made it respect
the 'srcdir' property. These are pretty minor changes, and they're tied
up with our change to what java.in classes get filtered during the
build. I don't think this should cause too much of an issue, and it will
likely help make it clear what needs to be done if the build system is
altered in the future.

At this point, we're actively trying to prep for an upstream patch.
https://github.com/polarislabs/pgjdbc/compare/POLARIS_XA

What is the preferred method for doing this?
    Perhaps a github pull request? ;-)

In our branch, the XA junit tests have been expanded quite a bit to
exercise all the corner cases we've seen crop up. We've also been
running the PGXADataSource against the jboss tm (narayana) QA test suite
as an additional smoke-test. I will be getting in contact with the
glassfish project (again) this week to determine what it takes to get
the PGXADataSource listed as a supported configuration.

http://docs.oracle.com/cd/E18930_01/html/821-2416/beamw.html#gjksj

Interleaving is currently enabled by default, but can be disabled (set
the xaAcquireTimeout parameter to 0). We do have test coverage for both
situations. If a TM tries to do interleaving when it's disabled, you'll
get an appropriate exception message telling you what to do about it.

Due to the nature of interleaving and isolation (and the way we've
implemented it so far) there is not a 1-to-1 relationship between
connections returned from the PGXADataSource and the number of physical
DB connections to the back-end. The actual number of physical
connections needed to service the XA invocations is totally application,
workload, and TM dependent. If your TM never attempts interleaving, and
your application doesn't do anything in local TX scopes, it's likely
that you'll see 1-to-1 connection count ratios.

We've added a high-water mark count that logs when the logger is set to
'info' so you can get a fairly decent idea of what the driver is doing.

Because the physical connection count is so dependent upon application
and TM, I don't feel comfortable giving a rule-of-thumb for configuring
things here. What I can say is that, we're seeing that in cases of
high-volume, we have about 12 physical connections per JVM more than the
configured 'max pool size' in GlassFish.

That's all I can think of for now. If anyone wants to start testing
builds, it should be reasonably safe to do so now.

Regards,
-Bryan Varner