Обсуждение: Fwd: Re: Integration of Psycopg with XTA

Поиск
Список
Период
Сортировка

Fwd: Re: Integration of Psycopg with XTA

От
Federico Di Gregorio
Дата:
On 10/04/2018 12:30 PM, Daniele Varrazzo wrote:
 > On Thu, Oct 4, 2018 at 11:24 AM Federico Di Gregorio<fog@dndg.it>  wrote:
 >> On 10/04/2018 12:13 PM, Daniele Varrazzo wrote:
 >>> On Thu, Oct 4, 2018 at 10:33 AM Federico Di Gregorio<fog@dndg.it> 
wrote:
 >>>> @Daniele, I see that a capsule has an implicit state, "invalid" if the
 >>>> pointer inside it is null. Does it makes sense to reuse the same 
capsule
 >>>> and keep a reference to it to be able to invalidate it when the
 >>>> connection is closed?
 >>> I can take a look at it but I suspect it's a lot of work for a feature
 >>> very rarely used.
 >> Nah, I can do it. I was just asking for your opinion. :D
 > I don't know much about the capsule object, its life cycle etc. Yes,
 > not a huge deal saving it in the connection state and returning always
 > the same. I just wonder if it's worth the effort.

I suppose it helps to debug the situation where the capsule is used 
after the connection has been closed. Instead of a BOOM you get a nice 
exception (if the consumer checks for it, obviously).

I'll work on this in my repo and if everything works we can integrate it 
for 2.8.

federico

-- 
Federico Di Gregorio                         federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it
   Gli avvoltoi cinesi si nutrono di arte, ma possono anche mangiare
    i `domani'.                                        -- Haruki Murakami


Re: Integration of Psycopg with XTA

От
Christian Ferrari
Дата:
Dear Federico,
I will test it as soon as it will be released on git.
Kind Regards
Ch.F.

-------------------------------------------------------------
Good design can't stop bad implementation


Il giovedì 4 ottobre 2018, 12:36:53 CEST, Federico Di Gregorio <fog@dndg.it> ha scritto:


On 10/04/2018 12:30 PM, Daniele Varrazzo wrote:
> On Thu, Oct 4, 2018 at 11:24 AM Federico Di Gregorio<fog@dndg.it>  wrote:
>> On 10/04/2018 12:13 PM, Daniele Varrazzo wrote:
>>> On Thu, Oct 4, 2018 at 10:33 AM Federico Di Gregorio<fog@dndg.it>
wrote:
>>>> @Daniele, I see that a capsule has an implicit state, "invalid" if the
>>>> pointer inside it is null. Does it makes sense to reuse the same
capsule
>>>> and keep a reference to it to be able to invalidate it when the
>>>> connection is closed?
>>> I can take a look at it but I suspect it's a lot of work for a feature
>>> very rarely used.
>> Nah, I can do it. I was just asking for your opinion. :D
> I don't know much about the capsule object, its life cycle etc. Yes,
> not a huge deal saving it in the connection state and returning always
> the same. I just wonder if it's worth the effort.

I suppose it helps to debug the situation where the capsule is used
after the connection has been closed. Instead of a BOOM you get a nice
exception (if the consumer checks for it, obviously).

I'll work on this in my repo and if everything works we can integrate it
for 2.8.

federico

--
Federico Di Gregorio                        federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it
  Gli avvoltoi cinesi si nutrono di arte, ma possono anche mangiare
    i `domani'.                                        -- Haruki Murakami

Re: Integration of Psycopg with XTA

От
Christian Ferrari
Дата:
In a previous mail I stated something like "the name of the method doesn't matter"...
... after better thinking I realized a decent name might be better than other ones...
Let me say:

get_native_conn()
get_native_connection()

might be better than "get_something_pgconn" because the same type of method could be useful for other databases in the future...
More or less all the databases use a "native connection", only PostgreSQL uses "PGconn connection".

Have a good w.e.
Ch.F.

-------------------------------------------------------------
Good design can't stop bad implementation


Il giovedì 4 ottobre 2018, 12:36:53 CEST, Federico Di Gregorio <fog@dndg.it> ha scritto:


On 10/04/2018 12:30 PM, Daniele Varrazzo wrote:
> On Thu, Oct 4, 2018 at 11:24 AM Federico Di Gregorio<fog@dndg.it>  wrote:
>> On 10/04/2018 12:13 PM, Daniele Varrazzo wrote:
>>> On Thu, Oct 4, 2018 at 10:33 AM Federico Di Gregorio<fog@dndg.it>
wrote:
>>>> @Daniele, I see that a capsule has an implicit state, "invalid" if the
>>>> pointer inside it is null. Does it makes sense to reuse the same
capsule
>>>> and keep a reference to it to be able to invalidate it when the
>>>> connection is closed?
>>> I can take a look at it but I suspect it's a lot of work for a feature
>>> very rarely used.
>> Nah, I can do it. I was just asking for your opinion. :D
> I don't know much about the capsule object, its life cycle etc. Yes,
> not a huge deal saving it in the connection state and returning always
> the same. I just wonder if it's worth the effort.

I suppose it helps to debug the situation where the capsule is used
after the connection has been closed. Instead of a BOOM you get a nice
exception (if the consumer checks for it, obviously).

I'll work on this in my repo and if everything works we can integrate it
for 2.8.

federico

--
Federico Di Gregorio                        federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it
  Gli avvoltoi cinesi si nutrono di arte, ma possono anche mangiare
    i `domani'.                                        -- Haruki Murakami

Re: Integration of Psycopg with XTA

От
Federico Di Gregorio
Дата:
On 10/05/2018 06:02 PM, Christian Ferrari wrote:
> In a previous mail I stated something like "the name of the method 
> doesn't matter"...
> ... after better thinking I realized a decent name might be better than 
> other ones...
> Let me say:
> 
> get_native_conn()
> get_native_connection()
> 
> might be better than "get_/something/_pgconn" because the same type of 
> method could be useful for other databases in the future...
> More or less all the databases use a "native connection", only 
> PostgreSQL uses "PGconn connection".

After some more checking I discovered that it isn't possible to 
"invalidate" a PyCapsule so there is no way to tell if the pointer 
inside it is still a valid libpq connection.

The current code (feel free to pull) adds a new method

get_native_connection()

that raise an exception if called on a closed connection. I better like 
a method here because it gives the idea that you need to call it every 
time you want a Capsule to pass to the target C API.

I also added some docs and a dumb test.

https://github.com/fogzot/psycopg2/tree/feature-expose-pgconn

@Daniele, if Christian reports no problems I'll merge this later for 2.8.

federico

-- 
Federico Di Gregorio                         federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it
  Io non sono romantica. La candelina sul tavolo mi vede e si spegne.
                                                       -- sisterconfusion


Re: Integration of Psycopg with XTA

От
Christian Ferrari
Дата:
Hi Federico,
I've just tryed out commit https://github.com/fogzot/psycopg2/commit/81addddaee2c690e925bb8f381e7bcb02ca97687 : get_native_connection() is in the interface, but unfortunately it seems to work differently than the old _raw_pgconn.

Here's my code snippet:

pgconn = rm1.get_native_connection()
#pgconn = rm1._raw_pgconn                                                       
xar1 = PostgresqlXaResource(pgconn, "PostgreSQL", "dbname=testdb")

using get_native_connection(), I catch an error from XTA, the PGconn pointer is null.

I have reversed back to commit https://github.com/fogzot/psycopg2/commit/ccae5cae34051d6640ec6cbce6d9313778c509d3 : _raw_pgconn works as before.

Here's the code snippet:

#pgconn = rm1.get_native_connection()                                           
pgconn = rm1._raw_pgconn
xar1 = PostgresqlXaResource(pgconn, "PostgreSQL", "dbname=testdb")

It works fine.

Should "get_native_connection()" used differently than "_raw_pgconn" ? What type of data are necessary to investigate the issue?

Thanks in advance
Kind Regards
Ch.F.

-------------------------------------------------------------
Good design can't stop bad implementation


Il domenica 7 ottobre 2018, 13:56:49 CEST, Federico Di Gregorio <fog@dndg.it> ha scritto:


On 10/05/2018 06:02 PM, Christian Ferrari wrote:
> In a previous mail I stated something like "the name of the method
> doesn't matter"...
> ... after better thinking I realized a decent name might be better than
> other ones...
> Let me say:
>
> get_native_conn()
> get_native_connection()
>
> might be better than "get_/something/_pgconn" because the same type of
> method could be useful for other databases in the future...
> More or less all the databases use a "native connection", only
> PostgreSQL uses "PGconn connection".

After some more checking I discovered that it isn't possible to
"invalidate" a PyCapsule so there is no way to tell if the pointer
inside it is still a valid libpq connection.

The current code (feel free to pull) adds a new method

get_native_connection()

that raise an exception if called on a closed connection. I better like
a method here because it gives the idea that you need to call it every
time you want a Capsule to pass to the target C API.

I also added some docs and a dumb test.

https://github.com/fogzot/psycopg2/tree/feature-expose-pgconn

@Daniele, if Christian reports no problems I'll merge this later for 2.8.


federico

--
Federico Di Gregorio                        federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it

  Io non sono romantica. La candelina sul tavolo mi vede e si spegne.
                                                      -- sisterconfusion


Re: Integration of Psycopg with XTA

От
Christian Ferrari
Дата:
Sorry for disturbing with my previous email... :(
It was my fault: I didn't remember to check the key associated to the pointer in the capsule, now it's "psycopg2.connection.native_connection"...

Everything works as expected, get_native_connection() can be even inlined like below:

xar1 = PostgresqlXaResource(rm1.get_native_connection(), "PostgreSQL", "dbname=testdb")

Thank you so much for your support.
Kind Regards
Ch.F.

Il domenica 7 ottobre 2018, 18:42:31 CEST, Christian Ferrari <camauz@yahoo.com> ha scritto:


Hi Federico,
I've just tryed out commit https://github.com/fogzot/psycopg2/commit/81addddaee2c690e925bb8f381e7bcb02ca97687 : get_native_connection() is in the interface, but unfortunately it seems to work differently than the old _raw_pgconn.

Here's my code snippet:

pgconn = rm1.get_native_connection()
#pgconn = rm1._raw_pgconn                                                       
xar1 = PostgresqlXaResource(pgconn, "PostgreSQL", "dbname=testdb")

using get_native_connection(), I catch an error from XTA, the PGconn pointer is null.

I have reversed back to commit https://github.com/fogzot/psycopg2/commit/ccae5cae34051d6640ec6cbce6d9313778c509d3 : _raw_pgconn works as before.

Here's the code snippet:

#pgconn = rm1.get_native_connection()                                           
pgconn = rm1._raw_pgconn
xar1 = PostgresqlXaResource(pgconn, "PostgreSQL", "dbname=testdb")

It works fine.

Should "get_native_connection()" used differently than "_raw_pgconn" ? What type of data are necessary to investigate the issue?

Thanks in advance
Kind Regards
Ch.F.

-------------------------------------------------------------
Good design can't stop bad implementation


Il domenica 7 ottobre 2018, 13:56:49 CEST, Federico Di Gregorio <fog@dndg.it> ha scritto:


On 10/05/2018 06:02 PM, Christian Ferrari wrote:
> In a previous mail I stated something like "the name of the method
> doesn't matter"...
> ... after better thinking I realized a decent name might be better than
> other ones...
> Let me say:
>
> get_native_conn()
> get_native_connection()
>
> might be better than "get_/something/_pgconn" because the same type of
> method could be useful for other databases in the future...
> More or less all the databases use a "native connection", only
> PostgreSQL uses "PGconn connection".

After some more checking I discovered that it isn't possible to
"invalidate" a PyCapsule so there is no way to tell if the pointer
inside it is still a valid libpq connection.

The current code (feel free to pull) adds a new method

get_native_connection()

that raise an exception if called on a closed connection. I better like
a method here because it gives the idea that you need to call it every
time you want a Capsule to pass to the target C API.

I also added some docs and a dumb test.

https://github.com/fogzot/psycopg2/tree/feature-expose-pgconn

@Daniele, if Christian reports no problems I'll merge this later for 2.8.


federico

--
Federico Di Gregorio                        federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it

  Io non sono romantica. La candelina sul tavolo mi vede e si spegne.
                                                      -- sisterconfusion


Re: Integration of Psycopg with XTA

От
Daniele Varrazzo
Дата:
On Sun, Oct 7, 2018 at 12:56 PM Federico Di Gregorio <fog@dndg.it> wrote:

> After some more checking I discovered that it isn't possible to
> "invalidate" a PyCapsule so there is no way to tell if the pointer
> inside it is still a valid libpq connection.
>
> The current code (feel free to pull) adds a new method
>
> get_native_connection()
>
> that raise an exception if called on a closed connection. I better like
> a method here because it gives the idea that you need to call it every
> time you want a Capsule to pass to the target C API.
>
> I also added some docs and a dumb test.
>
> https://github.com/fogzot/psycopg2/tree/feature-expose-pgconn
>
> @Daniele, if Christian reports no problems I'll merge this later for 2.8.

LGTM. Do we also want a cursor.get_native_result() to return its PGresult?

-- Daniele



Re: Integration of Psycopg with XTA

От
Daniele Varrazzo
Дата:
On Sun, Oct 7, 2018 at 12:56 PM Federico Di Gregorio <fog@dndg.it> wrote:

> After some more checking I discovered that it isn't possible to
> "invalidate" a PyCapsule so there is no way to tell if the pointer
> inside it is still a valid libpq connection.
>
> The current code (feel free to pull) adds a new method
>
> get_native_connection()
>
> that raise an exception if called on a closed connection. I better like
> a method here because it gives the idea that you need to call it every
> time you want a Capsule to pass to the target C API.
>
> I also added some docs and a dumb test.
>
> https://github.com/fogzot/psycopg2/tree/feature-expose-pgconn
>
> @Daniele, if Christian reports no problems I'll merge this later for 2.8.

LGTM. Do we also want a cursor.get_native_result() to return its PGresult?

-- Daniele


Re: Integration of Psycopg with XTA

От
Christian Ferrari
Дата:
> LGTM. Do we also want a cursor.get_native_result() to return its PGresult?

> -- Daniele


I have no idea about a use case that can use cursor.get_native_result() ...
...but who can tell?!

Kind Regards
Ch.F.

Re: Integration of Psycopg with XTA

От
Christian Ferrari
Дата:
> LGTM. Do we also want a cursor.get_native_result() to return its PGresult?

> -- Daniele


I have no idea about a use case that can use cursor.get_native_result() ...
...but who can tell?!

Kind Regards
Ch.F.

Re: Integration of Psycopg with XTA

От
Christian Ferrari
Дата:
By the way, PyMySQL/mysqlclient-python project asked me to rename the method from "get_native_conn()" to "_get_native_conn()".

I'm not a "naming addicted person", but from the developer point of view, using two different methods would be for sure not appreciated...

xar1 = PostgresqlXaResource(rm1.get_native_connection(), "PostgreSQL", "dbname=testdb")
xar2 = MysqlXaResource(rm2._get_native_connection(), "MySQL", "localhost,0,lixa,,lixa")

If you don't mind, I kindly ask you to wait a couple of days: I hope to close the pull request https://github.com/PyMySQL/mysqlclient-python/pull/269 shortly...

Thank you in advance
Kind Regards
Ch.F.

-------------------------------------------------------------
Good design can't stop bad implementation

Il domenica 7 ottobre 2018, 13:56:49 CEST, Federico Di Gregorio <fog@dndg.it> ha scritto:


On 10/05/2018 06:02 PM, Christian Ferrari wrote:
> In a previous mail I stated something like "the name of the method
> doesn't matter"...
> ... after better thinking I realized a decent name might be better than
> other ones...
> Let me say:
>
> get_native_conn()
> get_native_connection()
>
> might be better than "get_/something/_pgconn" because the same type of
> method could be useful for other databases in the future...
> More or less all the databases use a "native connection", only
> PostgreSQL uses "PGconn connection".

After some more checking I discovered that it isn't possible to
"invalidate" a PyCapsule so there is no way to tell if the pointer
inside it is still a valid libpq connection.

The current code (feel free to pull) adds a new method

get_native_connection()

that raise an exception if called on a closed connection. I better like
a method here because it gives the idea that you need to call it every
time you want a Capsule to pass to the target C API.

I also added some docs and a dumb test.

https://github.com/fogzot/psycopg2/tree/feature-expose-pgconn

@Daniele, if Christian reports no problems I'll merge this later for 2.8.


federico

--
Federico Di Gregorio                        federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it

  Io non sono romantica. La candelina sul tavolo mi vede e si spegne.
                                                      -- sisterconfusion


Re: Integration of Psycopg with XTA

От
Federico Di Gregorio
Дата:
On 10/10/2018 11:10 PM, Christian Ferrari wrote:
> By the way, PyMySQL/mysqlclient-python project 
> <https://github.com/PyMySQL/mysqlclient-python> asked me to rename the 
> method from "get_native_conn()" to "_get_native_conn()".
> 
> I'm not a "naming addicted person", but from the developer point of 
> view, using two different methods would be for sure not appreciated...
> 
> xar1 = PostgresqlXaResource(rm1.get_native_connection(), "PostgreSQL", 
> "dbname=testdb")
> xar2 = MysqlXaResource(rm2._get_native_connection(), "MySQL", 
> "localhost,0,lixa,,lixa")
> 
> If you don't mind, I kindly ask you to wait a couple of days: I hope to 
> close the pull request 
> https://github.com/PyMySQL/mysqlclient-python/pull/269 shortly...

I think that _if_ this should go cross-database a method without the 
underscore is better. The underscore usually means private and that's 
good as long as it is just that. An informal "standard" method name 
should not have the "_" in it.

federico

-- 
Federico Di Gregorio                         federico.digregorio@dndg.it
DNDG srl                                                  http://dndg.it
   Gli esseri umani, a volte, sono destinati, per il solo fatto di
    esistere, a fare del male a qualcuno.              -- Haruki Murakami


Re: Integration of Psycopg with XTA

От
Daniele Varrazzo
Дата:
On Wed, Oct 10, 2018 at 10:19 PM Federico Di Gregorio <fog@dndg.it> wrote:
>
> On 10/10/2018 11:10 PM, Christian Ferrari wrote:
> > By the way, PyMySQL/mysqlclient-python project
> > <https://github.com/PyMySQL/mysqlclient-python> asked me to rename the
> > method from "get_native_conn()" to "_get_native_conn()".

> I think that _if_ this should go cross-database a method without the
> underscore is better. The underscore usually means private and that's
> good as long as it is just that. An informal "standard" method name
> should not have the "_" in it.

+1

-- Daniele