Re: psycopg and gevent with pgbouncer

Поиск
Список
Период
Сортировка
От Raoul Duke
Тема Re: psycopg and gevent with pgbouncer
Дата
Msg-id CAF1_pe_52ss290Yh0Ms4JSM0E-Xn1hXL_mC+Zemy2cuneHJOrA@mail.gmail.com
обсуждение исходный текст
Ответ на Re: psycopg and gevent with pgbouncer  (Daniele Varrazzo <daniele.varrazzo@gmail.com>)
Список psycopg
On Thu, Oct 6, 2011 at 3:53 PM, Daniele Varrazzo <daniele.varrazzo@gmail.com> wrote:
On Thu, Oct 6, 2011 at 12:41 PM, Raoul Duke <knevcher@gmail.com> wrote:

> I use psycopgreen for gevent and psycopg2 and it works fine.

Good. Note that you haven't included the mailing list in your reply.
If your problem is solved, let it know to other who may stumble in
your request via google.
 
I don't understand. How can I do that?



> Is there any way to detect if my application reconnect to db (pgbouncer) on
> each client request? I use psygopg2.pool.SimpleConnectionPool.

Why do you need the psycopg2 pool if you are using pgbouncer? It
shouldn't save you a lot.
 
I use pooler to limit requests to DB. If decide not use pgbouncer I think pooler is required.


The simple connection pool is not appropriate for concurrent programs:
you should use probably the ThreadedConnectionPool, after checking
that the lock it uses is correctly made green by gevent

I use this tool https://github.com/tellapart/gevent_request_profiler to check if my app has locks.


On Thu, Oct 6, 2011 at 2:31 PM, Daniele Varrazzo <daniele.varrazzo@gmail.com> wrote:
On Thu, Oct 6, 2011 at 9:54 AM, Marko Kreen <markokr@gmail.com> wrote:
> On Thu, Oct 6, 2011 at 10:45 AM, Raoul Duke <knevcher@gmail.com> wrote:

>> 2. I use psycopg simple connection pooler :
>> pool = SimpleConnectionPool(1, 3, connection_string)
>>
>> if I add async option:
>> pool = SimpleConnectionPool(1, 3, connection_string, async=1)
>>
>> I've got an error, than I can't understand:
>>   File "/usr/lib/pymodules/python2.6/sqlalchemy/engine/base.py", line 940,
>> in __create_execution_context
>>     self._handle_dbapi_exception(e, kwargs.get('statement', None),
>> kwargs.get('parameters', None), None, None)
>>   File "/usr/lib/pymodules/python2.6/sqlalchemy/engine/base.py", line 931,
>> in _handle_dbapi_exception
>>     raise exc.DBAPIError.instance(statement, parameters, e,
>> connection_invalidated=is_disconnect)
>> OperationalError: (OperationalError) asynchronous connection attempt
>> underway None [{}]
>>
>> Do I understand that when using the gevent, I have to use async option?
>
> I leave that to psycopg guys to answer.

An async connection it is not usable just after it's been created:
it's in an half baked state where poll() returns POLL_READ or
POLL_WRITE until the client and the server have completed the
asynchronous login handshake. The protocol must be pushed by the
client as described in
<http://initd.org/psycopg/docs/advanced.html#asynchronous-support>.
The wait loop should be somewhere after the connection creation and
before the usage: I don't know if the libraries you are using allow
that.

Note that if you are using gevent and not twisted you may use the
green mode instead
<http://initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries>,
that provides you async connections whose behaviour is entirely
equivalent to the blocking one (they block a single greenlet and
schedule the other to go). I suspect you should try them instead of
the raw async mode.

-- Daniele

В списке psycopg по дате отправления:

Предыдущее
От: Raoul Duke
Дата:
Сообщение: Re: psycopg and gevent with pgbouncer
Следующее
От: Matthew Robinson
Дата:
Сообщение: 2.4.2: undefined symbol: lo_truncate