Обсуждение: Time to fix libpgtcl for async NOTIFY

About a month ago I wrote
> Yes, if anything were to be done along this line it'd also make sense
> to revise libpgtcl.  I think it ought to work more like this:
>   (a) the idle loop is invoked while waiting for a query response
>       (so that a pg_exec statement behaves sort of like "tkwait");
>   (b) a "listen" command is sent via a new pg_listen statement that
>       specifies a callback command string.  Subsequent notify responses
>       can occur whenever a callback is possible.
> I suppose (a) had better be an option to pg_exec statements so that
> we don't break existing Tcl code...

I was a little startled to discover that pg_listen already exists
(it's not documented!).  But the way things are currently set up,
the callbacks specified by pg_listen are executed only when the Tcl
script invokes the also-undocumented pg_notifies command.

pg_notifies executes an empty query (which is no longer necessary) and
then looks for notify responses.  A practical Tcl application would have
to execute pg_notifies often, perhaps every few seconds from an "after"
timer event.

What I'd like to do is eliminate pg_notifies and define pg_listen
callbacks as happening automatically whenever Tcl can fire event
callbacks (ie, at the idle loop).

There's some risk of breaking existing applications, since the apps
might not be prepared for pg_listen callbacks occurring except at the
specific time they execute pg_notifies.  But that doesn't seem really
likely to be a problem.  Besides, since both these commands are
undocumented, I imagine not very many libpgtcl applications use them.
(A quick search of the archives turned up only a message from Massimo
Dal Zotto about this topic, and he seemed to agree that getting rid of
pg_notifies would be better.)

Any comments?

BTW, I'm not currently planning to tackle the other point about providing
an asynchronous pg_exec capability in libpgtcl.  I've concluded that the
Tcl code I'm planning to write wouldn't use it, at least not soon; so
I'll leave that project for another person or another day.

            regards, tom lane

> There's some risk of breaking existing applications, since the apps
> might not be prepared for pg_listen callbacks occurring except at the
> specific time they execute pg_notifies.  But that doesn't seem really
> likely to be a problem.  Besides, since both these commands are
> undocumented, I imagine not very many libpgtcl applications use them.
> (A quick search of the archives turned up only a message from Massimo
> Dal Zotto about this topic, and he seemed to agree that getting rid of
> pg_notifies would be better.)
>
> Any comments?

Yep, get rid of the old stuff.  I am sure people didn't use it because
of the performance problem.  Your cleanup will make it use-able.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

>
> > There's some risk of breaking existing applications, since the apps
> > might not be prepared for pg_listen callbacks occurring except at the
> > specific time they execute pg_notifies.  But that doesn't seem really
> > likely to be a problem.  Besides, since both these commands are
> > undocumented, I imagine not very many libpgtcl applications use them.
> > (A quick search of the archives turned up only a message from Massimo
> > Dal Zotto about this topic, and he seemed to agree that getting rid of
> > pg_notifies would be better.)
> >
> > Any comments?
>
> Yep, get rid of the old stuff.  I am sure people didn't use it because
> of the performance problem.  Your cleanup will make it use-able.
>
> --
> Bruce Momjian                          |  830 Blythe Avenue
> maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
>   +  If your life is a hard drive,     |  (610) 353-9879(w)
>   +  Christ can be your backup.        |  (610) 853-3000(h)
>

The old stuff works fine, at least for me!

I'm using it, and my users like it very much. It suffers of performance
problems but they are caused by bottlenecks on pg_listener and not by
the pg_notifies loop in tcl (wich is done every 1 second by 30+ clients).
I'm probably the only one who uses this feature, partly because I never
found the time to write down some documentation about it (my fault), so
any change you make shouldn't break many applications.

If you are making changes to the tcl listen code please consider adding the
following missing feature:

- the possibility to temporarily suspend callbacks for a particular listen
  or for all listens. This would help to avoid the above possible problem.
  I currently do it with ad-hoc code in the application but having it done
  by a single call to libpgtcl would be better.

- an option to get the callback associated to a particular listen or to all
  listens

- better support for unlistening. There is an unlisten function in my
  contrib code but it should be integrated into the backend with a real
  UNLISTEN command.

--
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto                e-mail:  dz@cs.unitn.it            |
|  Via Marconi, 141                 phone:  ++39-461-534251            |
|  38057 Pergine Valsugana (TN)     www:  http://www.cs.unitn.it/~dz/  |
|  Italy                            pgp:  finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+

Massimo Dal Zotto <dz@cs.unitn.it> writes:
> If you are making changes to the tcl listen code please consider adding the
> following missing feature:
> - the possibility to temporarily suspend callbacks for a particular listen
>   or for all listens. This would help to avoid the above possible problem.

That seems reasonable.  I presume that if a notify comes in while the
callback is suspended, we should fire the callback after it gets
reenabled?  What happens if multiple notifies for the same name arrive
while suspended --- one callback afterwards, or many?  (I think I'd vote
for only one, but...)

> - an option to get the callback associated to a particular listen or to all
>   listens

Perhaps also reasonable.  In the code I submitted last night, it is
possible for several different Tcl interpreters to be listening within
one client process.  I think that an interpreter should only be able to
ask about its own callbacks, however --- anything else is a security
violation that defeats the point of multiple interpreters.

(Your "suspend callbacks" feature would likewise have to be interpreter-
local.)

> - better support for unlistening. There is an unlisten function in my
>   contrib code but it should be integrated into the backend with a real
>   UNLISTEN command.

I agree with that, and will do the libpq and libpgtcl changes if someone
else will implement UNLISTEN on the backend side.  I'm not competent to
twiddle the backend however.

            regards, tom lane

PS: I'm on vacation till next week.  Don't expect any fast responses.