Re: Listen / Notify rewrite

Unfortunately with all that payload-length discussion, the other part
of my email regarding ACID compliant behavior got completely lost. I
would appreciate some input on that part also...

Thanks,
Joachim

On Thu, Nov 12, 2009 at 12:17 PM, Joachim Wieland <joe@mcknight.de> wrote:
> On Thu, Nov 12, 2009 at 4:25 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> One possible solution would be to write to the queue before committing
>>> and adding the TransactionID.  Then other backends can check if our
>>> TransactionID has successfully committed or not. Not sure if this is
>>> worth the overhead however...
>>
>> That sounds reasonable, and it's certainly no more overhead than the
>> tuple visibility checks that happen in the current implementation.
>
> I am not too concerned about the runtime of the visibility checks,
> instead I suppose that most of the overhead will come from waiting for
> another transaction to either commit or abort...
>
> If transaction t1 scans the queue and at some point finds
> notifications from t2, then it will ask for the status of t2. If it
> finds out that t2 is still running, then it has no other option than
> to stop working on the queue and wait (it will be signalled again
> later once t2 has finished).
>
> This also means that we cannot at the same time write notifications to
> the queue and read from it and if a transaction places a few million
> notifications into the queue, readers need to wait until it has
> finished and only after that they can continue and read the
> notifications...
>
> And it means that if the queue is full, we might run into a
> deadlock... A transaction adding notifications will wait for the
> readers to proceed and the readers wait for the transaction to commit
> or abort...
>
> One option could be to write new notifications to a subdirectory, and
> create a bunch of new segment files there. Once this is done, the
> segment files could be moved over and renamed, so that they continue
> the slru queue... If we run out of disk space while filling that
> temporary subdirectory, then we can just delete the subdirectory and
> nobody has been blocked. We could still run into errors moving and
> renaming the segment files (e.g. permission problems) so that we still
> might need to abort the transaction...
>
>
>>> 2. The payload parameter is optional. A notifying client can either call
>>> "NOTIFY foo;" or "NOTIFY foo 'payload';". The length of the payload is
>>> currently limited to 128 characters... Not sure if we should allow longer
>>> payload strings...
>>
>> Might be a good idea to make the max the same as the max length for
>> prepared transaction GUIDs?  Not sure anyone would be shipping those
>> around, but it's a pre-existing limit of about the same size.
>
> Yes, sounds reasonable to have the same limit for user-defined identifiers...
>
>
> Joachim
>