Re: pg_background (and more parallelism infrastructure patches)

On 29/07/14 18:51, Robert Haas wrote:
> On Mon, Jul 28, 2014 at 1:50 PM, Andres Freund <andres@2ndquadrant.com> wrote:
>> What I'm thinking of is providing an actual API for the writes instead
>> of hooking into the socket API in a couple places. I.e. have something
>> like
>>
>> typedef struct DestIO DestIO;
>>
>> struct DestIO
>> {
>>      void (*flush)(struct DestIO *io);
>>      int (*putbytes)(struct DestIO *io, const char *s, size_t len);
>>      int (*getbytes)(struct DestIO *io, const char *s, size_t len);
>>      ...
>> }
>>
>> and do everything through it. I haven't thought much about the specific
>> API we want, but abstracting the communication properly instead of
>> adding hooks here and there is imo much more likely to succeed in the
>> long term.
>
> This sounds suspiciously like the DestReceiver thing we've already
> got, except that the DestReceiver only applies to tuple results, not
> errors and notices and so on.  I'm not totally unamenable to a bigger
> refactoring here, but right now it looks to me like a solution in
> search of a problem.  The hooks are simple and seem to work fine; I
> don't want to add notation for its own sake.
>

I am not sure if what Andres proposed is the right solution, but I do 
agree here that the hook definitely isn't.

I am also not sure that I like the pq_redirect_to_shm_mq being directly 
exposed for use in bgworker, what I would like is to have elog interface 
to which you tell that you want errors sent to shm_mq handle and that 
one would then do the necessary calls (It's basically same principle but 
for the sake of cleanliness/correctness I think it should be elog and 
not pq from the point of bgworker).

So the interface needs work.

I do agree that we don't need two way communication over this channel, I 
think the dsm_mq can be used directly quite well for generic usecase.

Otherwise I like the patch, and I am glad you also made the GUC 
save/restore infrastructure.

Please don't forget to add this to next commitfest.


--  Petr Jelinek                  http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &
Services