Re: bg worker: patch 1 of 6 - permanent process

Robert,

On 09/15/2010 07:23 PM, Robert Haas wrote:
> I haven't scrutinized your code but it seems like the
> minimum-per-database might be complicating things more than necessary.
>   You might find that you can make the logic simpler without that.  I
> might be wrong, though.

I still think of that as a feature, not something I'd like to simplify 
away. Are you arguing it could raise the chance of adaption of bgworkers 
to Postgres? I'm not seeing your point here.

> I guess the real issue here is whether it's possible to, and whether
> you're interested in, extracting a committable subset of this work,
> and if so what that subset should look like.

Well, as it doesn't currently provide any real benefit for autovacuum, 
it depends on how much hackers like to prepare for something like 
Postgres-R or parallel querying.

> There's sort of a
> chicken-and-egg problem with large patches; if you present them as one
> giant monolithic patch, they're too large to review.  But if you break
> them down into smaller patches, it doesn't really fix the problem
> unless the pieces have independent value.

I don't quite get what you are trying to say here. I splited the 
bgworker projet from Postgres-R into 6 separate patches. Are you saying 
that's too few or too much?

You are welcome to argue about independent patches, i.e. this patch 1 of 
6 (as $SUBJECT implies) might have some value, according to Alvaro.

Admittedly, patch 2 of 6 is the biggest and most important chunk of 
functionality of the whole set.

>> Also note that it would re-introduce some of the costs we try to avoid with
>> keeping the connected bgworker around.
>
> How?

I'm talking about the cost of connecting to a database (and 
disconnecting), most probably flushing caches, and very probably some 
kind of re-registering with the coordinator. Most of what a normal 
backend does at startup. About the only thing you'd save here is the 
fork() and very basic process setup. I really doubt that's worth the effort.

Having some more idle processes around doesn't hurt that much and solves 
the problem, I think.

Thanks for your feedback.

Regards

Markus Wanner