Re: Logical replication and multimaster
| От | Craig Ringer |
|---|---|
| Тема | Re: Logical replication and multimaster |
| Дата | |
| Msg-id | CAMsr+YE+RDz-Lzj4ro7aRXKOPCb1dBisiiRbc2MbukBjMXzemA@mail.gmail.com обсуждение исходный текст |
| Ответ на | Re: Logical replication and multimaster (konstantin knizhnik <k.knizhnik@postgrespro.ru>) |
| Ответы |
Re: Logical replication and multimaster
(Petr Jelinek <petr@2ndquadrant.com>)
|
| Список | pgsql-hackers |
On 3 December 2015 at 15:27, konstantin knizhnik <k.knizhnik@postgrespro.ru> wrote:
On Dec 3, 2015, at 4:18 AM, Craig Ringer wrote:
> Excellent.
>
> It should be possible to make that a separate extension. You can use C functions from other extensions by exposing a single pg_proc function with 'internal' return type that populates a struct of function pointers for the API. A single DirectFunctionCall lets you get the API struct. That's how pglogical_output handles hooks. The main downside is that you can't do that without a connection to a database with the extension installed so the pg_proc entry is exposed.
Actually, working under cluster and columnar storage extension I got several questions about PostgreSQL infrastructure.
I always found some workarounds, but may it is better to ask community about it:)
1. Why there is no "conditional event" synchronization primitive in PostgreSQL. There is latch, but it is implemented using sockets and I afraid that it is not very fast.
It will be nice to have some fast primitive like pthread condition variables.
The need for IPC makes things a bit more complex. Most places can get away with using a latch, testing one or more conditions, and resuming waiting.
While what you describe sounds possibly nice is there any evidence that it's a bottleneck or performance issue? Or is this premature optimisation at work?
2. PostgreSQL semaphores seems to be not intended for external use outside PostgreSQL core (for example in extensions).
There is no way to request additional amount of semaphores. Right now semaphores are allocated based on maximal number of backends and spinlocks.
Same with spinlocks AFAIK.
You can add your own LWLocks though.
3. What is the right way of creation of background worker requiring access to shared memory, i.e. having control structure in main memory?
This is documented and well established.
As far as I understand background workers have to be registered either PG_init, either outside Postmaster environment.
If extension requires access to shared memory, then it should be registered in shared_preload_libraries list and should be initialized using shmem_startup hook.
Correct.
You can use dynamic shmem instead, but there are some issues there IIRC. Petr may have more to say there.
Take a look at the BDR code for some examples, and there are some in contrib too I think.
My_shmem_startup is needed because in _PG_init it is not possible to allocate shared memory.
Correct, since it's in early postmaster start.
So if I need to allocate some control structure for background workers in shared memory, then I should do it in My_shmem_startup.
Yes.
But I can not register background workers in My_shmem_startup!
Correct. Register static bgworkers in _PG_init. Register dynamic bgworkers later, in a normal backend function or a bgworker main loop.
So I have to register background workers in PG_init while control structure for them is not yet ready.
Correct.
They aren't *started* until after shmem init, though.
When I have implemented pool of background workers, I solved this problem by proving function which return address of control structure later - when it will be actually allocated.
Beware of EXEC_BACKEND. You can't assume you have shared postmaster memory from fork().
I suggest that you allocate a static shmem array. Pass indexes into it as the arguments to the bgworkers. Have them look up their index in the array to get their struct pointer.
Read the BDR code to see how this can work; see bdr_perdb.c, bdr_apply.c, etc's bgworker main loops, bdr_supervisor.c and bdr_perdb.c's code for registering dynamic bgworkers, and the _PG_init function's setup of the static supervisor bgworker.
In your case I think you should probably be using dynamic bgworkers for your pool anyway, so you can grow and shrink them as-needed.
But it seems to be some design flaw in BGW, isn' it?
I don't think so. You're registering the worker, saying "when you're ready please start this". You're not starting it.
В списке pgsql-hackers по дате отправления: