Re: logical changeset generation v4

On 13-01-14 08:38 PM, Andres Freund wrote:
> Hi everyone,
>
> Here is the newest version of logical changeset generation.
>
>
>
> 2) Currently the logical replication infrastructure assigns a 'slot-id'
> when a new replica is setup. That slot id isn't really nice
> (e.g. "id-321578-3"). It also requires that [18] keeps state in a global
> variable to make writing regression tests easy.
>
> I think it would be better to make the user specify those replication
> slot ids, but I am not really sure about it.

Shortly after trying out the latest version I hit the following scenario
1. I started pg_receivellog but mistyped the name of my plugin
2. It looped and used up all of my logical replication slots

I killed pg_receivellog and restarted it with the correct plugin name 
but it won't do anything because I have no free slots.  I can't free the 
slots with -F because I have no clue what the names of the slots are.    
I can figure the names out by looking in pg_llog but if my replication 
program can't do that so it won't be able to clean up from a failed attempt.

I agree with you that we should make the user program specify a slot, we 
eventually might want to provide a view that shows the currently 
allocated slots. For a logical based slony I would just generate the 
slot name based on the remote node id.  If walsender generates the slot 
name then I would need to store a mapping between slot names and slons 
so when a slon restarted it would know which slot to resume using.   I'd 
have to use a table in the slony schema on the remote database for 
this.  There would always be a risk of losing track of a slot id if the 
slon crashed after getting the slot number but before committing the 
mapping on the remote database.




> 3) Currently no options can be passed to an output plugin. I am thinking
> about making "INIT_LOGICAL_REPLICATION 'plugin'" accept the now widely
> used ('option' ['value'], ...) syntax and pass that to the output
> plugin's initialization function.

I think we discussed this last CF, I like this idea.

> 4) Does anybody object to:
> -- allocate a permanent replication slot
> INIT_LOGICAL_REPLICATION 'plugin' 'slotname' (options);
>
> -- stream data
> START_LOGICAL_REPLICATION 'slotname' 'recptr';
>
> -- deallocate a permanent replication slot
> FREE_LOGICAL_REPLICATION 'slotname';

+1


> 5) Currently its only allowed to access catalog tables, its fairly
> trivial to extend this to additional tables if you can accept some
> (noticeable but not too big) overhead for modifications on those tables.
>
> I was thinking of making that an option for tables, that would be useful
> for replication solutions configuration tables.

I think this will make the life of anyone developing a new replication 
system easier.  Slony has a lot of infrastructure for allowing slonik 
scripts to wait for configuration changes to popogate everywhere before 
making other configuration changes because you can get race conditions.  
If I were designing a new replication system and I had this feature then 
I would try to use it to come up with a simpler model of propagating 
configuration changes.

>
> Andres Freund
>
>

Steve