Re: BDR: name conflict when joining a rebuilt node

Still having issues with this with BDR-0.9.3

This is how I join a new node to the cluster:

############################################
su - postgres
psql pgmirror

-- fire up BDR extensions
CREATE EXTENSION btree_gist;
CREATE EXTENSION bdr;

-- join BDR group via an existing node there
SELECT bdr.bdr_group_join(
    local_node_name := 'pg12-prod-uswest2-aws',
    node_external_dsn := 'host=pg12-prod-uswest2-aws dbname=pgmirror',
    join_using_dsn := 'host=pg11-prod-uswest2-aws dbname=pgmirror'
);

SELECT bdr.bdr_node_join_wait_for_ready();
############################################

This is how I remove a node from the cluster:

############################################
# Log into any other node in the cluster (NOT the node you want to
remove) and run:

su - postgres
psql pgmirror

SELECT bdr.bdr_part_by_node_names('{pg12-prod-uswest2-aws}');

# Log into the removed node and run:

su - postgres
psql pgmirror

BEGIN;
SET LOCAL bdr.permit_unsafe_ddl_commands = true;
SET LOCAL bdr.skip_ddl_locking = true;
SECURITY LABEL FOR 'bdr' ON DATABASE pgmirror IS '{"bdr": false}';
COMMIT;

# Now restart PostgreSQL.
############################################

Now let's say on the removed node I've dropped the pgmirror database,
performed some maintenance, re-created the pgmirror DB (empty), and now
I want to re-join the node to the cluster under the same name. I repeat
the join new node procedure described at the top. It gets stuck in
node_join_wait_for_ready().

On another node, the re-joined node is now listed twice in
bdr.bdr_nodes, once with status k, and again with status i. The logs on
the re-joined node show this:

############################################
2015-11-03 20:29:52.016 UTC,,,4916,,56391614.1334,219,,2015-11-03
20:16:20 UTC,,0,LOG,00000,"starting background worker process ""bdr db:
pgmirror""",,,,,,,,,""
2015-11-03 20:29:52.047 UTC,,,7222,"",56391940.1c36,1,"",2015-11-03
20:29:52 UTC,,0,LOG,00000,"connection received: host=127.0.0.1
port=21241",,,,,,,,,""
2015-11-03 20:29:52.050
UTC,"postgres","pgmirror",7222,"127.0.0.1:21241",56391940.1c36,2,"authentication",2015-11-03 20:29:52
UTC,4/321,0,LOG,00000,"replicationconnection authorized: user=postgres SSL enabled (protocol=TLSv1.2,
cipher=ECDHE-RSA-AES256-GCM-SHA384,compression=off)",,,,,,,,,"" 
2015-11-03 20:29:52.052 UTC,,,7221,,56391940.1c35,1,,2015-11-03 20:29:52
UTC,3/0,0,ERROR,55000,"System identification mismatch between connection
and slot","Connection for bdr (6212727469166484615,1,16387,) resulted in
slot on node bdr (6212727469166484615,1,17169,) instead of expected
node",,,,,,,,"bdr (6212727469166484615,1,17169,): perdb"
2015-11-03 20:29:52.053
UTC,"postgres","pgmirror",7222,"127.0.0.1:21241",56391940.1c36,3,"idle",2015-11-03 20:29:52 UTC,4/0,0,LOG,08006,"could
notreceive data from client: Connection reset by peer",,,,,,,,,"bdr (6212727469166484615,1,17169,):mkslot" 
2015-11-03 20:29:52.053
UTC,"postgres","pgmirror",7222,"127.0.0.1:21241",56391940.1c36,4,"idle",2015-11-03 20:29:52
UTC,,0,LOG,00000,"disconnection:session time: 0:00:00.006 user=postgres database=pgmirror host=127.0.0.1
port=21241",,,,,,,,,"bdr(6212727469166484615,1,17169,):mkslot" 
2015-11-03 20:29:52.053 UTC,,,4916,,56391614.1334,220,,2015-11-03
20:16:20 UTC,,0,LOG,00000,"worker process: bdr db: pgmirror (PID 7221)
exited with exit code 1",,,,,,,,,""
############################################

OS is Ubuntu 14.04, with these packages installed:

############################################
ii  postgresql-bdr-9.4               9.4.4-1trusty
amd64        object-relational SQL database, version 9.4 server
ii  postgresql-bdr-9.4-bdr-plugin    0.9.3-1trusty
amd64        BDR Plugin for PostgreSQL-BDR 9.4
ii  postgresql-bdr-client-9.4        9.4.4-1trusty
amd64        front-end programs for PostgreSQL-BDR 9.4
ii  postgresql-bdr-contrib-9.4       9.4.4-1trusty
amd64        additional facilities for PostgreSQL
ii  postgresql-bdr-server-dev-9.4    9.4.4-1trusty
amd64        development files for PostgreSQL-BDR 9.4 server-side
programming
ii  postgresql-client-common         169.pgdg14.04+1
all          manager for multiple PostgreSQL client versions
ii  postgresql-common                169.pgdg14.04+1
all          PostgreSQL database-cluster manager
############################################

--
Florin Andrei
http://florin.myip.org/