Re: How is this possible "publication does not exist"

On Thu, Dec 19, 2019 at 07:19:56PM +0100, Peter Eisentraut wrote:
>On 2019-12-19 19:15, Dave Cramer wrote:
>>It seems that if you drop the publication on an existing slot it 
>>needs to be recreated. Is this expected behaviour
>
>A publication is not associated with a slot.  Only a subscription is 
>associated with a slot.
>
>>drop publication dbz_publication ;
>>DROP PUBLICATION
>>postgres=# create publication dbz_publication for all tables;
>>CREATE PUBLICATION
>>postgres=# SELECT * FROM 
>>pg_logical_slot_get_binary_changes('debezium', NULL, 
>>NULL,'proto_version','1','publication_names','dbz_publication');
>>ERROR:  publication "dbz_publication" does not exist
>>CONTEXT:  slot "debezium", output plugin "pgoutput", in the change 
>>callback, associated LSN 0/4324180
>
>This must be something particular to Debezium.
>

Yeah, I don't see this error message anywhere in our sources on 11 or
12, so perhaps debezium does something funny? It's not clear to me
where, though, as this suggests it uses the pgoutput module.

While trying to reproduce this I however ran into a related issue with
pgoutput/pg_logical_slot_get_binary_changes. If you call the function
repeatedly (~10x) you'll get an error like this:

FATAL:  out of relcache_callback_list slots
CONTEXT:  slot "slot", output plugin "pgoutput", in the startup callback
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

The reason is very simple - each call executes pgoutput_startup, which
does CacheRegisterRelcacheCallback in init_rel_sync_cache. And we do
this on each pg_logical_slot_get_binary_changes() call and never remove
the callbacks, so we simply run out of MAX_RELCACHE_CALLBACKS slots.

Not sure if this is a known issue/behavior, but it seems a bit annoying
and possibly related to the issue reported by Dave.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services