[RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility

Hallo postgresql and replication hackers

This mail is an additional RFC which proposes a simple way to extend the
new logical replication feature so it can cover most usages of 
skytools/pgq/londiste

While the current work for BDR/LCR (bi-directional replication/logical 
replication)
using WAL is theoretically enought to cover _replication_ offered by 
Londiste it falls short in one important way - there is currently no support for 
pure queueing,
that is for "streams" of data which does not need to be stored in the 
source database.

Fortunately there is a simple solution - do not store it in the source 
database :)

The only thing needed for adding this is to have a table type which

a) generates a INSERT record in WAL

and

b) does not actually store the data in a local file

If implemented in userspace it would be a VIEW (or table) with a 
before/instead
trigger which logs the inserted data and then cancels the insert.

I'm sure this thing could be implemented, but I leave the tech 
discussion to those
who are currently deep in WAL generation/reconstruction .

If we implement logged only tables / queues we would not only enable a more
performant pgQ replacement for implementing full Londiste / skytools 
functionality
but would also become a very strong player to be used as persistent 
basis for
message queueing solutions like ActiveMQ, StorMQ, any Advanced Message
Queuing Protocol (AMQP) and so on.

comments ?


Hannu Krosing