On Thu, Apr 18, 2019 at 8:38 PM Michael Paquier <michael@paquier.xyz> wrote:
> On Thu, Apr 18, 2019 at 03:51:14PM -0400, Robert Haas wrote:
> > I was thinking that a dedicated background worker would be a good
> > option, but Stephen Frost seems concerned (over on the other thread)
> > about how much load that would generate. That never really occurred
> > to me as a serious issue and I suspect for many people it wouldn't be,
> > but there might be some.
>
> WAL segment size can go up to 1GB, and this does not depend on the
> compilation anymore. So scanning a very large segment is not going to
> be free.
The segment size doesn't have much to do with it. If you make
segments bigger, you'll have to scan fewer larger ones; if you make
them smaller, you'll have more smaller ones. The only thing that
really matters is the amount of I/O and CPU required, and that doesn't
change very much as you vary the segment size.
> I think that the performance concerns of Stephen are legit
> as now we have on the WAL partitions sequential read and write
> patterns.
As to that, what I'm proposing here is no different than what we are
already doing with physical and logical replication, except that it's
probably a bit cheaper. Physical replication reads all the WAL and
sends it all out over the network. Logical replication reads all the
WAL, does a bunch of computation, and then sends the results, possibly
filtered, out over the network. This would read the WAL and then
write a relatively small file to your local disk.
I think the impact will be about the same as having one additional
standby, give or take.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company