Re: Clock-skew management in logical replication

On 9/21/24 01:31, shihao zhong wrote:
> Nisha Moond <nisha.moond412@gmail.com> writes:
>> Thoughts? Looking forward to hearing others' opinions!
> 
> Had a productive conversation with Amit Kaplia today about time skew
> in distributed systems, and wanted to share some thoughts.
> Essentially, we're grappling with the classic distributed snapshot
> problem. In a multi-active environment, where multiple nodes can
> independently process transactions,  it becomes crucial to determine
> the visibility of these transactions across the system.  Time skew,
> where different machines have different timestamps make it a hard
> problem. How can we ensure consistent transaction ordering and
> visibility when time itself is unreliable?
> 
> As you mentioned, there are several ways to tackle the time skew
> problem in distributed systems. These approaches generally fall into
> three main categories:
> 
> 1. Centralized Timestamps (Timestamp Oracle)
> 2. Atomic Clocks (True Time)
> 3. Hybrid Logical Clocks
> 4 Local Clocks

> I recommend ...<snip>... implement a pluggable time access method. This
> allows users to integrate with different time services as needed.

Huge +1

> In the mid-term, implementing the HLC approach would provide highly
> consistent snapshot reads. This offers a significant advantage for
> many use cases.

agreed

> Long-term, we should consider integrating with a distributed time
> service like AWS Time Sync Service. This ensures high accuracy and
> scalability for demanding applications.

I think the pluggable access method should make this possible, no?

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com