Hi all!
I'm a DevOps Manager/Engineer by trade (though the place I work is not, unfortunately, using Postgres). I've been thinking quite a bit about what our ideal architecture at work will look like and what scaling looks like, both for work and for home projects (where I *am* looking at using Postgres :) ).
Tonight, (unrelated to work) I took the time to draw up a diagram of an architecture that I think would help Postgres move one step towards both more scalability, and better ease of use.
Since I'm not so hot at drawing ASCII art diagrams, I thought maybe the way to go would be to drop it into a Google Presentation and make that public.
It's a couple of diagrams (existing and proposed architecture), and a list of what I think the advantages and disadvantages are.
To keep it short, the proposal is that the stages from Parse through Plan be done in a separate process (and potentially on a separate server) from the Execute stage. The idea is:
- The Parse/Plan servers don't care whether they're read or write
- The Parse/Plan know which Execute server is the writer (and which the readers), and forward to the correct server for execution
I even wonder if this might not mean that the Parse/Plan servers can be deployed as K8s containers, with the Execute server being the external non-k8s server.
Note that in this e-mail, I've referred to:
- The Parse/Plan server (which my diagram calls the Postgres SQL server)
- The Execute server (which my diagram calls the Storage server)
I'm not sure what naming makes sense, but I intentionally used a couple of different names in hopes that one of them would get the idea across -- please disregard whichever names don't make sense, and feel free to suggest new ones.
I'm expecting that people will pick the idea apart, and wanted to know what people think of it.
Thanks!