They have two web applications A1 and A2. They have seperate hostnames/URLs. Both have a production and a test database A1p and A1t/ A2p and A2t.
What they've done is have both A1p and A2p on the same actual databaser server and A1t and A2t on the same server.
Are these two PostgreSQL instances running on the same hardware, or two databases within a single PostgreSQL instance?
So, I'm thinking - if a bug in application A1 crashes the application and database badly it will risk bringing down both services A1 and A2.
Is this a common occurrence? Of all the occurrences of downtime in recent memory (or better yet, from incidence documentation), what were the causes of them? Is this near the top of the list?
Also, are the two apps completely independent, or are they used together such that one being down makes the other one not very useful?
The same risk would be evident on a successful security breach.
On the other hand, more servers means more moving parts, means more opportunities for mistakes in configuration or maintenance that let breaches happen.
I would prefer to A1p and A2p on seperate servers, maybe keeping A1t and A2t on the same. (This is what seems to be happening when the database servers are being repladed).
I don't know what that last part means.
What is the general thought on the current setup?
In my experience, people acting on mere conjectures about what might cause downtime in the future and how to prevent it have caused more downtime than they have prevented.