On May 26, 2021, at 4:37 PM, Ian Harding <harding.ian@gmail.com> wrote:
There is an option to send the logs to cloudwatch which makes it less awful to look at them.
I have that but precious little of interest there. Lots of autovac, a smattering of hints to increase wal size!? I have yet to spot anything which corresponds to the “I/O failure” which the middle ware gets.
I don’t have query logging on, but I do see reports from my psql session fat-fingering.
As to the logs UI, the search is pretty feeble; I don’t understand why there are four channels of logs; the graphs are wearing the same rose-coloured as the logs.
And 24 hours without a peep from AWS support. (I don’t call mailing me what I sent them “contact”.)
My guess right now is that the entire tomcat connection pool is in a single transaction? That’s the only way the tables could disappear. I am making separate calls to JDBC getConnection () for each doPost.
We used Aurora (AWS hosted Postgres) and I agree that Cloudwatch search is pretty limited. I wrote a Python script to download cloudwatch logs to my laptop where I can use proper tools like grep to search them. It’s attached to this email. It’s hacky but not too terrible. I hope you find it useful.
Cheers
Philip
I may have found another difference: JDBC connections are not logged?! I just reproduce my report, and the CloudWatch view of the logs shows some psql interaction from before and after the test, but no mention of losing 7.5M records.