The line of code I’m executing is
cursor.copy_from(io.StringIO('\n'.join(list_of_strings)), 'tablename')
Every once in a while the command seems to run forever, causing my server to crash.
I'm running this on Python 3 inside a Flask app being served by Waitress on Heroku. The source of the data is a POST request and consists of tab-separated timestamps and numbers. There is nothing different happening on the client side when this error happens. The format of the data is the same with every POST.
This is the error I get from Heroku when I finally manually restart the dyno:
2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] process 23344 acquired ShareLock on transaction 720433 after 59127607.154 ms
Query: COPY tablename FROM stdin WITH DELIMITER AS ' ' NULL AS '\N'
2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] connection to client lost
2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] duration: 59127623.384 ms statement: COPY tablename FROM stdin WITH DELIMITER AS ' ' NULL AS '\N'
2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] could not send data to client: Connection timed out 2015-04-24T12:18:39Z app[postgres.13451]: [BLACK] unexpected EOF on client connection with an open transaction
So the copy command was running for 59 million milliseconds or 16 and a half hours before I finally killed it by restarting the dyno.
Any suggestions on where I should look for the cause? Alternatively, is it possible to simply put a timeout on the copy?
-Daniel