On 04/24/2015 07:06 AM, Daniel.Straight@Intralox.com wrote:
> The line of code I’m executing is
>
> cursor.copy_from(io.StringIO('\n'.join(list_of_strings)), 'tablename')
>
> Every once in a while the command seems to run forever, causing my
> server to crash.
So how long does it 'usually' run?
>
> I'm running this on Python 3 inside a Flask app being served by Waitress
> on Heroku. The source of the data is a POST request and consists of
> tab-separated timestamps and numbers. There is nothing different
> happening on the client side when this error happens. The format of the
> data is the same with every POST.
Define client.
Are you talking about the Web client making the POST or the Flask app as
client to Postgres?
>
> This is the error I get from Heroku when I finally manually restart the
> dyno:
>
> 2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] process 23344 acquired
> ShareLock on transaction 720433 after 59127607.154 ms
>
> Query: COPY tablename FROM stdin WITH DELIMITER AS ' ' NULL AS '\N'
>
> 2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] connection to client lost
>
> 2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] duration: 59127623.384
> ms statement: COPY tablename FROM stdin WITH DELIMITER AS ' ' NULL AS
> '\N'
>
> 2015-04-24T12:18:39Z app[postgres.23344]: [BLACK] could not send data to
> client: Connection timed out 2015-04-24T12:18:39Z app[postgres.13451]:
> [BLACK] unexpected EOF on client connection with an open transaction
This would Postgres complaining about losing connection to presumably
the code you show in the Flask app. Though it would be nice to know
where exactly the copy_from is in the Flask flow of things.
>
> So the copy command was running for 59 million milliseconds or 16 and a
> half hours before I finally killed it by restarting the dyno.
>
> Any suggestions on where I should look for the cause? Alternatively, is
> it possible to simply put a timeout on the copy?
I would say a more complete code snippet would help. Something
encompassing how the POST data gets into the list_of_strings and then
into the copy_from()
>
> -Daniel
>
--
Adrian Klaver
adrian.klaver@aklaver.com