> We try hard to release the GIL anytime we're about to call libpq > functions and, looking at the code, I don't see any obvious place where > the GIL is hold during a time-consuming call. > > Can you please send us your code (the shortest possible version that > actually can be run) so that we can investigate? >
Thanks, Federico. I have attached my program.
A few points -
1. Instead of using my home-grown threading/queue model, I have used loop.run_in_executor(), as this is more standard. It also shows the slowdown.
2. I test on two machines – Windows and linux. To save me commenting/uncommenting the connection details, I use ‘try import pyodbc’. If it succeeds, I assume Windows. If it fails, I assume linux and import psycopg2. In both cases I have an alternative connection that uses sqlite3, but then I do have to comment/uncomment to switch databases.
3. The program uses loop.run_forever(). To stop it, just press Enter. I use a separate thread to listen for that and shut everything down. I do that because I could never get KeyboardInterrupt working properly on Windows.
4. I use a home-grown connection pool – hope it looks ok! The first iteration of the test opens 10 connections, so best to ignore the first set of timings.
5. You will have to substitute your own database details and table name to test it.
6. I commented out some lines that show more information – uncomment them if required.
That is all I can think of. If you have any other problems, let me know.