On 9/30/23 14:54, pf@pfortin.com wrote:
> On Sat, 30 Sep 2023 13:40:37 -0700 Adrian Klaver wrote:
>
>> On 9/30/23 11:32, pf@pfortin.com wrote:
>
> As I told Tom, the "test" DB has this issue; the production and test1 DBs
> are fine; I should have thought to check those first... Sorry for the
> noise.
Still there was an issue with a database. Did you track down what was
wrong with "test"?
> We were working with a python script (using INSERT) someone wrote a couple
> years ago; with my version using COPY[1], imports have gone from 2+ hours
> to under a minute each for 8M and 33M row files. If the above links help
> even more that will be amazing!! THANK YOU!!
The move to COPY will have accounted for the speed up. Using the
psycopg2 COPY methods probably won't make a material difference, I just
find them easier to use. Also in the new psycopg(3) there are more ways
to use them per:
https://www.psycopg.org/psycopg3/docs/basic/copy.html.
--
Adrian Klaver
adrian.klaver@aklaver.com