Re: COPY from STDIN vs file with large CSVs

Поиск
Список
Период
Сортировка
От bricklen
Тема Re: COPY from STDIN vs file with large CSVs
Дата
Msg-id CAGrpgQ-zFD2aBz_+w4V5UWMH1wGOhjefy7bPz4S-VTcr52dusg@mail.gmail.com
обсуждение исходный текст
Ответ на COPY from STDIN vs file with large CSVs  (Wells Oliver <wells.oliver@gmail.com>)
Список pgsql-admin


On Wed, Jan 8, 2020 at 8:55 AM Wells Oliver <wells.oliver@gmail.com> wrote:
I have a CSV that's ~30GB. Some 400m rows. Would there be a meaningful performance difference to run COPY from STDIN using: cat f.csv | psql "COPY .. FROM STDIN WITH CSV" versus just doing "COPY ... FROM 'f.csv' WITH CSV"?

If you're looking to speed up the loading - and your disk subsystem is decent - consider running your csv through the "split" command to decompose it into smaller CSV files. You can then load them in parallel using multiple psql sessions.

В списке pgsql-admin по дате отправления:

Предыдущее
От: "David G. Johnston"
Дата:
Сообщение: Re: COPY from STDIN vs file with large CSVs
Следующее
От: bvo
Дата:
Сообщение: misbehavior slave