Re: [SPAM] COPY command & binary format

Поиск
Список
Период
Сортировка
От Nicolas Paris
Тема Re: [SPAM] COPY command & binary format
Дата
Msg-id CA+ssMOQ84ZF0NuXTd4WGooKa-2uUcixKC2nnZrcuhXyNTNL59g@mail.gmail.com
обсуждение исходный текст
Ответ на Re: [SPAM] COPY command & binary format  (Moreno Andreo <moreno.andreo@evolu-s.it>)
Ответы Re: [SPAM] COPY command & binary format  (Cat <cat@zip.com.au>)
Список pgsql-general
2016-05-10 14:47 GMT+02:00 Moreno Andreo <moreno.andreo@evolu-s.it>:
Il 10/05/2016 13:38, Nicolas Paris ha scritto:
2016-05-10 13:04 GMT+02:00 Moreno Andreo <moreno.andreo@evolu-s.it>:
Il 10/05/2016 12:56, Nicolas Paris ha scritto:
Hello,

What is the way to build a binary format (instead of a csv) ? Is there specification for this file ?
http://www.postgresql.org/docs/9.5/static/sql-copy.html
I always create binary files with
COPY table TO 'path/to/file' WITH BINARY

 
​ Fine, this works in this way :
postgresql -> binary
binary -> postgresql

The way I want is :
csv -> binary -> postgresql
Is this just to be quicker or are you going to add some business logic while converting CSV data?
As you mentioned ETL, I assume the second, as I don't think that converting CSV to binary and then loading it to PostgreSQL will be more convenient than loading directly from CSV... as quicker as it can be, you have anyway to load data from CSV.

Right, ETL process means huge business logic.
get the data (csv or other) -> transform it -> produce a binary -> copy from binary from stdin ​

Producing 100GO CSVs, is a waste of time.

 
Binary file format is briefly described in the last part of the doc you linked, under "Binary format", and there's also reference to source files.

And if possible, transforming csv to binary throught java​.
This is beyond my knowledge, ATM. I'm just starting with Java and JDBC is still in the TODO list, sorry... :-)

Cheers
Moreno.-

​Documentation explains a bit. Moreover, I have found a detailled answer here :
My ultimate goal is to encapsulate it in a Talend component. (talend is an open-source java based ETL software).

Thanks, I ll keep you aware.

В списке pgsql-general по дате отправления:

Предыдущее
От: Moreno Andreo
Дата:
Сообщение: Re: [SPAM] COPY command & binary format
Следующее
От: Cat
Дата:
Сообщение: Re: [SPAM] COPY command & binary format