Re: New Copy Formats - avro/orc/parquet

Поиск
Список
Период
Сортировка
От Nicolas Paris
Тема Re: New Copy Formats - avro/orc/parquet
Дата
Msg-id 20180210153011.o2w3tdriqmwvahph@gmail.com
обсуждение исходный текст
Ответ на Re: New Copy Formats - avro/orc/parquet  (Adrian Klaver <adrian.klaver@aklaver.com>)
Ответы Re: New Copy Formats - avro/orc/parquet
Список pgsql-general
> > I d'found useful to be able to import/export from postgres to those modern data
> > formats:
> > - avro (c writer=https://avro.apache.org/docs/1.8.2/api/c/index.html)
> > - parquet (c++ writer=https://github.com/apache/parquet-cpp)
> > - orc (all writers=https://github.com/apache/orc)
> > 
> > Something like :
> > COPY table TO STDOUT ORC;
> > 
> > Would be lovely.
> > 
> > This would greatly enhance how postgres integrates in big-data ecosystem.
> > 
> > Any thought ?
> 
> https://www.postgresql.org/docs/10/static/sql-copy.html
> 
> "PROGRAM
> 
>     A command to execute. In COPY FROM, the input is read from standard
> output of the command, and in COPY TO, the output is written to the standard
> input of the command.
> 
>     Note that the command is invoked by the shell, so if you need to pass
> any arguments to shell command that come from an untrusted source, you must
> be careful to strip or escape any special characters that might have a
> special meaning for the shell. For security reasons, it is best to use a
> fixed command string, or at least avoid passing any user input in it.
> "
>

PROGRAM would involve overhead of transforming data from CSV or BINARY
to AVRO for example. 

Here, I am talking about native format exports/imports for performance
considerations.


В списке pgsql-general по дате отправления:

Предыдущее
От: Adrian Klaver
Дата:
Сообщение: Re: New Copy Formats - avro/orc/parquet
Следующее
От: "David G. Johnston"
Дата:
Сообщение: Re: New Copy Formats - avro/orc/parquet