Обсуждение: Re: [GENERAL] Archiving data to another server using copy, psql withpipe
Il 05/04/2017 23:26, pinker ha scritto: > Hi, > I'm trying to write an archive manager which will be first copying data from > tables with where clause and then, after successful load into second server > - delete them. > The simplest (and probably fastest) solution I came up with is to use copy: > psql -h localhost postgres -c "copy (SELECT * FROM a WHERE time < now()) to > stdout " | psql -h localhost postgres -c "copy b from stdin" both psql -h are on localhost. Is it a typo? > > I have made very simple test to check if I can be sure about "transactional" > safety. It's not two phase commit of course but it's seems to throw an error > if something went wrong and it's atomic (i assume). The test was: > > CREATE TABLE public.a > ( > id integer, > k01 numeric (3) > ); > > CREATE TABLE public.b > ( > id integer, > k01 numeric (1) > ); > > insert into a select n,n from generate_series(1,100) n; > > and then: > psql -h localhost postgres -c "copy a to stdout "|psql -h localhost > postgres -c "copy b from stdin" > > so psql has thrown an error ... and what is the error? > and no rows were inserted to the b table - so it > seems to be ok. > > Is there maybe something I'm missing? > Some specific condition when something could go wrong and make the process > not atomic? (i don't care about data consistency in this particular case). Without knowing OS and psql version of both servers, how they are connected, or what error you get, it's hard for me to help you further. Best regards Moreno. > > > > > -- > View this message in context: http://www.postgresql-archive.org/Archiving-data-to-another-server-using-copy-psql-with-pipe-tp5954469.html > Sent from the PostgreSQL - general mailing list archive at Nabble.com. > >
Re: [GENERAL] Archiving data to another server using copy, psql with pipe
От
"David G. Johnston"
Дата:
psql -h localhost postgres -c "copy (SELECT * FROM a WHERE time < now()) to
stdout " | psql -h localhost postgres -c "copy b from stdin"
The first question at hand is whether the source psql command will provoke an EOF (which is the only thing that will stop the copy-from) even though the complete contents of the copy-to have not yet been sent.
The second question is whether, even if it does send EOF, the second command will be allowed to see that EOF and complete its command. Pipeline failure mode might impact this (from bash experience).
Unfortunately I do not know the answers to those questions.
The source code might be of some help on the first.
Another option is to replace the first psql process with custom program that will send data to stdout and in the middle of doing so die with a non-zero exit code. That should be a workable simulation of the copy to command and evaluation of the target can be done to see what behavior is exhibitied.
David J.