Обсуждение: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
Hi
I keep getting the following error when trying to restore a database on Postgres 9.2.4.
I used pg_dump –Fc with no owner from one server (8.4.2). It’s working fine for other similar databases.
And then restoring the database on another server running on 9.2.4 using pg_restore –Fc. I get the get following error but I don’t get any errors when I dump the Database.
pg_restore: custom archiver unexpected end of file
ERROR: missing data for column "fpp_cde"
CONTEXT: COPY audit, line 2565159: "4585427 21~21~21 5143 0 2013-12-01 15:03:18.336718 <?xml version="1.0" encoding="UTF-8"?>\r\n<nonF..."
STATEMENT: COPY audit (aud_id, grp_cde, act_typ, aud_ver, usr_id, aud_device_id, aud_ts, aud_xml, aud_replication, fpp_cde, aud_doc_no, aud_doc_ts, br_cde, obo_br_cde, obo_aud_id, orig_trf_aud_id, orig_trf_br_cde, aud_build_number) FROM stdin;
but there is data for a column called fpp_cde
SELECT fpp_cde from audit where aud_id =4585427;
fpp_cde
---------
201311
(1 row)
Please help ,
Kind Regards,
Khangelani
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
Hi there,Hi
I keep getting the following error when trying to restore a database on Postgres 9.2.4.
I used pg_dump –Fc with no owner from one server (8.4.2). It’s working fine for other similar databases.
And then restoring the database on another server running on 9.2.4 using pg_restore –Fc. I get the get following error but I don’t get any errors when I dump the Database.
pg_restore: custom archiver unexpected end of file
ERROR: missing data for column "fpp_cde"
CONTEXT: COPY audit, line 2565159: "4585427 21~21~21 5143 0 2013-12-01 15:03:18.336718 <?xml version="1.0" encoding="UTF-8"?>\r\n<nonF..."
STATEMENT: COPY audit (aud_id, grp_cde, act_typ, aud_ver, usr_id, aud_device_id, aud_ts, aud_xml, aud_replication, fpp_cde, aud_doc_no, aud_doc_ts, br_cde, obo_br_cde, obo_aud_id, orig_trf_aud_id, orig_trf_br_cde, aud_build_number) FROM stdin;
but there is data for a column called fpp_cde
SELECT fpp_cde from audit where aud_id =4585427;
fpp_cde
---------
201311
(1 row)
Please help ,
Kind Regards,
Khangelani
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Did you do the pg_dump using 9.2 binaries or the 8.4 binaries? It's recommended that you always use the newer binaries for dumping. Also, have tried to do a pg_upgrade instead of the dump/restore?
Ziggy
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Ziggy Skalski
Sent: Tuesday, June 17, 2014 5:43 PM
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 2014-06-17, 5:33 AM, Khangelani Gama wrote:
Hi
I keep getting the following error when trying to restore a database on Postgres 9.2.4.
I used pg_dump –Fc with no owner from one server (8.4.2). It’s working fine for other similar databases.
And then restoring the database on another server running on 9.2.4 using pg_restore –Fc. I get the get following error but I don’t get any errors when I dump the Database.
pg_restore: custom archiver unexpected end of file
ERROR: missing data for column "fpp_cde"
CONTEXT: COPY audit, line 2565159: "4585427 21~21~21 5143 0 2013-12-01 15:03:18.336718 <?xml version="1.0" encoding="UTF-8"?>\r\n<nonF..."
STATEMENT: COPY audit (aud_id, grp_cde, act_typ, aud_ver, usr_id, aud_device_id, aud_ts, aud_xml, aud_replication, fpp_cde, aud_doc_no, aud_doc_ts, br_cde, obo_br_cde, obo_aud_id, orig_trf_aud_id, orig_trf_br_cde, aud_build_number) FROM stdin;
but there is data for a column called fpp_cde
SELECT fpp_cde from audit where aud_id =4585427;
fpp_cde
---------
201311
(1 row)
Please help ,
Kind Regards,
Khangelani
Hi there,
Did you do the pg_dump using 9.2 binaries or the 8.4 binaries? It's recommended that you always use the newer binaries for dumping. Also, have tried to do a pg_upgrade instead of the dump/restore?
No I was using 8.4 binaries…Thanks, I will try that now. we will be upgrading the 8.4 server after converting the service into another operating System .
Ziggy
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
I keep getting the following error when trying to restore a database on Postgres 9.2.4.
I used pg_dump –Fc with no owner from one server (8.4.2). It’s working fine for other similar databases.
Hi there,
Did you do the pg_dump using 9.2 binaries or the 8.4 binaries? It's recommended that you always use the newer binaries for dumping. Also, have tried to do a pg_upgrade instead of the dump/restore?
No I was using 8.4 binaries…Thanks, I will try that now. we will be upgrading the 8.4 server after converting the service into another operating System .
Sorry I am not knowledgeable about this , I copied the binary file onto the server where postgres 8.4 is running, untarred it (file is postgresql-9.2.4.tar.gz). But I can’t find the pg_dump script path. It looks like I have to configure it and do the gmake n gmake install. But how do I do that without affecting the current running postgres version 8.4.
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
Hi,
I keep getting the following error when trying to restore a database on Postgres 9.2.4.
I used pg_dump –Fc with no owner from one server (8.4.2). It’s working fine for other similar databases.
Hi there,
Did you do the pg_dump using 9.2 binaries or the 8.4 binaries? It's recommended that you always use the newer binaries for dumping. Also, have tried to do a pg_upgrade instead of the dump/restore?
No I was using 8.4 binaries…Thanks, I will try that now. we will be upgrading the 8.4 server after converting the service into another operating System .
Sorry I am not knowledgeable about this , I copied the binary file onto the server where postgres 8.4 is running, untarred it (file is postgresql-9.2.4.tar.gz). But I can’t find the pg_dump script path. It looks like I have to configure it and do the gmake n gmake install. But how do I do that without affecting the current running postgres version 8.4.
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
This part of the documentation describes how to apply appropriate switches to your configure to place the 9.2 install into alternate location:
http://www.postgresql.org/docs/9.2/static/install-procedure.html
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Ziggy Skalski
Sent: Tuesday, June 17, 2014 8:17 PM
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 2014-06-17, 8:08 AM, Khangelani Gama wrote:
I keep getting the following error when trying to restore a database on Postgres 9.2.4.
I used pg_dump –Fc with no owner from one server (8.4.2). It’s working fine for other similar databases.
Hi there,
Did you do the pg_dump using 9.2 binaries or the 8.4 binaries? It's recommended that you always use the newer binaries for dumping. Also, have tried to do a pg_upgrade instead of the dump/restore?
No I was using 8.4 binaries…Thanks, I will try that now. we will be upgrading the 8.4 server after converting the service into another operating System .
Sorry I am not knowledgeable about this , I copied the binary file onto the server where postgres 8.4 is running, untarred it (file is postgresql-9.2.4.tar.gz). But I can’t find the pg_dump script path. It looks like I have to configure it and do the gmake n gmake install. But how do I do that without affecting the current running postgres version 8.4.
Hi,
This part of the documentation describes how to apply appropriate switches to your configure to place the 9.2 install into alternate location:
http://www.postgresql.org/docs/9.2/static/install-procedure.html
- Thanks very much Ziggy
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 17 June 2014 14:53, Khangelani Gama <kgama@argility.com> wrote: > This part of the documentation describes how to apply appropriate switches > to your configure to place the 9.2 install into alternate location: > http://www.postgresql.org/docs/9.2/static/install-procedure.html > > - Thanks very much Ziggy Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. -- If you can't see the forest for the trees, Cut the trees and you'll see there is no forest.
Alban Hertroys <haramrae@gmail.com> writes: > Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. FWIW, the described behavior sounded like the dump file had gotten truncated somewhere along the line. Maybe it's just necessary to re-do the transfer of the existing dump file. regards, tom lane
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 2014-06-17 10:25:45 -0400, Tom Lane wrote: > Alban Hertroys <haramrae@gmail.com> writes: > > Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. > > FWIW, the described behavior sounded like the dump file had gotten > truncated somewhere along the line. Maybe it's just necessary to > re-do the transfer of the existing dump file. Any chance there was windows in the mix somewhere? IIRC there were bugs with large dumps there for a while... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
-----Original Message----- From: Andres Freund [mailto:andres@2ndquadrant.com] Sent: Tuesday, June 17, 2014 4:38 PM To: Tom Lane Cc: Alban Hertroys; Khangelani Gama; Ziggy Skalski; pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column On 2014-06-17 10:25:45 -0400, Tom Lane wrote: > Alban Hertroys <haramrae@gmail.com> writes: > > Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. > > FWIW, the described behavior sounded like the dump file had gotten > truncated somewhere along the line. Maybe it's just necessary to > re-do the transfer of the existing dump file. Hi Tom, I tried the dump several times, and dumped it more than once but no luck. Any chance there was windows in the mix somewhere? IIRC there were bugs with large dumps there for a while... Hi, Andres servers are running on Linux Thanks Alban Hertroys, I managed to install postgres 9.2.4 binaries onto the postgres 8 server, on a different path, and have dumped the database using postgres 9 bin with no errors.....Now busy copying the files over the network because these servers are far from each other. I will advise on the pg_restore results, hopefully it will work. -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 06/17/2014 10:37 AM, Khangelani Gama wrote: > -----Original Message----- > From: Andres Freund [mailto:andres@2ndquadrant.com] > Sent: Tuesday, June 17, 2014 4:38 PM > To: Tom Lane > Cc: Alban Hertroys; Khangelani Gama; Ziggy Skalski; > pgsql-general@postgresql.org > Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file > , ERROR: missing data for column > > On 2014-06-17 10:25:45 -0400, Tom Lane wrote: >> Alban Hertroys <haramrae@gmail.com> writes: >>> Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. >> >> FWIW, the described behavior sounded like the dump file had gotten >> truncated somewhere along the line. Maybe it's just necessary to >> re-do the transfer of the existing dump file. > > Hi Tom, I tried the dump several times, and dumped it more than once but > no luck. > My suspicion is the XML field two places before the fpp_cde field. Interested in seeing if the 9.4 dump will handle it better. -- Adrian Klaver adrian.klaver@aklaver.com
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
-----Original Message----- From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] Sent: Wednesday, June 18, 2014 4:32 AM To: Khangelani Gama; Andres Freund; Tom Lane Cc: Alban Hertroys; Ziggy Skalski; pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column On 06/17/2014 10:37 AM, Khangelani Gama wrote: > -----Original Message----- > From: Andres Freund [mailto:andres@2ndquadrant.com] > Sent: Tuesday, June 17, 2014 4:38 PM > To: Tom Lane > Cc: Alban Hertroys; Khangelani Gama; Ziggy Skalski; > pgsql-general@postgresql.org > Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of > file , ERROR: missing data for column > > On 2014-06-17 10:25:45 -0400, Tom Lane wrote: >> Alban Hertroys <haramrae@gmail.com> writes: >>> Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. >> >> FWIW, the described behavior sounded like the dump file had gotten >> truncated somewhere along the line. Maybe it's just necessary to >> re-do the transfer of the existing dump file. > > Hi Tom, I tried the dump several times, and dumped it more than once > but no luck. > My suspicion is the XML field two places before the fpp_cde field. Interested in seeing if the 9.4 dump will handle it better. Adrian you might be right I have just tried to pg_restore on the same server(8.4.2), I get the same errors related to xml. Can someone advise if the following XML file as an example is in the correct state or not: <?xml version="1.0" encoding="UTF-8"?>\r + <nonFinancialEventVer01 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="accountEjb.xsd"><actH01><uiD01><userId /><deviceId /></uiD01><transData01><actionType>5 143</actionType><isFinancial>false</isFinancial><docDate>1385903062657</docDate><docNo>4585427</docNo><buildNumber>2013-11-08 :11:14:16</buildNumber><isCredit>false</isCredit><isCashType>false</isC ashType></transData01><seqD01><audit>4585427</audit><finProcPer>201311</finProcPer><branchCode>0023</branchCode><grpCode>21~21~21</grpCode><hpDoc>1807125</hpDoc></seqD01></actH01><accountData01><br anch>0023</branch><account>103558</account><contract>1</contract><valueData01><value>0.00</value></valueData01><statusData01><finProc>201311</finProc><status>INACTIVE</status><balance>0.00</balance ><arrears>0.00</arrears><due>0.00</due><fupStat >/><inst>101.21</inst><mthNotPay>0</mthNotPay><provArrMth>0</provArrMth><provInstPay>0</provInstPay><provAmt>0.00</provAmt><provPcnt>0.0</provPcnt><de bMtd>0.00</debMtd><crdMtd>0.00</crdMtd><payMtd>0.00</payMtd><intMtd>0.00</intMtd><rebFin>0.00</rebFin><rebIns>0.00</rebIns><finProv>0.00</finProv><insProv>0.00</insProv><provNPTyp /><depMtd>0.00</d epMtd><openArrears>0.00</openArrears><rebClub>0.00</rebClub><nxtMthOpnRecency>0</nxtMthOpnRecency><finRebMtd>0.00</finRebMtd><insRebMtd>0.00</insRebMtd><empericaScore>0</empericaScore><avgDelq>0</a vgDelq><worstAge>8224</worstAge><monthsDelq>0</monthsDelq><openNowDue>0.0</openNowDue><queuePriority /><triadQueue>bdc03</triadQueue><hpfstatNcaIntMtd>0.00</hpfstatNcaIntMtd><hpfstatNcaIntLtd>0.00< /hpfstatNcaIntLtd><hpfstatAddIntMtd>0.00</hpfstatAddIntMtd><hpfstatAddIntLtd>0.00</hpfstatAddIntLtd><hpfstatCrLifeInsMtd>0.00</hpfstatCrLifeInsMtd><hpfstatCrLifeInsLtd>0.00</hpfstatCrLifeInsLtd><hp fstatAddCrLifeInsMtd>0.00</hpfstatAddCrLifeInsMtd><hpfstatAddCrLifeInsLtd>0.00</hpfstatAddCrLifeInsLtd><hpfstatClubMtd>0.00</hpfstatClubMtd><hpfstatClubLtd>0.00</hpfstatClubLtd><hpfstatServiceFeeMt d>0.00</hpfstatServiceFeeMtd><hpfstatServiceFeeLtd>0.00</hpfstatServiceFeeLtd><hpfstatOptInsMtd>0.00</hpfstatOptInsMtd><hpfstatOptInsLtd>0.00</hpfstatOptInsLtd><hpfstatWoff>0.00</hpfstatWoff><hpfst atdebltd>1437.06</hpfstatdebltd><hpfstatcrdltd>-25.01</hpfstatcrdltd><hpfstatpayltd>-1412.05</hpfstatpayltd><hpfstatintltd>0.00</hpfstatintltd><hpfstatdepltd>400.00</hpfstatdepltd><contDelq>0</cont Delq><hpfstatLastPayDte>948405600000</hpfstatLastPayDte><hpfstatIndupBal>0.00</hpfstatIndupBal><hpfstatIndupLtd>0.00</hpfstatIndupLtd></statusData01></accountData01><docValueData01><branch>0023</br anch><account>103558</account><value>0.00</value></docValueData01></nonFinancialEventVer01>\r+ -- Adrian Klaver adrian.klaver@aklaver.com CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
-----Original Message----- From: Khangelani Gama [mailto:kgama@argility.com] Sent: Wednesday, June 18, 2014 1:06 PM To: 'Adrian Klaver'; 'Andres Freund'; 'Tom Lane' Cc: 'Alban Hertroys'; 'Ziggy Skalski'; 'pgsql-general@postgresql.org' Subject: RE: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column -----Original Message----- From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] Sent: Wednesday, June 18, 2014 4:32 AM To: Khangelani Gama; Andres Freund; Tom Lane Cc: Alban Hertroys; Ziggy Skalski; pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column On 06/17/2014 10:37 AM, Khangelani Gama wrote: > -----Original Message----- > From: Andres Freund [mailto:andres@2ndquadrant.com] > Sent: Tuesday, June 17, 2014 4:38 PM > To: Tom Lane > Cc: Alban Hertroys; Khangelani Gama; Ziggy Skalski; > pgsql-general@postgresql.org > Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of > file , ERROR: missing data for column > > On 2014-06-17 10:25:45 -0400, Tom Lane wrote: >> Alban Hertroys <haramrae@gmail.com> writes: >>> Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. >> >> FWIW, the described behavior sounded like the dump file had gotten >> truncated somewhere along the line. Maybe it's just necessary to >> re-do the transfer of the existing dump file. > > Hi Tom, I tried the dump several times, and dumped it more than once > but no luck. > My suspicion is the XML field two places before the fpp_cde field. Interested in seeing if the 9.4 dump will handle it better. Adrian you might be right I have just tried to pg_restore on the same server(8.4.2), I get the same errors related to xml. Can someone advise if the following XML file as an example is in the correct state or not: <?xml version="1.0" encoding="UTF-8"?>\r + <nonFinancialEventVer01 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="accountEjb.xsd"><actH01><uiD01><userId /><deviceId /></uiD01><transData01><actionType>5 143</actionType><isFinancial>false</isFinancial><docDate>1385903062657</docDate><docNo>4585427</docNo><buildNumber>2013-11-08 :11:14:16</buildNumber><isCredit>false</isCredit><isCashType>false</isC ashType></transData01><seqD01><audit>4585427</audit><finProcPer>201311</ ashType>finProcPer><branchCode>0023</branchCode><grpCode>21~21~21</grpCo ashType>de><hpDoc>1807125</hpDoc></seqD01></actH01><accountData01><br anch>0023</branch><account>103558</account><contract>1</contract><valueD anch>ata01><value>0.00</value></valueData01><statusData01><finProc>20131 anch>1</finProc><status>INACTIVE</status><balance>0.00</balance ><arrears>0.00</arrears><due>0.00</due><fupStat >/><inst>101.21</inst><mthNotPay>0</mthNotPay><provArrMth>0</provArrMth> ><provInstPay>0</provInstPay><provAmt>0.00</provAmt><provPcnt>0.0</provP >cnt><de bMtd>0.00</debMtd><crdMtd>0.00</crdMtd><payMtd>0.00</payMtd><intMtd>0.00 bMtd></intMtd><rebFin>0.00</rebFin><rebIns>0.00</rebIns><finProv>0.00</f bMtd>inProv><insProv>0.00</insProv><provNPTyp /><depMtd>0.00</d epMtd><openArrears>0.00</openArrears><rebClub>0.00</rebClub><nxtMthOpnRe epMtd>cency>0</nxtMthOpnRecency><finRebMtd>0.00</finRebMtd><insRebMtd>0. epMtd>00</insRebMtd><empericaScore>0</empericaScore><avgDelq>0</a vgDelq><worstAge>8224</worstAge><monthsDelq>0</monthsDelq><openNowDue>0. vgDelq>0</openNowDue><queuePriority vgDelq>/><triadQueue>bdc03</triadQueue><hpfstatNcaIntMtd>0.00</hpfstatNc vgDelq>aIntMtd><hpfstatNcaIntLtd>0.00< /hpfstatNcaIntLtd><hpfstatAddIntMtd>0.00</hpfstatAddIntMtd><hpfstatAddIntLtd>0.00</hpfstatAddIntLtd><hpfstatCrLifeInsMtd>0.00</hpfstatCrLifeInsMtd><hpfstatCrLifeInsLtd>0.00</hpfstatCrLifeInsLtd><hp fstatAddCrLifeInsMtd>0.00</hpfstatAddCrLifeInsMtd><hpfstatAddCrLifeInsLt fstatAddCrLifeInsMtd>d>0.00</hpfstatAddCrLifeInsLtd><hpfstatClubMtd>0.00 fstatAddCrLifeInsMtd></hpfstatClubMtd><hpfstatClubLtd>0.00</hpfstatClubL fstatAddCrLifeInsMtd>td><hpfstatServiceFeeMt d>0.00</hpfstatServiceFeeMtd><hpfstatServiceFeeLtd>0.00</hpfstatServiceF d>eeLtd><hpfstatOptInsMtd>0.00</hpfstatOptInsMtd><hpfstatOptInsLtd>0.00< d>/hpfstatOptInsLtd><hpfstatWoff>0.00</hpfstatWoff><hpfst atdebltd>1437.06</hpfstatdebltd><hpfstatcrdltd>-25.01</hpfstatcrdltd><hp atdebltd>fstatpayltd>-1412.05</hpfstatpayltd><hpfstatintltd>0.00</hpfsta atdebltd>tintltd><hpfstatdepltd>400.00</hpfstatdepltd><contDelq>0</cont Delq><hpfstatLastPayDte>948405600000</hpfstatLastPayDte><hpfstatIndupBal Delq>>0.00</hpfstatIndupBal><hpfstatIndupLtd>0.00</hpfstatIndupLtd></sta Delq>tusData01></accountData01><docValueData01><branch>0023</br anch><account>103558</account><value>0.00</value></docValueData01></nonF anch>inancialEventVer01>\r+ Is \r + and \r+ valid on this XML file? , I need to tell application Developers if this right or not. -- Adrian Klaver adrian.klaver@aklaver.com CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 06/18/2014 04:05 AM, Khangelani Gama wrote: > -----Original Message----- > From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] > Sent: Wednesday, June 18, 2014 4:32 AM > To: Khangelani Gama; Andres Freund; Tom Lane > Cc: Alban Hertroys; Ziggy Skalski; pgsql-general@postgresql.org > Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , > ERROR: missing data for column > > On 06/17/2014 10:37 AM, Khangelani Gama wrote: >> -----Original Message----- >> From: Andres Freund [mailto:andres@2ndquadrant.com] >> Sent: Tuesday, June 17, 2014 4:38 PM >> To: Tom Lane >> Cc: Alban Hertroys; Khangelani Gama; Ziggy Skalski; >> pgsql-general@postgresql.org >> Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of >> file , ERROR: missing data for column >> >> On 2014-06-17 10:25:45 -0400, Tom Lane wrote: >>> Alban Hertroys <haramrae@gmail.com> writes: >>>> Or just connect your 9.2.4 pg_dump remotely to your 8.4 server. >>> >>> FWIW, the described behavior sounded like the dump file had gotten >>> truncated somewhere along the line. Maybe it's just necessary to >>> re-do the transfer of the existing dump file. >> >> Hi Tom, I tried the dump several times, and dumped it more than once >> but no luck. >> > > My suspicion is the XML field two places before the fpp_cde field. > Interested in seeing if the 9.4 dump will handle it better. > > > Adrian you might be right I have just tried to pg_restore on the same > server(8.4.2), I get the same errors related to xml. Can someone advise if > the following XML file as an example is in the correct state or not: The other strange thing is: ERROR: missing data for column "fpp_cde" CONTEXT: COPY audit, line 2565159: "4585427 21~21~21 5143 0 2013-12-01 15:03:18.336718 <?xml version="1.0" encoding="UTF-8"?>\r\n<nonF..." STATEMENT: COPY audit (aud_id, grp_cde, act_typ, aud_ver, usr_id, aud_device_id, aud_ts, aud_xml, aud_id 4585427 grp_code 21~21~21 act_type 5143 aud_ver 0 usr_id 2013-12-01 15:03:18.336718 ???? Is this not aud_ts? aud_device_id <?xml version="1.0" encoding="UTF-8"?>\r\n<nonF... ????? Is this not aud_xml? Which begs the question where did usr_id and aud_device_id go? -- Adrian Klaver adrian.klaver@aklaver.com
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
> Adrian you might be right I have just tried to pg_restore on the same > server(8.4.2), I get the same errors related to xml. Can someone > advise if the following XML file as an example is in the correct state or > not: The other strange thing is: ERROR: missing data for column "fpp_cde" CONTEXT: COPY audit, line 2565159: "4585427 21~21~21 5143 0 2013-12-01 15:03:18.336718 <?xml version="1.0" encoding="UTF-8"?>\r\n<nonF..." STATEMENT: COPY audit (aud_id, grp_cde, act_typ, aud_ver, usr_id, aud_device_id, aud_ts, aud_xml, aud_id 4585427 grp_code 21~21~21 act_type 5143 aud_ver 0 usr_id 2013-12-01 15:03:18.336718 ???? Is this not aud_ts? aud_device_id <?xml version="1.0" encoding="UTF-8"?>\r\n<nonF... ????? Is this not aud_xml? Which begs the question where did usr_id and aud_device_id go? I have tried to restore the dump which I created using postgres 9 binaries from postgres 8 database, I got more data issues. The problem it's the data inside the database that's broken. Application developers are looking at the problem. Thanks very much for your help , much appreciated. -- Adrian Klaver adrian.klaver@aklaver.com CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
> Adrian you might be right I have just tried to pg_restore on the same > server(8.4.2), I get the same errors related to xml. Can someone > advise if the following XML file as an example is in the correct state or > not: Hi all I have a question, why is that pg_dump does not fail when dumping the database but only fails when restoring the database? Dumping the database from same server and restoring it on the same server. Since there is data issues inside the database why is the pg_dump not failing as well? This is postgres 8.4.2. Please assist. This is a frustrating , application developers need answers from us. Regards, Khangelani CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 19 June 2014 13:11, Khangelani Gama <kgama@argility.com> wrote: >> Adrian you might be right I have just tried to pg_restore on the same >> server(8.4.2), I get the same errors related to xml. Can someone >> advise if the following XML file as an example is in the correct state or >> not: > > Hi all > > I have a question, why is that pg_dump does not fail when dumping the > database but only fails when restoring the database? > Dumping the database from same server and restoring it on the same server. > Since there is data issues inside the database why is the pg_dump not > failing as well? Because then you wouldn't be able to get your data out of your database once some corruption occurred. You would be forced to fix the issue on your live database. Now you can edit the dump and attempt to restore it until it succeeds on a different system, after which you know for certain that your data matches at least your integrity constraints. -- If you can't see the forest for the trees, Cut the trees and you'll see there is no forest.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
-----Original Message----- From: Alban Hertroys [mailto:haramrae@gmail.com] Sent: Thursday, June 19, 2014 1:54 PM To: Khangelani Gama Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column On 19 June 2014 13:11, Khangelani Gama <kgama@argility.com> wrote: >> Adrian you might be right I have just tried to pg_restore on the same >> server(8.4.2), I get the same errors related to xml. Can someone >> advise if the following XML file as an example is in the correct >> state or >> not: > > Hi all > > I have a question, why is that pg_dump does not fail when dumping the > database but only fails when restoring the database? > Dumping the database from same server and restoring it on the same server. > Since there is data issues inside the database why is the pg_dump not > failing as well? Because then you wouldn't be able to get your data out of your database once some corruption occurred. You would be forced to fix the issue on your live database. Now you can edit the dump and attempt to restore it until it succeeds on a different system, after which you know for certain that your data matches at least your integrity constraints. That makes sense to me, thanks ....Is there is an easier or better way to edit the binary dump file? Like how I can I fix a certain row or a column of pg_dump -Fc file? .... I am sorry I have never done this before....please give a clue on how I can do it. -- If you can't see the forest for the trees, Cut the trees and you'll see there is no forest. CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
On 06/19/2014 05:03 AM, Khangelani Gama wrote: > -----Original Message----- > From: Alban Hertroys [mailto:haramrae@gmail.com] > Sent: Thursday, June 19, 2014 1:54 PM > To: Khangelani Gama > Cc: pgsql-general@postgresql.org > Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , > ERROR: missing data for column > > On 19 June 2014 13:11, Khangelani Gama <kgama@argility.com> wrote: >>> Adrian you might be right I have just tried to pg_restore on the same >>> server(8.4.2), I get the same errors related to xml. Can someone >>> advise if the following XML file as an example is in the correct >>> state or >>> not: >> >> Hi all >> >> I have a question, why is that pg_dump does not fail when dumping the >> database but only fails when restoring the database? >> Dumping the database from same server and restoring it on the same server. >> Since there is data issues inside the database why is the pg_dump not >> failing as well? > > Because then you wouldn't be able to get your data out of your database once > some corruption occurred. You would be forced to fix the issue on your live > database. > > Now you can edit the dump and attempt to restore it until it succeeds on a > different system, after which you know for certain that your data matches at > least your integrity constraints. > > That makes sense to me, thanks ....Is there is an easier or better way to > edit the binary dump file? Like how I can I fix a certain row or a column of > pg_dump -Fc file? .... I am sorry I have never done this before....please > give a clue on how I can do it. You can think of the binary dump as a 'frozen' version of your database. Generally you restore to another database, but it is also possible to restore to a text file: http://www.postgresql.org/docs/9.3/interactive/app-pgrestore.html So if you use: -f filename --file=filename Specify output file for generated script, or for the listing when used with -l. Default is the standard output. instead of a database name you will create a text version of the database dump. A word of caution, the text based version will be considerably bigger than the binary compressed version. As I recall you said the database in question is large so you might not want to convert the whole thing. The same switches apply to restoring to a file that work with restoring to a database. So it is possible to select one or more tables and/or data and restore that only. The text file that is generated can then be inspected. What cannot be done is after making changes is reincorporating back into the binary dump. What you do after making the changes depends on the relationships between the changed tables and the other tables in the database. That would require more information. Another thought. Since pg_dump uses COPY(unless you specify INSERTs) you might want to look at the COPY command and see if that is a better way of seeing what is being retrieved from the table: http://www.postgresql.org/docs/9.3/interactive/sql-copy.html IMPORTANT: " COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible to the server and the name must be specified from the viewpoint of the server. When PROGRAM is specified, the server executes the given command, and reads from its standard input, or writes to its standard output. The command must be specified from the viewpoint of the server, and be executable by the postgres user. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server" With COPY you can use a query so it is possible to restrict the data you retrieve to a subset of the total. > > > > > -- Adrian Klaver adrian.klaver@aklaver.com
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
-----Original Message----- From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] Sent: Thursday, June 19, 2014 3:34 PM To: Khangelani Gama; Alban Hertroys Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column On 06/19/2014 05:03 AM, Khangelani Gama wrote: > -----Original Message----- > From: Alban Hertroys [mailto:haramrae@gmail.com] > Sent: Thursday, June 19, 2014 1:54 PM > To: Khangelani Gama > Cc: pgsql-general@postgresql.org > Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of > file , > ERROR: missing data for column > > On 19 June 2014 13:11, Khangelani Gama <kgama@argility.com> wrote: >>> Adrian you might be right I have just tried to pg_restore on the >>> same server(8.4.2), I get the same errors related to xml. Can >>> someone advise if the following XML file as an example is in the >>> correct state or >>> not: >> >> Hi all >> >> I have a question, why is that pg_dump does not fail when dumping the >> database but only fails when restoring the database? >> Dumping the database from same server and restoring it on the same >> server. >> Since there is data issues inside the database why is the pg_dump not >> failing as well? > > Because then you wouldn't be able to get your data out of your > database once some corruption occurred. You would be forced to fix the > issue on your live database. > > Now you can edit the dump and attempt to restore it until it succeeds > on a different system, after which you know for certain that your data > matches at least your integrity constraints. > > That makes sense to me, thanks ....Is there is an easier or better way > to edit the binary dump file? Like how I can I fix a certain row or a > column of pg_dump -Fc file? .... I am sorry I have never done this > before....please give a clue on how I can do it. You can think of the binary dump as a 'frozen' version of your database. Generally you restore to another database, but it is also possible to restore to a text file: http://www.postgresql.org/docs/9.3/interactive/app-pgrestore.html So if you use: -f filename --file=filename Specify output file for generated script, or for the listing when used with -l. Default is the standard output. instead of a database name you will create a text version of the database dump. A word of caution, the text based version will be considerably bigger than the binary compressed version. As I recall you said the database in question is large so you might not want to convert the whole thing. The same switches apply to restoring to a file that work with restoring to a database. So it is possible to select one or more tables and/or data and restore that only. The text file that is generated can then be inspected. What cannot be done is after making changes is reincorporating back into the binary dump. What you do after making the changes depends on the relationships between the changed tables and the other tables in the database. That would require more information. Another thought. Since pg_dump uses COPY(unless you specify INSERTs) you might want to look at the COPY command and see if that is a better way of seeing what is being retrieved from the table: http://www.postgresql.org/docs/9.3/interactive/sql-copy.html IMPORTANT: " COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible to the server and the name must be specified from the viewpoint of the server. When PROGRAM is specified, the server executes the given command, and reads from its standard input, or writes to its standard output. The command must be specified from the viewpoint of the server, and be executable by the postgres user. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server" With COPY you can use a query so it is possible to restrict the data you retrieve to a subset of the total. Many Thanks, I think I will begin with pg_dump that uses INTERTS and see what I get. > > > > > -- Adrian Klaver adrian.klaver@aklaver.com CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: pg_restore: custom archiver unexpected end of file , ERROR: missing data for column
> Now you can edit the dump and attempt to restore it until it succeeds > on a different system, after which you know for certain that your data > matches at least your integrity constraints. > > That makes sense to me, thanks ....Is there is an easier or better way > to edit the binary dump file? Like how I can I fix a certain row or a > column of pg_dump -Fc file? .... I am sorry I have never done this > before....please give a clue on how I can do it. You can think of the binary dump as a 'frozen' version of your database. Generally you restore to another database, but it is also possible to restore to a text file: http://www.postgresql.org/docs/9.3/interactive/app-pgrestore.html So if you use: -f filename --file=filename Specify output file for generated script, or for the listing when used with -l. Default is the standard output. instead of a database name you will create a text version of the database dump. A word of caution, the text based version will be considerably bigger than the binary compressed version. As I recall you said the database in question is large so you might not want to convert the whole thing. The same switches apply to restoring to a file that work with restoring to a database. So it is possible to select one or more tables and/or data and restore that only. The text file that is generated can then be inspected. What cannot be done is after making changes is reincorporating back into the binary dump. What you do after making the changes depends on the relationships between the changed tables and the other tables in the database. That would require more information. Another thought. Since pg_dump uses COPY(unless you specify INSERTs) you might want to look at the COPY command and see if that is a better way of seeing what is being retrieved from the table: http://www.postgresql.org/docs/9.3/interactive/sql-copy.html IMPORTANT: " COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible to the server and the name must be specified from the viewpoint of the server. When PROGRAM is specified, the server executes the given command, and reads from its standard input, or writes to its standard output. The command must be specified from the viewpoint of the server, and be executable by the postgres user. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server" With COPY you can use a query so it is possible to restrict the data you retrieve to a subset of the total. Many Thanks, I think I will begin with pg_dump that uses INTERTS and see what I get. Hi all, Latest thing I picked now is that the disc is broken. It has Input /Output Errors. We are going to try an land the file to a remote server when dumping the database. > > > > > -- Adrian Klaver adrian.klaver@aklaver.com CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.