Обсуждение: database restoration problem- data became incorrect caused by incorrect date in the server
database restoration problem- data became incorrect caused by incorrect date in the server
Hi all
I am a junior DBA:
Please give an advice on how I should solve the following problem,
We are still using 7.3.4 PostgreSQL but will be upgrading very soon, database is running on CentOS 5 O/S .
Problem: - it’s about the wrong data in the system:
Everything was fine until a certain process(for financial period processing) that normally generates many transactions in the database took place when the server lost the date. The date changed from 07-04-2010 to 15-08-2003…hence the process ran until it finished which leaded to the generation of the wrong data.
When we thought of restoring from a successful dump file(backup file) for the 07-04-2010 so that we could restore it to have everything before the date changed, we were not lucky because the only latest successful dump file was on the 03/04/2010.
Question:
How can we removed everything that got processed on the wrong date and have everything restored so that it’s in the correct state as at 07/04/2010… I tried to search for a solution but I can’t find anything.
Your help would be much appreciated
Thanks
Khangelani
Confidentiality Notice:http://ucs.co.za/conf.html
The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: database restoration problem- data became incorrect caused by incorrect date in the server
To add on to previous comments:
There is only one database in the server and the backup server also has incorrect data because all the wrong data replicated to the backup server.
From: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] On Behalf Of Khangelani Gama
Sent: Friday, April 09, 2010 2:22 PM
To: pgsql-admin@postgresql.org
Subject: [ADMIN] database restoration problem- data became incorrect caused by incorrect date in the server
Hi all
I am a junior DBA:
Please give an advice on how I should solve the following problem,
We are still using 7.3.4 PostgreSQL but will be upgrading very soon, database is running on CentOS 5 O/S .
Problem: - it’s about the wrong data in the system:
Everything was fine until a certain process(for financial period processing) that normally generates many transactions in the database took place when the server lost the date. The date changed from 07-04-2010 to 15-08-2003…hence the process ran until it finished which leaded to the generation of the wrong data.
When we thought of restoring from a successful dump file(backup file) for the 07-04-2010 so that we could restore it to have everything before the date changed, we were not lucky because the only latest successful dump file was on the 03/04/2010.
Question:
How can we removed everything that got processed on the wrong date and have everything restored so that it’s in the correct state as at 07/04/2010… I tried to search for a solution but I can’t find anything.
Your help would be much appreciated
Thanks
Khangelani
Confidentiality Notice:http://ucs.co.za/conf.html
The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: database restoration problem- data became incorrect caused by incorrect date in the server
Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote: > We are still using 7.3.4 PostgreSQL Do you really mean that, or is that a typo? What sort of backups do you have? (pg_dump, file system copy, etc.) -Kevin
Re: Re: database restoration problem- data became incorrect caused by incorrect date in the server
Hi I really mean it....the backups we have uses pg_dump command. The automatic backup dumps file happen daily and gets savedin a certain directory But the issue now is that we don't have any backup file for us to restore the database but exclude every data that got createwhen the date in server was incorrect. Thanks -----Original Message----- From: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] On Behalf Of Kevin Grittner Sent: Friday, April 09, 2010 4:14 PM To: pgsql-admin@postgresql.org; Khangelani Gama Subject: [ADMIN] Re: database restoration problem- data became incorrect caused by incorrect date in the server Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote: > We are still using 7.3.4 PostgreSQL Do you really mean that, or is that a typo? What sort of backups do you have? (pg_dump, file system copy, etc.) -Kevin -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential informationof UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intendedaddressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroythe e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthoriseduse of their e-mail facilities or e-mails sent other than strictly for business purposes.
Re: Re: database restoration problem- data became incorrect caused by incorrect date in the server
Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote: > the backups we have uses pg_dump command. The automatic > backup dumps file happen daily and gets saved in a certain > directory No off-site copies or archives to fall back on? If not, my ideas are: (1) I hope you made a file copy as soon as you found the problem. If not, stop the database and make one immediately. If the old versions of the rows haven't yet been vacuumed away, it might be possible to fish some or all of the corrupted data out of the database. This is very tricky work, though, and you might want to contract with an expert. (I don't do that anymore, but there are several good companies with people who do.) (2) Are there any source documents you can fall back on to reconstruct the data? (3) Is there any way to "reverse" the particular calculations or operations which mangled the data? Once you do get upgraded, you should look into Point In Time Recovery (PITR) backups. Had you been running this, you could have replayed the transactions right up to the moment of corruption. Well, if you kept a base backup from before the corruption and all the WAL files since that base backup. -Kevin
Re: Re: database restoration problem- data became incorrect caused by incorrect date in the server
Thanks for the ideas -----Original Message----- From: Kevin Grittner [mailto:Kevin.Grittner@wicourts.gov] Sent: Friday, April 09, 2010 8:03 PM To: pgsql-admin@postgresql.org; Khangelani Gama Subject: RE: [ADMIN] Re: database restoration problem- data became incorrect caused by incorrect date in the server Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote: > the backups we have uses pg_dump command. The automatic > backup dumps file happen daily and gets saved in a certain > directory No off-site copies or archives to fall back on? If not, my ideas are: (1) I hope you made a file copy as soon as you found the problem. If not, stop the database and make one immediately. If the old versions of the rows haven't yet been vacuumed away, it might be possible to fish some or all of the corrupted data out of the database. This is very tricky work, though, and you might want to contract with an expert. (I don't do that anymore, but there are several good companies with people who do.) (2) Are there any source documents you can fall back on to reconstruct the data? (3) Is there any way to "reverse" the particular calculations or operations which mangled the data? Once you do get upgraded, you should look into Point In Time Recovery (PITR) backups. Had you been running this, you could have replayed the transactions right up to the moment of corruption. Well, if you kept a base backup from before the corruption and all the WAL files since that base backup. -Kevin The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential informationof UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intendedaddressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroythe e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthoriseduse of their e-mail facilities or e-mails sent other than strictly for business purposes.