Обсуждение: PostgreSQL 9.6.1 - Windows 64bit - HDD crash due to power outage

Поиск
Список
Период
Сортировка

PostgreSQL 9.6.1 - Windows 64bit - HDD crash due to power outage

От
Ertan Küçükoğlu
Дата:
Hi All,

One of my test systems had electricity problem and operating system is not
booting now. I can recover data by connecting that HDD to another computer
as a USB disc. I confirm that I did a file copy complete data directory of
several GB in size just fine. I have no idea about their structural
integrity though.

Since this computer was used for tests I did not setup any backup plan on
it. However, some of my colleagues seems to have some important database
structure *and* data saved in that system. Of course, I learn that after
disaster happens, but that's another story.

* There was only one database on the server.
* No replication of any kind was setup. Server was a standalone.
* Auto vacuum setting was not changed after server setup, and I do not know
its defaults. I do know that it is was set to do auto vacuum.
* New data was added to database after midnight just once a day. Data to be
inserted was fetched over internet. If first trial fails there were
additional trials set for every 30 minutes for 5 hours max (10 trials
total). If any data insert trial is a success other trials were executed,
but did not insert any data.
* System otherwise is used for select queries. So, I am guessing  that there
was no data inserted for several hours at the time of power outage. Do not
know about vacuum status or if database server was doing something else.

My questions are;
- Is there a backup tool/way to have some kind of a backup from raw
PostgreSQL database files (data read from data directory rather than
database server) which can be restored on a different PostgreSQL server?
- If I setup a new PostgreSQL server using identical binaries as the crashed
one, and copy rescued data directory onto that new server data directory,
overwriting new system data directory when server is down. Will that work
assuming recovered data is structurally fine? If that method works, how can
I test that database files are fine and there is no problem with them?
- I am open to all other suggestions here.

Thanks & regards,
Ertan Küçükoğlu




Re: PostgreSQL 9.6.1 - Windows 64bit - HDD crash due to power outage

От
Samed YILDIRIM
Дата:
Hi Ertan,
 
Most probably, It will work by copying your old data directory to a new identical PostgreSQL server's data directory. But, you should be sure that you have copied your data directory correctly, not only base directory, etc.
 
1. Install an identical PostgreSQL server
2. Copy your old data directory to your new installation. Please do not move, copy them. Be sure that you have always a backup without any modification of your failed server.
3. Check folders' and files' permissions. As I remember, in Windows PostgreSQL works with Network Service user/permissions. But I'm not sure.
4. Check your logs before starting PostgreSQL Server
5. If everything is ok, delete postmaster.pid if exists.
6. Start your PostgreSQL server and follow your logs. You may need to start with recovery.conf.
 
Best regards.
 
İyi çalışmalar.
Samed YILDIRIM
 
 
 
03.04.2017, 23:06, "Ertan Küçükoğlu" <ertan.kucukoglu@1nar.com.tr>:

Hi All,

One of my test systems had electricity problem and operating system is not
booting now. I can recover data by connecting that HDD to another computer
as a USB disc. I confirm that I did a file copy complete data directory of
several GB in size just fine. I have no idea about their structural
integrity though.

Since this computer was used for tests I did not setup any backup plan on
it. However, some of my colleagues seems to have some important database
structure *and* data saved in that system. Of course, I learn that after
disaster happens, but that's another story.

* There was only one database on the server.
* No replication of any kind was setup. Server was a standalone.
* Auto vacuum setting was not changed after server setup, and I do not know
its defaults. I do know that it is was set to do auto vacuum.
* New data was added to database after midnight just once a day. Data to be
inserted was fetched over internet. If first trial fails there were
additional trials set for every 30 minutes for 5 hours max (10 trials
total). If any data insert trial is a success other trials were executed,
but did not insert any data.
* System otherwise is used for select queries. So, I am guessing that there
was no data inserted for several hours at the time of power outage. Do not
know about vacuum status or if database server was doing something else.

My questions are;
- Is there a backup tool/way to have some kind of a backup from raw
PostgreSQL database files (data read from data directory rather than
database server) which can be restored on a different PostgreSQL server?
- If I setup a new PostgreSQL server using identical binaries as the crashed
one, and copy rescued data directory onto that new server data directory,
overwriting new system data directory when server is down. Will that work
assuming recovered data is structurally fine? If that method works, how can
I test that database files are fine and there is no problem with them?
- I am open to all other suggestions here.

Thanks & regards,
Ertan Küçükoğlu



 --
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: PostgreSQL 9.6.1 - Windows 64bit - HDD crash due to power outage

От
"David G. Johnston"
Дата:
On Monday, April 3, 2017, Ertan Küçükoğlu <ertan.kucukoglu@1nar.com.tr> wrote:
- If I setup a new PostgreSQL server using identical binaries as the crashed
one, and copy rescued data directory onto that new server data directory,
overwriting new system data directory when server is down. Will that work
assuming recovered data is structurally fine?

If you can copy over the last versions of the WAL and data files (including non-default tablespaces) (and ideally configuration files) and point the same binaries to those it will work in theory.
 
 If that method works, how can
I test that database files are fine and there is no problem with them?

Performing a full pg_dump on the database/cluster is the only routine I can think of - but my experience here is limited.

David J.

Re: PostgreSQL 9.6.1 - Windows 64bit - HDD crash due to power outage

От
Ertan Küçükoğlu
Дата:
- Using identical binary distribution,
- Setting data folder permissions
- Copying over data directory on a new server
did the job.

Everything is back in track at the moment. Including backup plan.

Thank you all.



Re: PostgreSQL 9.6.1 - Windows 64bit - HDD crash due to power outage

От
Steven Chang
Дата:
Hello Samed ,

    No need recovery.conf due to no wal archive , I guess.
    From Ertan's words, the testing environment was setup by default.
    Crash recovery mechanism would cope with recovery from WAL as shutdown immediate.
  
Hello Ertan,
    
       Just proceed your recovery operations.
       Good luck.

Best Regards,
Steven

2017-04-04 4:20 GMT+08:00 Samed YILDIRIM <samed@reddoc.net>:
Hi Ertan,
 
Most probably, It will work by copying your old data directory to a new identical PostgreSQL server's data directory. But, you should be sure that you have copied your data directory correctly, not only base directory, etc.
 
1. Install an identical PostgreSQL server
2. Copy your old data directory to your new installation. Please do not move, copy them. Be sure that you have always a backup without any modification of your failed server.
3. Check folders' and files' permissions. As I remember, in Windows PostgreSQL works with Network Service user/permissions. But I'm not sure.
4. Check your logs before starting PostgreSQL Server
5. If everything is ok, delete postmaster.pid if exists.
6. Start your PostgreSQL server and follow your logs. You may need to start with recovery.conf.
 
Best regards.
 
İyi çalışmalar.
Samed YILDIRIM
 
 
 
03.04.2017, 23:06, "Ertan Küçükoğlu" <ertan.kucukoglu@1nar.com.tr>:

Hi All,

One of my test systems had electricity problem and operating system is not
booting now. I can recover data by connecting that HDD to another computer
as a USB disc. I confirm that I did a file copy complete data directory of
several GB in size just fine. I have no idea about their structural
integrity though.

Since this computer was used for tests I did not setup any backup plan on
it. However, some of my colleagues seems to have some important database
structure *and* data saved in that system. Of course, I learn that after
disaster happens, but that's another story.

* There was only one database on the server.
* No replication of any kind was setup. Server was a standalone.
* Auto vacuum setting was not changed after server setup, and I do not know
its defaults. I do know that it is was set to do auto vacuum.
* New data was added to database after midnight just once a day. Data to be
inserted was fetched over internet. If first trial fails there were
additional trials set for every 30 minutes for 5 hours max (10 trials
total). If any data insert trial is a success other trials were executed,
but did not insert any data.
* System otherwise is used for select queries. So, I am guessing that there
was no data inserted for several hours at the time of power outage. Do not
know about vacuum status or if database server was doing something else.

My questions are;
- Is there a backup tool/way to have some kind of a backup from raw
PostgreSQL database files (data read from data directory rather than
database server) which can be restored on a different PostgreSQL server?
- If I setup a new PostgreSQL server using identical binaries as the crashed
one, and copy rescued data directory onto that new server data directory,
overwriting new system data directory when server is down. Will that work
assuming recovered data is structurally fine? If that method works, how can
I test that database files are fine and there is no problem with them?
- I am open to all other suggestions here.

Thanks & regards,
Ertan Küçükoğlu



 --
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin