Обсуждение: Linux group access to ..../psql/data & subdirectories

Hi again

9.2.4 on RHEL 6

I have a backup that runs by cronjob every day which is basically this:

00 19 * * *   /backup_scripts/locsng_backup.sh > /cron/locsng_backup.out 2>&1

View /backup_scripts/locsng_backup.sh:

# carry out schema only  backup of the db database
/pg_dumpall -l db -U postgres -s > /backups/pg_db_schema_backups/db_ci_schema_dump$date_of_backup.sql 2>&1

# carry out backup of db database
/pg_dump -Fc db -U postgres > /backups/pg_db_db_backups/db_ci_backup$date_of_backup

# carry out cluster wide metadata backup
/pg_dumpall -g > /backups/pg_cluster_dump/cluster_ci_medadata$date_of_backup.sql


Is there a best practise way of how to monitor this sort of backup?

 - my initial ideas are:

1. Check by cron if /cron/locsng_backup.out > 24 hours old (of course this will not work if cron fails) and if so notify the DBA by email?

2. As part of the script check if cron/locsng_backup.out > 0 bytes in size (of course this will not work if cron fails) and if so notify the DBA by email?

3. Does pg_dump,  pg_dumpall return an error code if it is unsuccessful and can be trapped in same way and then used to notify the DBA by email?


Thanks in advance

Austen


Austen Birchall  Senior Database Administrator
Met Office


--
Sent via pgsql-novice mailing list (pgsql-novice@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Payal

 

Thanks for this:

schedule email to DBA if there is any error in the backup creation script itself (mail if there is anything in the stderr)

 

will the 2>&1 part of the crontab direct the stderr into the /cron/db_backup.out file ?

 

 

00 19 * * *   /backup_scripts/db_backup.sh > /cron/db_backup.out 2>&1

 

If so I could check by cron if this file > 0 bytes and if so mail to DBA?

 

Austen

 

Austen Birchall  Senior Database Administrator

Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
Tel: +44 (0)1392 884481 Fax: +44 (0)1392 885681
E-mail: austen.birchall@metoffice.gov.uk Website: http://www.metoffice.gov.uk

 

From: Payal Singh [mailto:payal@omniti.com]
Sent: 14 October 2013 16:19
To: Birchall, Austen
Cc: pgsql-novice@postgresql.org
Subject: Re: [NOVICE] How to Monitor backups

 

Hey,

 

Any automated monitoring job is based on the assumption that the tool automating it will work fine. With this assumption in mind, option 1 seems like the simplest and most efficient way. 

 

Of course, yet another precaution you can take is to schedule email to DBA if there is any error in the backup creation script itself (mail if there is anything in the stderr), which will also take into account any error generated by pg_dump/dumpall commands.

Payal Singh,
OmniTi Computer Consulting Inc.
Junior Database Architect,
Phone: 240.646.0770 x 253

 

On Mon, Oct 14, 2013 at 11:08 AM, Birchall, Austen <austen.birchall@metoffice.gov.uk> wrote:

Hi again

9.2.4 on RHEL 6

I have a backup that runs by cronjob every day which is basically this:

00 19 * * *   /backup_scripts/locsng_backup.sh > /cron/locsng_backup.out 2>&1

View /backup_scripts/locsng_backup.sh:

# carry out schema only  backup of the db database
/pg_dumpall -l db -U postgres -s > /backups/pg_db_schema_backups/db_ci_schema_dump$date_of_backup.sql 2>&1

# carry out backup of db database
/pg_dump -Fc db -U postgres > /backups/pg_db_db_backups/db_ci_backup$date_of_backup

# carry out cluster wide metadata backup
/pg_dumpall -g > /backups/pg_cluster_dump/cluster_ci_medadata$date_of_backup.sql


Is there a best practise way of how to monitor this sort of backup?

 - my initial ideas are:

1. Check by cron if /cron/locsng_backup.out > 24 hours old (of course this will not work if cron fails) and if so notify the DBA by email?

2. As part of the script check if cron/locsng_backup.out > 0 bytes in size (of course this will not work if cron fails) and if so notify the DBA by email?

3. Does pg_dump,  pg_dumpall return an error code if it is unsuccessful and can be trapped in same way and then used to notify the DBA by email?


Thanks in advance

Austen


Austen Birchall  Senior Database Administrator
Met Office


--
Sent via pgsql-novice mailing list (pgsql-novice@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

 

Austen Birchall wrote:
> We are currently looking at Barman and have just installed it on a test server.
>
> However, our Sys Admin has said:
>
> A couple of notes though, the work order asks for the instructions found at http://docs.pgbarman.org/
> be followed for installation. However they request that you add an additional repository to the server
> in order to access the rpms. We cannot add additional repos to servers and so to get the packages
> installed on the box they were downloaded manually on a desktop machine and then moved to the server
> and installed locally. This method will not scale to production and someone will need to take
> ownership of this piece of software and provide us with rpms (or preferably in the long term a local
> repo containing them) ensuring that they have been provided by a trusted source.
>
> I’ll be the first to admit that I don’t fully understand all aout rpms etc. etc. so does anybody have
> any any insights, comments etc. on how to proceed here?

I'd say that either an enterprise is small enough that they can
afford to distribute and install software manually or it
is big enough to have its own software distribution system.

How do you distribute other software (like PostgreSQL) to your servers?

Maybe I am missing something there, but what keeps you from adding the
RPMs from http://yum.postgresql.org/ to your own software distribution
system?

So I don't understand this argument at all.

The second argument ("ownership of the software", "trusted source")
sounds slightly like "it doesn't come from a large corporation, so we
don't trust it".

Maybe I read that wrong, but it seems to call for skills in advocacy
more than anything else:
If you don't trust the RPMs, what reason do you have to trust the
source code behind it?
Why do you trust RedHat (or whatever vendor your Linux comes from)?
They don't own Linux.
But they are big, so they are trustworthy, right?
You sure?

But of course you don't have to trust Barman, that's the point.
You can examine the source code to form your own opinion.
You can roll your own RPMs if the ones provided don't suit you
(that's not such a hard exercise).

Yours,
Laurenz Albe

--
Sent via pgsql-novice mailing list (pgsql-novice@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

If RPMs are giving trouble, try installing from source code instead. 

Payal Singh,
OmniTi Computer Consulting Inc.
Junior Database Architect,
Phone: 240.646.0770 x 253

On Fri, Nov 29, 2013 at 8:32 AM, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:

Austen Birchall wrote:
> We are currently looking at Barman and have just installed it on a test server.
>
> However, our Sys Admin has said:
>
> A couple of notes though, the work order asks for the instructions found at http://docs.pgbarman.org/
> be followed for installation. However they request that you add an additional repository to the server
> in order to access the rpms. We cannot add additional repos to servers and so to get the packages
> installed on the box they were downloaded manually on a desktop machine and then moved to the server
> and installed locally. This method will not scale to production and someone will need to take
> ownership of this piece of software and provide us with rpms (or preferably in the long term a local
> repo containing them) ensuring that they have been provided by a trusted source.
>
> I’ll be the first to admit that I don’t fully understand all aout rpms etc. etc. so does anybody have
> any any insights, comments etc. on how to proceed here?

I'd say that either an enterprise is small enough that they can
afford to distribute and install software manually or it
is big enough to have its own software distribution system.

How do you distribute other software (like PostgreSQL) to your servers?

Maybe I am missing something there, but what keeps you from adding the
RPMs from http://yum.postgresql.org/ to your own software distribution
system?

So I don't understand this argument at all.

The second argument ("ownership of the software", "trusted source")
sounds slightly like "it doesn't come from a large corporation, so we
don't trust it".

Maybe I read that wrong, but it seems to call for skills in advocacy
more than anything else:
If you don't trust the RPMs, what reason do you have to trust the
source code behind it?
Why do you trust RedHat (or whatever vendor your Linux comes from)?
They don't own Linux.
But they are big, so they are trustworthy, right?
You sure?

But of course you don't have to trust Barman, that's the point.
You can examine the source code to form your own opinion.
You can roll your own RPMs if the ones provided don't suit you
(that's not such a hard exercise).

Yours,
Laurenz Albe

--
Sent via pgsql-novice mailing list (pgsql-novice@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

If RPMs are giving trouble, try installing from source code instead. 

Payal Singh,
OmniTi Computer Consulting Inc.
Junior Database Architect,
Phone: 240.646.0770 x 253

On Fri, Nov 29, 2013 at 8:32 AM, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:

Austen Birchall wrote:
> We are currently looking at Barman and have just installed it on a test server.
>
> However, our Sys Admin has said:
>
> A couple of notes though, the work order asks for the instructions found at http://docs.pgbarman.org/
> be followed for installation. However they request that you add an additional repository to the server
> in order to access the rpms. We cannot add additional repos to servers and so to get the packages
> installed on the box they were downloaded manually on a desktop machine and then moved to the server
> and installed locally. This method will not scale to production and someone will need to take
> ownership of this piece of software and provide us with rpms (or preferably in the long term a local
> repo containing them) ensuring that they have been provided by a trusted source.
>
> I’ll be the first to admit that I don’t fully understand all aout rpms etc. etc. so does anybody have
> any any insights, comments etc. on how to proceed here?

I'd say that either an enterprise is small enough that they can
afford to distribute and install software manually or it
is big enough to have its own software distribution system.

How do you distribute other software (like PostgreSQL) to your servers?

Maybe I am missing something there, but what keeps you from adding the
RPMs from http://yum.postgresql.org/ to your own software distribution
system?

So I don't understand this argument at all.

The second argument ("ownership of the software", "trusted source")
sounds slightly like "it doesn't come from a large corporation, so we
don't trust it".

Maybe I read that wrong, but it seems to call for skills in advocacy
more than anything else:
If you don't trust the RPMs, what reason do you have to trust the
source code behind it?
Why do you trust RedHat (or whatever vendor your Linux comes from)?
They don't own Linux.
But they are big, so they are trustworthy, right?
You sure?

But of course you don't have to trust Barman, that's the point.
You can examine the source code to form your own opinion.
You can roll your own RPMs if the ones provided don't suit you
(that's not such a hard exercise).

Yours,
Laurenz Albe

--
Sent via pgsql-novice mailing list (pgsql-novice@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice