Обсуждение: error on online backup using pg_basebackup tool

Поиск
Список
Период
Сортировка

error on online backup using pg_basebackup tool

От
Gerdan Rezende dos Santos
Дата:
Hi 4 all!!

As Ido not know if it is my mistake or actually a bug, sending the two topics.

When trying to perform online backup using the pg_basebackup tool, I get the message: The folder is not empty.

steps to reproduce the error:
1 - install postgres in the / usr / local / pgsql
2 - initialize the database: / usr / local / pgsql / bin / -D /data
3 - set up pg_hba.conf to allow replication (used by online backup with pg_basebackup)
4 - create the directories /data/newtblspc and /data/pg_tblspc/newtblspc2
5 - start with postgres -D /data
6 - create table spaces:
CREATE TABLESPACE newtblspc LOCATION '/data/newtblspc';
CREATE TABLESPACE newtblspc2 LOCATION '/data/pg_tblspc/newtblspc2'
7 - mkdir /data2
8 - Now perform a backup using pg_basebackup -D /data2 and get the error message.





--
T.'.A.'.F.'.,
Gerdan Rezende dos Santos
Po
stgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525

Re: [BUGS] error on online backup using pg_basebackup tool

От
Michael Paquier
Дата:
On Thu, Sep 3, 2015 at 10:28 AM, Gerdan Rezende dos Santos
<gerdan@gmail.com> wrote:
> 6 - create table spaces:
> CREATE TABLESPACE newtblspc LOCATION '/data/newtblspc';
> CREATE TABLESPACE newtblspc2 LOCATION '/data/pg_tblspc/newtblspc2'
> 7 - mkdir /data2
> 8 - Now perform a backup using pg_basebackup -D /data2 and get the error
> message.

You are performing a base backup on a node that has tablespaces on the
same hosts, so tablespaces are going to overlap. Isn't that your
problem? Note that 9.4 you can use tablespace-mapping to address the
problem, for older versions you are going to need some manual
operations before being able to run the standby and the master on the
same server.
--
Michael


Re: error on online backup using pg_basebackup tool

От
Gerdan Rezende dos Santos
Дата:

I use the tablespaces without problem but the backup online not complete... but when I attempt to online backup using the pg_basebackup displays the error message saying q pg_tblspc is not clean and the process is terminated without sucess


Em quarta-feira, 2 de setembro de 2015, Michael Paquier <michael.paquier@gmail.com> escreveu:
On Thu, Sep 3, 2015 at 10:28 AM, Gerdan Rezende dos Santos
<gerdan@gmail.com> wrote:
> 6 - create table spaces:
> CREATE TABLESPACE newtblspc LOCATION '/data/newtblspc';
> CREATE TABLESPACE newtblspc2 LOCATION '/data/pg_tblspc/newtblspc2'
> 7 - mkdir /data2
> 8 - Now perform a backup using pg_basebackup -D /data2 and get the error
> message.

You are performing a base backup on a node that has tablespaces on the
same hosts, so tablespaces are going to overlap. Isn't that your
problem? Note that 9.4 you can use tablespace-mapping to address the
problem, for older versions you are going to need some manual
operations before being able to run the standby and the master on the
same server.
--
Michael


--
T.'.A.'.F.'.,
Gerdan Rezende dos Santos
Po
stgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525

Re: error on online backup using pg_basebackup tool

От
Michael Paquier
Дата:
On Thu, Sep 3, 2015 at 12:20 PM, Gerdan Rezende dos Santos
<gerdan@gmail.com> wrote:
>[...]

Note 1: Please don't top-post.
Note 2: Please don't post across multiple mailing lists.

> I use the tablespaces without problem but the backup online not complete...
> but when I attempt to online backup using the pg_basebackup displays the
> error message saying q pg_tblspc is not clean and the process is terminated
> without sucess.

That's what I am saying. You will need to redirect manually the soft
links of pg_tbspc into other paths if you want to run a standby on the
same host as the master.

That's not a bug, just an operation mistake.
--
Michael


Re: error on online backup using pg_basebackup tool

От
Gerdan Rezende dos Santos
Дата:


Em quinta-feira, 3 de setembro de 2015, Michael Paquier <michael.paquier@gmail.com> escreveu:
On Thu, Sep 3, 2015 at 12:20 PM, Gerdan Rezende dos Santos
<gerdan@gmail.com> wrote:
>[...]

Note 1: Please don't top-post.
Note 2: Please don't post across multiple mailing lists.

> I use the tablespaces without problem but the backup online not complete...
> but when I attempt to online backup using the pg_basebackup displays the
> error message saying q pg_tblspc is not clean and the process is terminated
> without sucess.

That's what I am saying. You will need to redirect manually the soft
links of pg_tbspc into other paths if you want to run a standby on the
same host as the master.

That's not a bug, just an operation mistake.
--
Michael


In other server with same configuration the proccess not complete too!


--
T.'.A.'.F.'.,
Gerdan Rezende dos Santos
Po
stgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525

Re: error on online backup using pg_basebackup tool

От
Martín Marqués
Дата:
El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>
>
> In other server with same configuration the proccess not complete too!

Add --verbose to the pg_basebackup execution and send over the output
error message.

I find your procedure confusing, and I'd recommend you *not* to put
tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
the links to where the actual tablespace is (normally on another partition)

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


Re: error on online backup using pg_basebackup tool

От
Gerdan Rezende dos Santos
Дата:
On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com> wrote:
El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>
>
> In other server with same configuration the proccess not complete too!

Add --verbose to the pg_basebackup execution and send over the output
error message.

I find your procedure confusing, and I'd recommend you *not* to put
tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
the links to where the actual tablespace is (normally on another partition)

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/" exists but is not empty

From machine A to machine B 

pb_basebackup started in machine B, connecting on machine A


On machine B
pb_basebackup -D /data/pg_tblspc/ -h machineA


T.'.A.'.F.'.,
Gerdan Rezende dos Santos 
Po
stgreSQL & EnterpriseDB Specialist, Support, Training & Services 
+55 (61) 9645-1525

Re: error on online backup using pg_basebackup tool

От
Martín Marqués
Дата:
2015-09-03 16:48 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
> On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com>
> wrote:
>>
>> El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>> >
>> >
>> > In other server with same configuration the proccess not complete too!
>>
>> Add --verbose to the pg_basebackup execution and send over the output
>> error message.
>>
>> I find your procedure confusing, and I'd recommend you *not* to put
>> tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
>> the links to where the actual tablespace is (normally on another
>> partition)
>
> This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/"
> exists but is not empty

The error message is quite clear, it has to use /data/pg_tblspc/ to
put a tablespace, but that directory is not empty.

Why are you putting tablespaces inside PGDATA? It really makes no
sense. Just leave the objects you have there in the default
tablespace.

Regards,

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


Re: error on online backup using pg_basebackup tool

От
Gerdan Rezende dos Santos
Дата:
On Thu, Sep 3, 2015 at 5:01 PM, Martín Marqués <martin@2ndquadrant.com> wrote:
2015-09-03 16:48 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
> On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com>
> wrote:
>>
>> El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>> >
>> >
>> > In other server with same configuration the proccess not complete too!
>>
>> Add --verbose to the pg_basebackup execution and send over the output
>> error message.
>>
>> I find your procedure confusing, and I'd recommend you *not* to put
>> tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
>> the links to where the actual tablespace is (normally on another
>> partition)
>
> This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/"
> exists but is not empty

The error message is quite clear, it has to use /data/pg_tblspc/ to
put a tablespace, but that directory is not empty.

Why are you putting tablespaces inside PGDATA? It really makes no
sense. Just leave the objects you have there in the default
tablespace.

Regards,

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


This problem, I found in my client ...
Why are you putting in PGDATA table ? He can not say to me
The solution then is to continue using the start_backup , rsync and stop_backup. ;(

Thanks!

Re: error on online backup using pg_basebackup tool

От
Martín Marqués
Дата:
Hi Gerdan,

2015-09-03 17:41 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
>
> This problem, I found in my client ...
> Why are you putting in PGDATA table ? He can not say to me
> The solution then is to continue using the start_backup , rsync and
> stop_backup. ;(

Or use repmgr. ;)

If you intend to use repmgr3 (will only work with postgres 9.3 and
above) you'll need to pass it the --rsync-only option so it doesn't
use pg_basebackup to clone.

Regards,

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


Re: error on online backup using pg_basebackup tool

От
Gerdan Rezende dos Santos
Дата:
On Thu, Sep 3, 2015 at 5:53 PM, Martín Marqués <martin@2ndquadrant.com> wrote:
Hi Gerdan,

2015-09-03 17:41 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
>
> This problem, I found in my client ...
> Why are you putting in PGDATA table ? He can not say to me
> The solution then is to continue using the start_backup , rsync and
> stop_backup. ;(

Or use repmgr. ;)

If you intend to use repmgr3 (will only work with postgres 9.3 and
above) you'll need to pass it the --rsync-only option so it doesn't
use pg_basebackup to clone.

Regards,

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


Matín, the client have a BART from EDB. 
EDB BART only use pg_basebackup.
Its my problem.

Re: error on online backup using pg_basebackup tool

От
Joseph Kregloh
Дата:


On Thu, Sep 3, 2015 at 4:41 PM, Gerdan Rezende dos Santos <gerdan@gmail.com> wrote:
On Thu, Sep 3, 2015 at 5:01 PM, Martín Marqués <martin@2ndquadrant.com> wrote:
2015-09-03 16:48 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
> On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com>
> wrote:
>>
>> El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>> >
>> >
>> > In other server with same configuration the proccess not complete too!
>>
>> Add --verbose to the pg_basebackup execution and send over the output
>> error message.
>>
>> I find your procedure confusing, and I'd recommend you *not* to put
>> tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
>> the links to where the actual tablespace is (normally on another
>> partition)
>
> This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/"
> exists but is not empty

The error message is quite clear, it has to use /data/pg_tblspc/ to
put a tablespace, but that directory is not empty.

Why are you putting tablespaces inside PGDATA? It really makes no
sense. Just leave the objects you have there in the default
tablespace.

Regards,

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


This problem, I found in my client ...
Why are you putting in PGDATA table ? He can not say to me
The solution then is to continue using the start_backup , rsync and stop_backup. ;(


You can always move the folder out of there. Depending on the version of PostgreSQL different steps need to be take. If I remember correctly for version 9.3 you can move the symlink to point to the new directory. Pre 9.3 you need to move the symlink and the pointer inside the pg tables.

-Joseph Kregloh
 
Thanks!

Re: error on online backup using pg_basebackup tool

От
Martín Marqués
Дата:
Hi Gerdan,

2015-09-03 18:06 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
>
> Matín, the client have a BART from EDB.
> EDB BART only use pg_basebackup.
> Its my problem.

AFAIK BART is a backup & recovery tool, and I thought you were trying
to spin new standbys, which is why I recommended repmgr. If you need
to do backups then repmgr is not the way to go. There are other
alternatives.

Regards,

--
Martín Marqués                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


Re: error on online backup using pg_basebackup tool

От
David Steele
Дата:
On 9/3/15 5:07 PM, Joseph Kregloh wrote:
> You can always move the folder out of there. Depending on the version of
> PostgreSQL different steps need to be take. If I remember correctly for
> version 9.3 you can move the symlink to point to the new directory. Pre
> 9.3 you need to move the symlink and the pointer inside the pg tables.

You can move the directory and modify the symlink on 9.2 or later
without any additional complications.

However - and this is very important - PostgreSQL must be shut down
during this operation.

My recommendation would be to move both tablespaces out of $PGDATA.
There has been a lot of discussion about disallowing it in future
releases as it causes problems for various tools.

--
-David
david@pgmasters.net


Вложения