Обсуждение: pg_export_snapshot on standby side
Hi, We cannot run parallel pg_dump on the standby server because pg_export_snapshot() always fails on the standby. Is this the oversight of parallel pg_dump or pg_export_snapshot()? pg_export_snapshot() fails in the standby because it always assigns new XID and which is not allowed in the standby. Do we really need to assign new XID even in the standby for the exportable snapshot? Regards, -- Fujii Masao
On 21 May 2013 19:16, Fujii Masao <masao.fujii@gmail.com> wrote: > We cannot run parallel pg_dump on the standby server because > pg_export_snapshot() always fails on the standby. Is this the oversight > of parallel pg_dump or pg_export_snapshot()? > > pg_export_snapshot() fails in the standby because it always assigns > new XID and which is not allowed in the standby. Do we really need > to assign new XID even in the standby for the exportable snapshot? Having looked at the code, I say No, we don't *need* to. There are various parts of the code that deal with takenDuringRecovery, so much of this was clearly intended to work in recovery. We use the topXid for the name of the snapshot file. That is clearly unnecessary and we should be using the virtualxid instead like we do elsewhere. We also use the topXid to test whether it is still running, though again, we could equally use the virtualxid instead. There is no problem with virtualxids possibly not being active anymore, since if we didn't have an xid before and don't have one now, and the xmin is the same, the snapshot is still valid. I think we should treat this as a bug and fix it in 9.3 while we're still in beta. Why? Because once we begin using the topXid as the filename we can't then change later to using the vxid. --Simon Riggs http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On Sat, May 25, 2013 at 6:18 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > On 21 May 2013 19:16, Fujii Masao <masao.fujii@gmail.com> wrote: > >> We cannot run parallel pg_dump on the standby server because >> pg_export_snapshot() always fails on the standby. Is this the oversight >> of parallel pg_dump or pg_export_snapshot()? >> >> pg_export_snapshot() fails in the standby because it always assigns >> new XID and which is not allowed in the standby. Do we really need >> to assign new XID even in the standby for the exportable snapshot? > > Having looked at the code, I say No, we don't *need* to. Good to hear. > There are various parts of the code that deal with > takenDuringRecovery, so much of this was clearly intended to work in > recovery. > > We use the topXid for the name of the snapshot file. That is clearly > unnecessary and we should be using the virtualxid instead like we do > elsewhere. We also use the topXid to test whether it is still running, > though again, we could equally use the virtualxid instead. There is no > problem with virtualxids possibly not being active anymore, since if > we didn't have an xid before and don't have one now, and the xmin is > the same, the snapshot is still valid. > > I think we should treat this as a bug and fix it in 9.3 while we're > still in beta. +1 > Why? Because once we begin using the topXid as the > filename we can't then change later to using the vxid. I'm afraid that pg_export_snapshot() in 9.2 has already been using topXid as the filename. Regards, -- Fujii Masao
Fujii Masao escribió: > On Sat, May 25, 2013 at 6:18 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > > I think we should treat this as a bug and fix it in 9.3 while we're > > still in beta. > > +1 > > > Why? Because once we begin using the topXid as the > > filename we can't then change later to using the vxid. > > I'm afraid that pg_export_snapshot() in 9.2 has already been using > topXid as the filename. I don't think this matters at all. The filename is an implementation detail which we don't even need to keep compatible across pg_upgrade. If we think this is a bug, we should backpatch the fix to 9.2. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Uh, was this ever addressed? I don't think so. --------------------------------------------------------------------------- On Sat, May 25, 2013 at 10:18:57AM +0100, Simon Riggs wrote: > On 21 May 2013 19:16, Fujii Masao <masao.fujii@gmail.com> wrote: > > > We cannot run parallel pg_dump on the standby server because > > pg_export_snapshot() always fails on the standby. Is this the oversight > > of parallel pg_dump or pg_export_snapshot()? > > > > pg_export_snapshot() fails in the standby because it always assigns > > new XID and which is not allowed in the standby. Do we really need > > to assign new XID even in the standby for the exportable snapshot? > > Having looked at the code, I say No, we don't *need* to. > > There are various parts of the code that deal with > takenDuringRecovery, so much of this was clearly intended to work in > recovery. > > We use the topXid for the name of the snapshot file. That is clearly > unnecessary and we should be using the virtualxid instead like we do > elsewhere. We also use the topXid to test whether it is still running, > though again, we could equally use the virtualxid instead. There is no > problem with virtualxids possibly not being active anymore, since if > we didn't have an xid before and don't have one now, and the xmin is > the same, the snapshot is still valid. > > I think we should treat this as a bug and fix it in 9.3 while we're > still in beta. Why? Because once we begin using the topXid as the > filename we can't then change later to using the vxid. > > -- > Simon Riggs http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. +
On 11 January 2014 18:34, Bruce Momjian <bruce@momjian.us> wrote: > > Uh, was this ever addressed? I don't think so. It appears not. -- Simon Riggs http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Seems we still have not addressed this. --------------------------------------------------------------------------- On Sat, May 25, 2013 at 10:18:57AM +0100, Simon Riggs wrote: > On 21 May 2013 19:16, Fujii Masao <masao.fujii@gmail.com> wrote: > > > We cannot run parallel pg_dump on the standby server because > > pg_export_snapshot() always fails on the standby. Is this the oversight > > of parallel pg_dump or pg_export_snapshot()? > > > > pg_export_snapshot() fails in the standby because it always assigns > > new XID and which is not allowed in the standby. Do we really need > > to assign new XID even in the standby for the exportable snapshot? > > Having looked at the code, I say No, we don't *need* to. > > There are various parts of the code that deal with > takenDuringRecovery, so much of this was clearly intended to work in > recovery. > > We use the topXid for the name of the snapshot file. That is clearly > unnecessary and we should be using the virtualxid instead like we do > elsewhere. We also use the topXid to test whether it is still running, > though again, we could equally use the virtualxid instead. There is no > problem with virtualxids possibly not being active anymore, since if > we didn't have an xid before and don't have one now, and the xmin is > the same, the snapshot is still valid. > > I think we should treat this as a bug and fix it in 9.3 while we're > still in beta. Why? Because once we begin using the topXid as the > filename we can't then change later to using the vxid. > > -- > Simon Riggs http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. +
On Thu, Aug 7, 2014 at 2:17 AM, Bruce Momjian <bruce@momjian.us> wrote: > > Seems we still have not addressed this. Thanks for the reminder :) I'm not sure if I have time to work on this for a while. If anyone is interested in this, please feel free to try it. Regards, -- Fujii Masao
On Thu, Aug 7, 2014 at 01:26:24PM +0900, Fujii Masao wrote: > On Thu, Aug 7, 2014 at 2:17 AM, Bruce Momjian <bruce@momjian.us> wrote: > > > > Seems we still have not addressed this. > > Thanks for the reminder :) I'm not sure if I have time to work on this > for a while. If anyone is interested in this, please feel free to try it. Added to TODO: Allow pg_export_snapshot() to run on hot standby servers This will allow parallel pg_dump on such servers. pg_export_snapshoton standby side -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Roman grave inscription +