Обсуждение: Another pg_dump using split and gzip for large databases
The following documentation comment has been logged on the website: Page: https://www.postgresql.org/docs/10/backup-dump.html Description: pg_dump ccbank | split -b 2048m --filter='gzip > $FILE.gz'
On Sat, Jul 17, 2021 at 03:09:52PM +0000, PG Doc comments form wrote: > The following documentation comment has been logged on the website: > > Page: https://www.postgresql.org/docs/10/backup-dump.html > Description: > > pg_dump ccbank | split -b 2048m --filter='gzip > $FILE.gz' Wow, odd we didn't have an example of using gzip and split together; patch attached. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Вложения
They key to my comment is splits —filter command which allows gzip each of the split files
On Wed, Jul 28, 2021 at 18:46 Bruce Momjian <bruce@momjian.us> wrote:
On Sat, Jul 17, 2021 at 03:09:52PM +0000, PG Doc comments form wrote:
> The following documentation comment has been logged on the website:
>
> Page: https://www.postgresql.org/docs/10/backup-dump.html
> Description:
>
> pg_dump ccbank | split -b 2048m --filter='gzip > $FILE.gz'
Wow, odd we didn't have an example of using gzip and split together;
patch attached.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
James M Doherty President Revive Consulting LLC
> On 29 Jul 2021, at 01:46, Bruce Momjian <bruce@momjian.us> wrote: > On Sat, Jul 17, 2021 at 03:09:52PM +0000, PG Doc comments form wrote: >> pg_dump ccbank | split -b 2048m --filter='gzip > $FILE.gz' > > Wow, odd we didn't have an example of using gzip and split together; > patch attached. Maybe because the --filter option is only available in GNU split and not in POSIX, and thus not on FreeBSD and macOS etc. I'm not sure about what level of compatibility we strive for in the examples though. -- Daniel Gustafsson https://vmware.com/
Good point.
On Thu, Jul 29, 2021 at 04:27 Daniel Gustafsson <daniel@yesql.se> wrote:
> On 29 Jul 2021, at 01:46, Bruce Momjian <bruce@momjian.us> wrote:
> On Sat, Jul 17, 2021 at 03:09:52PM +0000, PG Doc comments form wrote:
>> pg_dump ccbank | split -b 2048m --filter='gzip > $FILE.gz'
>
> Wow, odd we didn't have an example of using gzip and split together;
> patch attached.
Maybe because the --filter option is only available in GNU split and not in
POSIX, and thus not on FreeBSD and macOS etc. I'm not sure about what level of
compatibility we strive for in the examples though.
--
Daniel Gustafsson https://vmware.com/
James M Doherty President Revive Consulting LLC
On Thu, Jul 29, 2021 at 11:27:20AM +0200, Daniel Gustafsson wrote: > > On 29 Jul 2021, at 01:46, Bruce Momjian <bruce@momjian.us> wrote: > > On Sat, Jul 17, 2021 at 03:09:52PM +0000, PG Doc comments form wrote: > > >> pg_dump ccbank | split -b 2048m --filter='gzip > $FILE.gz' > > > > Wow, odd we didn't have an example of using gzip and split together; > > patch attached. > > Maybe because the --filter option is only available in GNU split and not in > POSIX, and thus not on FreeBSD and macOS etc. I'm not sure about what level of > compatibility we strive for in the examples though. OK, I can adjust it to say "GNU split" or just skip this addition. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
> On 29 Jul 2021, at 16:58, Bruce Momjian <bruce@momjian.us> wrote: > OK, I can adjust it to say "GNU split" or just skip this addition. I don't have strong feelings either way, with a slight preference for writing it as "GNU split" since it's still a good tip for a large amount of readers. -- Daniel Gustafsson https://vmware.com/
On Thu, Jul 29, 2021 at 05:23:20PM +0200, Daniel Gustafsson wrote: > > On 29 Jul 2021, at 16:58, Bruce Momjian <bruce@momjian.us> wrote: > > > OK, I can adjust it to say "GNU split" or just skip this addition. > > I don't have strong feelings either way, with a slight preference for writing > it as "GNU split" since it's still a good tip for a large amount of readers. Yes, I am on the fence for the same reasons but lean toward moving forward for your same reasons. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Вложения
> On 29 Jul 2021, at 17:29, Bruce Momjian <bruce@momjian.us> wrote: > > On Thu, Jul 29, 2021 at 05:23:20PM +0200, Daniel Gustafsson wrote: >>> On 29 Jul 2021, at 16:58, Bruce Momjian <bruce@momjian.us> wrote: >> >>> OK, I can adjust it to say "GNU split" or just skip this addition. >> >> I don't have strong feelings either way, with a slight preference for writing >> it as "GNU split" since it's still a good tip for a large amount of readers. > > Yes, I am on the fence for the same reasons but lean toward moving > forward for your same reasons. Making it two separate examples is a good idea, tipping the scale to a +1 from me. + If using <application>GNU split</application>, it is possible to A small nitpick is that we write this as GNU <application>..</application> elsewhere on this page (like GNU tar for example). -- Daniel Gustafsson https://vmware.com/
On Thu, Jul 29, 2021 at 07:15:21PM +0200, Daniel Gustafsson wrote: > > On 29 Jul 2021, at 17:29, Bruce Momjian <bruce@momjian.us> wrote: > > > > On Thu, Jul 29, 2021 at 05:23:20PM +0200, Daniel Gustafsson wrote: > >>> On 29 Jul 2021, at 16:58, Bruce Momjian <bruce@momjian.us> wrote: > >> > >>> OK, I can adjust it to say "GNU split" or just skip this addition. > >> > >> I don't have strong feelings either way, with a slight preference for writing > >> it as "GNU split" since it's still a good tip for a large amount of readers. > > > > Yes, I am on the fence for the same reasons but lean toward moving > > forward for your same reasons. > > Making it two separate examples is a good idea, tipping the scale to a +1 from me. > > + If using <application>GNU split</application>, it is possible to > A small nitpick is that we write this as GNU <application>..</application> > elsewhere on this page (like GNU tar for example). OK, updated patch attached. FYI, I put the GNU inside the application name in the first version because we specifically only want the GNU version of that application, but I think taking it out for consistency is fine. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Вложения
> On 29 Jul 2021, at 21:10, Bruce Momjian <bruce@momjian.us> wrote: > > On Thu, Jul 29, 2021 at 07:15:21PM +0200, Daniel Gustafsson wrote: >>> On 29 Jul 2021, at 17:29, Bruce Momjian <bruce@momjian.us> wrote: >>> >>> On Thu, Jul 29, 2021 at 05:23:20PM +0200, Daniel Gustafsson wrote: >>>>> On 29 Jul 2021, at 16:58, Bruce Momjian <bruce@momjian.us> wrote: >>>> >>>>> OK, I can adjust it to say "GNU split" or just skip this addition. >>>> >>>> I don't have strong feelings either way, with a slight preference for writing >>>> it as "GNU split" since it's still a good tip for a large amount of readers. >>> >>> Yes, I am on the fence for the same reasons but lean toward moving >>> forward for your same reasons. >> >> Making it two separate examples is a good idea, tipping the scale to a +1 from me. >> >> + If using <application>GNU split</application>, it is possible to >> A small nitpick is that we write this as GNU <application>..</application> >> elsewhere on this page (like GNU tar for example). > > OK, updated patch attached. FYI, I put the GNU inside the application > name in the first version because we specifically only want the GNU > version of that application, but I think taking it out for consistency > is fine. LGTM -- Daniel Gustafsson https://vmware.com/
On Thu, Jul 29, 2021 at 03:10:34PM -0400, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 07:15:21PM +0200, Daniel Gustafsson wrote: > > > On 29 Jul 2021, at 17:29, Bruce Momjian <bruce@momjian.us> wrote: > > > > > > On Thu, Jul 29, 2021 at 05:23:20PM +0200, Daniel Gustafsson wrote: > > >>> On 29 Jul 2021, at 16:58, Bruce Momjian <bruce@momjian.us> wrote: > > >> > > >>> OK, I can adjust it to say "GNU split" or just skip this addition. > > >> > > >> I don't have strong feelings either way, with a slight preference for writing > > >> it as "GNU split" since it's still a good tip for a large amount of readers. > > > > > > Yes, I am on the fence for the same reasons but lean toward moving > > > forward for your same reasons. > > > > Making it two separate examples is a good idea, tipping the scale to a +1 from me. > > > > + If using <application>GNU split</application>, it is possible to > > A small nitpick is that we write this as GNU <application>..</application> > > elsewhere on this page (like GNU tar for example). > > OK, updated patch attached. FYI, I put the GNU inside the application > name in the first version because we specifically only want the GNU > version of that application, but I think taking it out for consistency > is fine. Patch applied to all supported versions. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.