Обсуждение: WAL-G shipping to the cloud
Hi all,
I'd like to get postgres log files to the cloud but i want this process going live i want to automate test process and after that i want it go live.
What i have in my mind is setting archive_mode is on and archiving_command with a script. After that i want to execute pgbench to get more WAL files and trigger with pg_switch_wal() to change WAL files.
Is there any idea do you have about it or is there anyone shipped their log to the cloud to give me an idea?
Thanks.
On 3/18/21 2:18 PM, aslı cokay wrote: > Hi all, > > I'd like to get postgres log files to the cloud but i want this process > going live i want to automate test process and after that i want it go > live. > > What i have in my mind is setting archive_mode is on and > archiving_command with a script. After that i want to execute pgbench to > get more WAL files and trigger with pg_switch_wal() to change WAL files. > > Is there any idea do you have about it or is there anyone shipped their > log to the cloud to give me an idea? https://github.com/wal-e/wal-e > > Thanks. > -- Adrian Klaver adrian.klaver@aklaver.com
On Thu, Mar 18, 2021 at 05:18:30PM -0400, aslı cokay wrote: > Hi all, > > I'd like to get postgres log files to the cloud but i want this process going > live i want to automate test process and after that i want it go live. > > What i have in my mind is setting archive_mode is on and archiving_command with > a script. After that i want to execute pgbench to get more WAL files and > trigger with pg_switch_wal() to change WAL files. > > Is there any idea do you have about it or is there anyone shipped their log to > the cloud to give me an idea? Well, there is Wal-E and Wal-G, which archive to the cloud. pg_backreset also supports cloud WAL storage. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Thank you, Bruce. Actually I mean I am using WAL-G what I am curious about is that I want to test shipping process automate, and I am looking for the ideas. But thanks for your help.
Bruce Momjian <bruce@momjian.us>, 18 Mar 2021 Per, 17:24 tarihinde şunu yazdı:
On Thu, Mar 18, 2021 at 05:18:30PM -0400, aslı cokay wrote:
> Hi all,
>
> I'd like to get postgres log files to the cloud but i want this process going
> live i want to automate test process and after that i want it go live.
>
> What i have in my mind is setting archive_mode is on and archiving_command with
> a script. After that i want to execute pgbench to get more WAL files and
> trigger with pg_switch_wal() to change WAL files.
>
> Is there any idea do you have about it or is there anyone shipped their log to
> the cloud to give me an idea?
Well, there is Wal-E and Wal-G, which archive to the cloud.
pg_backreset also supports cloud WAL storage.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
Hi,
I'm using Wal-G like this, in pg12:
archive_mode = always
archive_command = 'wal-g wal-push %p'
archive_timeout = 60
restore_command = 'wal-g wal-fetch %f %p'
The only change for pg10 is last line (restore_command) has to go into a separate file, recovery.conf.
Please note, however, I'm using this in Kubernetes, so the relevant docker image has already got all the relevant ENV config for Wall-G loaded beforehand.
But Wal-G is really straightforward to configure, however.
Adalberto
Il giorno ven 19 mar 2021 alle ore 00:30 asli cokay <aslicokay@gmail.com> ha scritto:
Thank you, Bruce. Actually I mean I am using WAL-G what I am curious about is that I want to test shipping process automate, and I am looking for the ideas. But thanks for your help.Bruce Momjian <bruce@momjian.us>, 18 Mar 2021 Per, 17:24 tarihinde şunu yazdı:On Thu, Mar 18, 2021 at 05:18:30PM -0400, aslı cokay wrote:
> Hi all,
>
> I'd like to get postgres log files to the cloud but i want this process going
> live i want to automate test process and after that i want it go live.
>
> What i have in my mind is setting archive_mode is on and archiving_command with
> a script. After that i want to execute pgbench to get more WAL files and
> trigger with pg_switch_wal() to change WAL files.
>
> Is there any idea do you have about it or is there anyone shipped their log to
> the cloud to give me an idea?
Well, there is Wal-E and Wal-G, which archive to the cloud.
pg_backreset also supports cloud WAL storage.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
Thanks for replying. I am just curious about that why are you using archive_mode = always? In this situation WAL-G trying to load same file at the same place i think.
Thanks.
19 Mart 2021 Cuma tarihinde Adalberto Caccia <adacaccia@gmail.com> yazdı:
19 Mart 2021 Cuma tarihinde Adalberto Caccia <adacaccia@gmail.com> yazdı:
Hi,I'm using Wal-G like this, in pg12:archive_mode = always
archive_command = 'wal-g wal-push %p'
archive_timeout = 60
restore_command = 'wal-g wal-fetch %f %p'The only change for pg10 is last line (restore_command) has to go into a separate file, recovery.conf.Please note, however, I'm using this in Kubernetes, so the relevant docker image has already got all the relevant ENV config for Wall-G loaded beforehand.But Wal-G is really straightforward to configure, however.AdalbertoIl giorno ven 19 mar 2021 alle ore 00:30 asli cokay <aslicokay@gmail.com> ha scritto:Thank you, Bruce. Actually I mean I am using WAL-G what I am curious about is that I want to test shipping process automate, and I am looking for the ideas. But thanks for your help.Bruce Momjian <bruce@momjian.us>, 18 Mar 2021 Per, 17:24 tarihinde şunu yazdı:On Thu, Mar 18, 2021 at 05:18:30PM -0400, aslı cokay wrote:
> Hi all,
>
> I'd like to get postgres log files to the cloud but i want this process going
> live i want to automate test process and after that i want it go live.
>
> What i have in my mind is setting archive_mode is on and archiving_command with
> a script. After that i want to execute pgbench to get more WAL files and
> trigger with pg_switch_wal() to change WAL files.
>
> Is there any idea do you have about it or is there anyone shipped their log to
> the cloud to give me an idea?
Well, there is Wal-E and Wal-G, which archive to the cloud.
pg_backreset also supports cloud WAL storage.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.