Обсуждение: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

Поиск
Список
Период
Сортировка

Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Andres Freund
Дата:
Hi,

When testing EXEC_BACKEND on linux I see occasional test failures as long as I
don't disable ASLR. There's a code comment to that effect:

     * If testing EXEC_BACKEND on Linux, you should run this as root before
     * starting the postmaster:
     *
     * echo 0 >/proc/sys/kernel/randomize_va_space

but I don't like doing that on a system wide basis.

Linux allows disabling ASLR on a per-process basis using
personality(ADDR_NO_RANDOMIZE). There's a wrapper binary to do that as well,
setarch --addr-no-randomize.

I was wondering if we should have postmaster do personality(ADDR_NO_RANDOMIZE)
for EXEC_BACKEND builds? It seems nicer to make it automatically work than
have people remember that they need to call "setarch --addr-no-randomize make check".

Not that it actually matters for EXEC_BACKEND, but theoretically doing
personality(ADDR_NO_RANDOMIZE) in postmaster is a tad more secure than doing
it via setarch, as in the personality() case postmaster's layout itself is
still randomized...


Or perhaps we should just add a comment mentioning setarch.

Greetings,

Andres Freund



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Andrew Dunstan
Дата:
On 8/5/21 11:29 PM, Andres Freund wrote:
> Hi,
>
> When testing EXEC_BACKEND on linux I see occasional test failures as long as I
> don't disable ASLR. There's a code comment to that effect:
>
>      * If testing EXEC_BACKEND on Linux, you should run this as root before
>      * starting the postmaster:
>      *
>      * echo 0 >/proc/sys/kernel/randomize_va_space
>
> but I don't like doing that on a system wide basis.
>
> Linux allows disabling ASLR on a per-process basis using
> personality(ADDR_NO_RANDOMIZE). There's a wrapper binary to do that as well,
> setarch --addr-no-randomize.
>
> I was wondering if we should have postmaster do personality(ADDR_NO_RANDOMIZE)
> for EXEC_BACKEND builds? It seems nicer to make it automatically work than
> have people remember that they need to call "setarch --addr-no-randomize make check".
>
> Not that it actually matters for EXEC_BACKEND, but theoretically doing
> personality(ADDR_NO_RANDOMIZE) in postmaster is a tad more secure than doing
> it via setarch, as in the personality() case postmaster's layout itself is
> still randomized...
>
>
> Or perhaps we should just add a comment mentioning setarch.
>

If we can set it conveniently then that seems worth doing.


(Thinks: do we have non-Windows buildfarm members doing EXEC_BACKEND?)


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com




Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Alvaro Herrera
Дата:
On 2021-Aug-06, Andrew Dunstan wrote:

> On 8/5/21 11:29 PM, Andres Freund wrote:

> > I was wondering if we should have postmaster do personality(ADDR_NO_RANDOMIZE)
> > for EXEC_BACKEND builds? It seems nicer to make it automatically work than
> > have people remember that they need to call "setarch --addr-no-randomize make check".

How common is to get a failure?  I know I've run tests under
EXEC_BACKEND and not seen any failures.  Not many runs though.

> > Not that it actually matters for EXEC_BACKEND, but theoretically doing
> > personality(ADDR_NO_RANDOMIZE) in postmaster is a tad more secure than doing
> > it via setarch, as in the personality() case postmaster's layout itself is
> > still randomized...

True.  I think the security aspect is not critically important, since
hopefully nobody should be using such builds for production.


> (Thinks: do we have non-Windows buildfarm members doing EXEC_BACKEND?)

culicidae does that.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Pido que me den el Nobel por razones humanitarias" (Nicanor Parra)



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Robert Haas
Дата:
On Mon, Aug 9, 2021 at 1:30 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> How common is to get a failure?  I know I've run tests under
> EXEC_BACKEND and not seen any failures.  Not many runs though.

On macOS, failures are extremely common. Sometimes I have to run
simple tests many times to get even one success. The proposal on the
table won't help with that problem since it's Linux-specific, but if
there's any way to do something similar on macOS it would be a _huge_
help.

-- 
Robert Haas
EDB: http://www.enterprisedb.com



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Andres Freund
Дата:
Hi,

On 2021-08-09 13:43:03 -0400, Robert Haas wrote:
> On Mon, Aug 9, 2021 at 1:30 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > How common is to get a failure?  I know I've run tests under
> > EXEC_BACKEND and not seen any failures.  Not many runs though.

I get check-world failures in about 1/2-1/3 of the runs, and a plain check
fails in maybe 1/4 of the cases. It's pretty annoying because it often isn't
trivial to distinguish whether I've broken something or whether it's
randomization related...

The frequency of failures isn't stable over time, making it a bit harder to
know what's going on.


> On macOS, failures are extremely common. Sometimes I have to run
> simple tests many times to get even one success. The proposal on the
> table won't help with that problem since it's Linux-specific, but if
> there's any way to do something similar on macOS it would be a _huge_
> help.

There does seem to be a way of doing so, because I found blog posts talking
about how to get e.g. lldb to *enable* ASLR, which it disables by default...

Greetings,

Andres Freund



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Tom Lane
Дата:
Andres Freund <andres@anarazel.de> writes:
> On 2021-08-09 13:43:03 -0400, Robert Haas wrote:
>> On Mon, Aug 9, 2021 at 1:30 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> How common is to get a failure?  I know I've run tests under
> EXEC_BACKEND and not seen any failures.  Not many runs though.

> I get check-world failures in about 1/2-1/3 of the runs, and a plain check
> fails in maybe 1/4 of the cases. It's pretty annoying because it often isn't
> trivial to distinguish whether I've broken something or whether it's
> randomization related...

I don't have numbers, but I do know that on Linux EXEC_BACKEND builds fail
often enough to be annoying if you don't disable ASLR.  If we can do
something not-too-invasive about that, it'd be great.

            regards, tom lane



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Andres Freund
Дата:
Hi,

On 2021-08-09 13:54:25 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2021-08-09 13:43:03 -0400, Robert Haas wrote:
> >> On Mon, Aug 9, 2021 at 1:30 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > How common is to get a failure?  I know I've run tests under
> > EXEC_BACKEND and not seen any failures.  Not many runs though.
> 
> > I get check-world failures in about 1/2-1/3 of the runs, and a plain check
> > fails in maybe 1/4 of the cases. It's pretty annoying because it often isn't
> > trivial to distinguish whether I've broken something or whether it's
> > randomization related...
> 
> I don't have numbers, but I do know that on Linux EXEC_BACKEND builds fail
> often enough to be annoying if you don't disable ASLR.  If we can do
> something not-too-invasive about that, it'd be great.

I now not sure if personality(NO_RANDOMIZE) in postmaster is actually
sufficient. It does seem to drastically reduce the frequency of issues, but
there's still conceivable failures where postmaster's layout would class with
the non-randomized children - obviously personality(NO_RANDOMIZE) cannot
retroactively change the layout of the already running binary.

So maybe we should put it into pg_ctl?

Greetings,

Andres Freund



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Tue, Aug 10, 2021 at 5:43 AM Robert Haas <robertmhaas@gmail.com> wrote:
> On Mon, Aug 9, 2021 at 1:30 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > How common is to get a failure?  I know I've run tests under
> > EXEC_BACKEND and not seen any failures.  Not many runs though.
>
> On macOS, failures are extremely common. Sometimes I have to run
> simple tests many times to get even one success. The proposal on the
> table won't help with that problem since it's Linux-specific, but if
> there's any way to do something similar on macOS it would be a _huge_
> help.

Yeah, make check always fails for me on macOS 11.  With the attached
experimental hack, it fails only occasionally (1 in 8 runs or so).  I
don't know why.

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
"Andres Freund"
Дата:
Hi,

On Tue, Aug 10, 2021, at 15:19, Thomas Munro wrote:
> On Tue, Aug 10, 2021 at 5:43 AM Robert Haas <robertmhaas@gmail.com> wrote:
> > On Mon, Aug 9, 2021 at 1:30 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > > How common is to get a failure?  I know I've run tests under
> > > EXEC_BACKEND and not seen any failures.  Not many runs though.
> >
> > On macOS, failures are extremely common. Sometimes I have to run
> > simple tests many times to get even one success. The proposal on the
> > table won't help with that problem since it's Linux-specific, but if
> > there's any way to do something similar on macOS it would be a _huge_
> > help.
> 
> Yeah, make check always fails for me on macOS 11.  With the attached
> experimental hack, it fails only occasionally (1 in 8 runs or so).  I
> don't know why.

I suspect you'd need to use the hack in pg_ctl to make it reliable. The layout of normally stayed position independent
postmastercan be incompatible with the non ASLR spawned child.
 

Andres




Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Wed, Aug 11, 2021 at 2:12 AM Andres Freund <andres@anarazel.de> wrote:
> On Tue, Aug 10, 2021, at 15:19, Thomas Munro wrote:
> > Yeah, make check always fails for me on macOS 11.  With the attached
> > experimental hack, it fails only occasionally (1 in 8 runs or so).  I
> > don't know why.
>
> I suspect you'd need to use the hack in pg_ctl to make it reliable. The layout of normally stayed position
independentpostmaster can be incompatible with the non ASLR spawned child.
 

Yeah, but the patch already changes both pg_ctl.c and postmaster.c.



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Wed, Aug 11, 2021 at 7:07 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Wed, Aug 11, 2021 at 2:12 AM Andres Freund <andres@anarazel.de> wrote:
> > On Tue, Aug 10, 2021, at 15:19, Thomas Munro wrote:
> > > Yeah, make check always fails for me on macOS 11.  With the attached
> > > experimental hack, it fails only occasionally (1 in 8 runs or so).  I
> > > don't know why.
> >
> > I suspect you'd need to use the hack in pg_ctl to make it reliable. The layout of normally stayed position
independentpostmaster can be incompatible with the non ASLR spawned child.
 
>
> Yeah, but the patch already changes both pg_ctl.c and postmaster.c.

/me stares at vmmap output for a while

Oooh. It's working perfectly (for example if you export
PATH=binarys:$PATH, pg_ctl -D pgdata start, make installcheck), but
pg_regress.c has its own separate fork/exec to launch the temporary
cluster that needs to be similarly hacked.  Unfortunately I have to
give this Macintosh back and go and do some real work on a different
computer now.  That does seem to be a working solution to the problem,
though, and could be polished into proposable form.

I saw claims that you can also link with -Wl,-no_pie or toggle the PIE
bit on your executable and libraries, but that didn't work for me on
11, Intel (no effect) or ARM (linker option gone).



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Andrew Dunstan
Дата:
On 8/10/21 7:59 PM, Thomas Munro wrote:
> On Wed, Aug 11, 2021 at 7:07 AM Thomas Munro <thomas.munro@gmail.com> wrote:
>> On Wed, Aug 11, 2021 at 2:12 AM Andres Freund <andres@anarazel.de> wrote:
>>> On Tue, Aug 10, 2021, at 15:19, Thomas Munro wrote:
>>>> Yeah, make check always fails for me on macOS 11.  With the attached
>>>> experimental hack, it fails only occasionally (1 in 8 runs or so).  I
>>>> don't know why.
>>> I suspect you'd need to use the hack in pg_ctl to make it reliable. The layout of normally stayed position
independentpostmaster can be incompatible with the non ASLR spawned child.
 
>> Yeah, but the patch already changes both pg_ctl.c and postmaster.c.
> /me stares at vmmap output for a while
>
> Oooh. It's working perfectly (for example if you export
> PATH=binarys:$PATH, pg_ctl -D pgdata start, make installcheck), but
> pg_regress.c has its own separate fork/exec to launch the temporary
> cluster that needs to be similarly hacked.  


initdb also runs postgres a bunch of times via system(), similarly to
pg_regress but without the "exec". Does it also need adjusting?


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com




Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Robert Haas
Дата:
On Tue, Aug 10, 2021 at 7:59 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I saw claims that you can also link with -Wl,-no_pie or toggle the PIE
> bit on your executable and libraries, but that didn't work for me on
> 11, Intel (no effect) or ARM (linker option gone).

I think that worked for me on older macOS releases, and then it
stopped working at some point after some update or reinstall or
something. Unfortunately I don't know any more precisely than that,
but it does seem like we have to find some other approach to work on
modern systems.

Hopefully they don't keep whacking this around...

-- 
Robert Haas
EDB: http://www.enterprisedb.com



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Tom Lane
Дата:
Andrew Dunstan <andrew@dunslane.net> writes:
> initdb also runs postgres a bunch of times via system(), similarly to
> pg_regress but without the "exec". Does it also need adjusting?

That shouldn't be an issue, because we're only running the single
"standalone backend" process, not a cluster.

            regards, tom lane



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Thu, Aug 12, 2021 at 1:45 AM Robert Haas <robertmhaas@gmail.com> wrote:
> I think that worked for me on older macOS releases, and then it
> stopped working at some point after some update or reinstall or
> something. Unfortunately I don't know any more precisely than that,
> but it does seem like we have to find some other approach to work on
> modern systems.

I gave up on trying to make that work once I realised that
/usr/lib/dyld doesn't seem to obey the flag, so although other
segments become deterministic and the success rate is fairly high,
there's still a 600kb wrecking ball swinging around.  I wondered what
the "slide" range could be... it appears to be fairly small
(vm_map_get_max_aslr_slide_pages() seems to be the place that's
determined and it's a 16MB or 256MB window, depending on architecture,
if I read that right).  Given that, the death of 32 bit processes
since Catalina, and the typical layout we see, I think just doing
something like (/me rolls dice) export PG_SHMEM_ADDR=0x80000000000 is
a good candidate for something that works on both architectures, being
many TB away from everything else (above everything on ARM, between
heap etc and libs on Intel but with 8TB of space below it and 120TB
above).  That gets the tests passing consistently with unpatched
master, -DEXEC_BACKEND, on both flavours of silicon.



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Robert Haas
Дата:
On Wed, Aug 11, 2021 at 6:24 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Thu, Aug 12, 2021 at 1:45 AM Robert Haas <robertmhaas@gmail.com> wrote:
> > I think that worked for me on older macOS releases, and then it
> > stopped working at some point after some update or reinstall or
> > something. Unfortunately I don't know any more precisely than that,
> > but it does seem like we have to find some other approach to work on
> > modern systems.
>
> I gave up on trying to make that work once I realised that
> /usr/lib/dyld doesn't seem to obey the flag, so although other
> segments become deterministic and the success rate is fairly high,
> there's still a 600kb wrecking ball swinging around.  I wondered what
> the "slide" range could be... it appears to be fairly small
> (vm_map_get_max_aslr_slide_pages() seems to be the place that's
> determined and it's a 16MB or 256MB window, depending on architecture,
> if I read that right).  Given that, the death of 32 bit processes
> since Catalina, and the typical layout we see, I think just doing
> something like (/me rolls dice) export PG_SHMEM_ADDR=0x80000000000 is
> a good candidate for something that works on both architectures, being
> many TB away from everything else (above everything on ARM, between
> heap etc and libs on Intel but with 8TB of space below it and 120TB
> above).  That gets the tests passing consistently with unpatched
> master, -DEXEC_BACKEND, on both flavours of silicon.

Ugh, OK. So, is there a way that we can get an "easy button" committed
to the tree?

-- 
Robert Haas
EDB: http://www.enterprisedb.com



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Tom Lane
Дата:
Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Aug 11, 2021 at 6:24 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>> ... I think just doing
>> something like (/me rolls dice) export PG_SHMEM_ADDR=0x80000000000 is
>> a good candidate for something that works on both architectures, being
>> many TB away from everything else (above everything on ARM, between
>> heap etc and libs on Intel but with 8TB of space below it and 120TB
>> above).  That gets the tests passing consistently with unpatched
>> master, -DEXEC_BACKEND, on both flavours of silicon.

> Ugh, OK. So, is there a way that we can get an "easy button" committed
> to the tree?

I don't see why that approach couldn't be incorporated into pg_ctl,
or the postmaster itself.  Given Andres' point that Linux ASLR
disable probably has to happen in pg_ctl, it seems like doing it
in pg_ctl in all cases is the way to move forward.

            regards, tom lane



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Fri, Aug 13, 2021 at 3:13 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > Ugh, OK. So, is there a way that we can get an "easy button" committed
> > to the tree?
>
> I don't see why that approach couldn't be incorporated into pg_ctl,
> or the postmaster itself.  Given Andres' point that Linux ASLR
> disable probably has to happen in pg_ctl, it seems like doing it
> in pg_ctl in all cases is the way to move forward.

I think doing it in the postmaster is best, since otherwise you have
to put code into pg_regress.c and pg_ctl.c.  Here's a patch like that.

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Tom Lane
Дата:
Thomas Munro <thomas.munro@gmail.com> writes:
> On Fri, Aug 13, 2021 at 3:13 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I don't see why that approach couldn't be incorporated into pg_ctl,
>> or the postmaster itself.  Given Andres' point that Linux ASLR
>> disable probably has to happen in pg_ctl, it seems like doing it
>> in pg_ctl in all cases is the way to move forward.

> I think doing it in the postmaster is best, since otherwise you have
> to put code into pg_regress.c and pg_ctl.c.  Here's a patch like that.

Hmm, ok.  Small thought: it might be better to put the #if inside
the "else { .... }".  That way it scales easily to allow other
platform-specific defaults if we find anything useful.  As-is,
the obvious extension would end up with multiple else-blocks,
which seems likely to confuse pgindent if nothing else.

            regards, tom lane



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Fri, Aug 13, 2021 at 9:59 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Hmm, ok.  Small thought: it might be better to put the #if inside
> the "else { .... }".  That way it scales easily to allow other
> platform-specific defaults if we find anything useful.  As-is,
> the obvious extension would end up with multiple else-blocks,
> which seems likely to confuse pgindent if nothing else.

True.  Thanks.  Pushed to all live branches.

Obviously the address may have to be adjusted if Apple moves stuff
around, or if the initial layout and ASLR slide range turn out to be
less constrained than I inferred by nosing around the source code and
testing on a couple of systems.



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Tom Lane
Дата:
Thomas Munro <thomas.munro@gmail.com> writes:
> Obviously the address may have to be adjusted if Apple moves stuff
> around, or if the initial layout and ASLR slide range turn out to be
> less constrained than I inferred by nosing around the source code and
> testing on a couple of systems.

Sure.  But we're no worse off than before; the workaround of "set
PG_SHMEM_ADDR by hand" is just as applicable as ever.

            regards, tom lane



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
Here's a patch for Linux and also FreeBSD.  The latter OS decided to
turn on ASLR by default recently, causing my workstation to fail like
this quite reliably, which reminded me to follow up with this.  It
disables ASLR in pg_ctl and pg_regress, which is enough for check and
check-world, but doesn't help you if you run the server directly
(unlike the different hack done for macOS).

For whatever random reason the failures are rarer on Linux (could be
my imagination, but I think they might be clustered, I didn't look
into the recipe for the randomness), but even without reproducing a
failure it's clear to see using pmap that this has the right effect.
I didn't bother with a check for the existence of ADDR_NO_RANDOMIZE
because it's since 2.6.12 which is definitely ancient enough.

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
"Bossart, Nathan"
Дата:
On 11/23/21, 11:29 AM, "Thomas Munro" <thomas.munro@gmail.com> wrote:
> Here's a patch for Linux and also FreeBSD.  The latter OS decided to
> turn on ASLR by default recently, causing my workstation to fail like
> this quite reliably, which reminded me to follow up with this.  It
> disables ASLR in pg_ctl and pg_regress, which is enough for check and
> check-world, but doesn't help you if you run the server directly
> (unlike the different hack done for macOS).
>
> For whatever random reason the failures are rarer on Linux (could be
> my imagination, but I think they might be clustered, I didn't look
> into the recipe for the randomness), but even without reproducing a
> failure it's clear to see using pmap that this has the right effect.
> I didn't bother with a check for the existence of ADDR_NO_RANDOMIZE
> because it's since 2.6.12 which is definitely ancient enough.

FWIW I just found this patch very useful for testing some EXEC_BACKEND
stuff on Linux.

Nathan


Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Sat, Jan 8, 2022 at 9:20 AM Bossart, Nathan <bossartn@amazon.com> wrote:
> FWIW I just found this patch very useful for testing some EXEC_BACKEND
> stuff on Linux.

Thanks for testing.  Tidied and pushed, to master only for now.



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Michael Paquier
Дата:
On Tue, Jan 11, 2022 at 12:29:44AM +1300, Thomas Munro wrote:
> Thanks for testing.  Tidied and pushed, to master only for now.

I have noticed the following failure for v11~14 on one of my hosts
that compiles with -DEXEC_BACKEND, and Nathan has redirected me here:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=gokiburi&dt=2023-01-31%2012%3A07%3A32
FATAL:  could not reattach to shared memory (key=1050468, addr=0xffff97eb2000): Invalid argument

Could it be worth back-patching f3e7806?  I don't mind changing this
animal setup by switching the kernel configuration or reducing the
branch scope, but this solution is less invasive because it would not
influence parallel runs.

Thoughts?
--
Michael

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Tom Lane
Дата:
Michael Paquier <michael@paquier.xyz> writes:
> Could it be worth back-patching f3e7806?

That's aged long enough now that it seems like a pretty safe
thing to do.

            regards, tom lane



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Michael Paquier
Дата:
On Tue, Jan 31, 2023 at 08:37:29PM -0500, Tom Lane wrote:
> That's aged long enough now that it seems like a pretty safe
> thing to do.

Thanks.  I'll wait for a few days before doing something for my
buildfarm stuff, in case somebody thinks this is a bad idea..
--
Michael

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Wed, Feb 1, 2023 at 2:58 PM Michael Paquier <michael@paquier.xyz> wrote:
> On Tue, Jan 31, 2023 at 08:37:29PM -0500, Tom Lane wrote:
> > That's aged long enough now that it seems like a pretty safe
> > thing to do.
>
> Thanks.  I'll wait for a few days before doing something for my
> buildfarm stuff, in case somebody thinks this is a bad idea..

+1, go for it.  It shouldn't affect Unix build releases, and on
Windows the function does nothing.



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Michael Paquier
Дата:
On Wed, Feb 01, 2023 at 03:06:16PM +1300, Thomas Munro wrote:
> +1, go for it.  It shouldn't affect Unix build releases, and on
> Windows the function does nothing.

Thanks for confirming.  I am wondering what these animals may complain
about next, but based on some tests on this buildfarm host with the
same configuration, things are looking OK once this stuff is applied
on 11~14.
--
Michael

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Michael Paquier
Дата:
On Thu, Feb 02, 2023 at 10:06:15AM +0900, Michael Paquier wrote:
> Thanks for confirming.  I am wondering what these animals may complain
> about next, but based on some tests on this buildfarm host with the
> same configuration, things are looking OK once this stuff is applied
> on 11~14.

Actually, I completely forgot to take into account that there is a
minor release planned for next week:
https://www.postgresql.org/developer/roadmap/

So I'll hold on a bit longer here, until the next versions get their
tags.
--
Michael

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Michael Paquier
Дата:
On Thu, Feb 02, 2023 at 11:14:39AM +0900, Michael Paquier wrote:
> Actually, I completely forgot to take into account that there is a
> minor release planned for next week:
> https://www.postgresql.org/developer/roadmap/
>
> So I'll hold on a bit longer here, until the next versions get their
> tags.

So, the backpatch down to 12 is straight-forward, with some conflicts
in ./configure.  ~12 handles also differently its CFLAGS with
pg_config.h.win32.  11 is more annoying because it lacks
HAVE_SYS_PROCCTL_H and it would need a partial backport of f98b847.  I
am not completely sure if this could have more side effects, though,
even if the part of pmsignal.c is left out.  It should not..

At the end, I have just done this stuff down to ~12, 11 does not seem
worth the trouble as the next stable version to go out of support.
I'll reduce gokiburi's script a bit, as a result, until the oldest
version support is v12.
--
Michael

Вложения

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Andres Freund
Дата:
On 2023-02-08 13:12:15 +0900, Michael Paquier wrote:
> On Thu, Feb 02, 2023 at 11:14:39AM +0900, Michael Paquier wrote:
> > Actually, I completely forgot to take into account that there is a
> > minor release planned for next week:
> > https://www.postgresql.org/developer/roadmap/
> > 
> > So I'll hold on a bit longer here, until the next versions get their
> > tags.
> 
> So, the backpatch down to 12 is straight-forward, with some conflicts
> in ./configure.  ~12 handles also differently its CFLAGS with
> pg_config.h.win32.  11 is more annoying because it lacks
> HAVE_SYS_PROCCTL_H and it would need a partial backport of f98b847.  I
> am not completely sure if this could have more side effects, though,
> even if the part of pmsignal.c is left out.  It should not..
> 
> At the end, I have just done this stuff down to ~12, 11 does not seem
> worth the trouble as the next stable version to go out of support.
> I'll reduce gokiburi's script a bit, as a result, until the oldest
> version support is v12.

That seems reasonable to me.



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Thomas Munro
Дата:
On Wed, Feb 8, 2023 at 5:12 PM Michael Paquier <michael@paquier.xyz> wrote:
> At the end, I have just done this stuff down to ~12, 11 does not seem
> worth the trouble as the next stable version to go out of support.
> I'll reduce gokiburi's script a bit, as a result, until the oldest
> version support is v12.

For the record, according to [1] it's not necessary to use
--reset-author when back-patching.  (Maybe a little confusingly,
because it's not quite clear whether our policies consider the author
field to be meaningful or not.)

[1] https://www.postgresql.org/message-id/flat/1347459696.16215.11.camel%40vanquo.pezone.net



Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

От
Michael Paquier
Дата:
On Fri, Feb 10, 2023 at 08:15:21AM +1300, Thomas Munro wrote:
> For the record, according to [1] it's not necessary to use
> --reset-author when back-patching.  (Maybe a little confusingly,
> because it's not quite clear whether our policies consider the author
> field to be meaningful or not.)
>
> [1] https://www.postgresql.org/message-id/flat/1347459696.16215.11.camel%40vanquo.pezone.net

Oops, sorry about that.  Using --reset-author is a habit when it comes
to backpatch.  It looks like my mistake when back-patching something
only to stable branches.
--
Michael

Вложения