Обсуждение: null iv parameter passed to combo_init()

Поиск
Список
Период
Сортировка

null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:
Hi,
In contrib/pgcrypto/pgcrypto.c :

    err = px_combo_init(c, (uint8 *) VARDATA_ANY(key), klen, NULL, 0);

Note: NULL is passed as iv.

When combo_init() is called,

        if (ivlen > ivs)
            memcpy(ivbuf, iv, ivs);
        else
            memcpy(ivbuf, iv, ivlen);

It seems we need to consider the case of null being passed as iv for memcpy() because of this:

/usr/include/string.h:44:28: note: nonnull attribute specified here

What do you think of the following patch ?

Cheers
Вложения

Re: null iv parameter passed to combo_init()

От
Noah Misch
Дата:
On Fri, Jan 07, 2022 at 04:32:01PM -0800, Zhihong Yu wrote:
> In contrib/pgcrypto/pgcrypto.c :
> 
>     err = px_combo_init(c, (uint8 *) VARDATA_ANY(key), klen, NULL, 0);
> 
> Note: NULL is passed as iv.
> 
> When combo_init() is called,
> 
>         if (ivlen > ivs)
>             memcpy(ivbuf, iv, ivs);
>         else
>             memcpy(ivbuf, iv, ivlen);
> 
> It seems we need to consider the case of null being passed as iv for
> memcpy() because of this:
> 
> /usr/include/string.h:44:28: note: nonnull attribute specified here

I agree it's time to fix cases like this, given
https://postgr.es/m/flat/20200904023648.GB3426768@rfd.leadboat.com.  However,
it should be one patch fixing all (or at least many) of them.

> --- a/contrib/pgcrypto/px.c
> +++ b/contrib/pgcrypto/px.c
> @@ -198,10 +198,13 @@ combo_init(PX_Combo *cx, const uint8 *key, unsigned klen,
>      if (ivs > 0)
>      {
>          ivbuf = palloc0(ivs);
> -        if (ivlen > ivs)
> -            memcpy(ivbuf, iv, ivs);
> -        else
> -            memcpy(ivbuf, iv, ivlen);
> +        if (iv != NULL)
> +        {
> +            if (ivlen > ivs)
> +                memcpy(ivbuf, iv, ivs);
> +            else
> +                memcpy(ivbuf, iv, ivlen);
> +        }
>      }

If someone were to pass NULL iv with nonzero ivlen, that will silently
malfunction.  I'd avoid that risk by writing this way:

--- a/contrib/pgcrypto/px.c
+++ b/contrib/pgcrypto/px.c
@@ -202,3 +202,3 @@ combo_init(PX_Combo *cx, const uint8 *key, unsigned klen,
             memcpy(ivbuf, iv, ivs);
-        else
+        else if (ivlen > 0)
             memcpy(ivbuf, iv, ivlen);

That also gives the compiler an additional optimization strategy.



Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Sat, Jan 8, 2022 at 5:52 PM Noah Misch <noah@leadboat.com> wrote:
On Fri, Jan 07, 2022 at 04:32:01PM -0800, Zhihong Yu wrote:
> In contrib/pgcrypto/pgcrypto.c :
>
>     err = px_combo_init(c, (uint8 *) VARDATA_ANY(key), klen, NULL, 0);
>
> Note: NULL is passed as iv.
>
> When combo_init() is called,
>
>         if (ivlen > ivs)
>             memcpy(ivbuf, iv, ivs);
>         else
>             memcpy(ivbuf, iv, ivlen);
>
> It seems we need to consider the case of null being passed as iv for
> memcpy() because of this:
>
> /usr/include/string.h:44:28: note: nonnull attribute specified here

I agree it's time to fix cases like this, given
https://postgr.es/m/flat/20200904023648.GB3426768@rfd.leadboat.com.  However,
it should be one patch fixing all (or at least many) of them.

> --- a/contrib/pgcrypto/px.c
> +++ b/contrib/pgcrypto/px.c
> @@ -198,10 +198,13 @@ combo_init(PX_Combo *cx, const uint8 *key, unsigned klen,
>       if (ivs > 0)
>       {
>               ivbuf = palloc0(ivs);
> -             if (ivlen > ivs)
> -                     memcpy(ivbuf, iv, ivs);
> -             else
> -                     memcpy(ivbuf, iv, ivlen);
> +             if (iv != NULL)
> +             {
> +                     if (ivlen > ivs)
> +                             memcpy(ivbuf, iv, ivs);
> +                     else
> +                             memcpy(ivbuf, iv, ivlen);
> +             }
>       }

If someone were to pass NULL iv with nonzero ivlen, that will silently
Hi,
If iv is NULL, none of the memcpy() would be called (based on my patch).
Can you elaborate your suggestion in more detail ?

Patch v2 is attached, covering more files.

Since the referenced email was old, line numbers have changed.
It would be nice if an up-to-date list is provided in case more places should be changed.

Cheers
 
malfunction.  I'd avoid that risk by writing this way:

--- a/contrib/pgcrypto/px.c
+++ b/contrib/pgcrypto/px.c
@@ -202,3 +202,3 @@ combo_init(PX_Combo *cx, const uint8 *key, unsigned klen,
                        memcpy(ivbuf, iv, ivs);
-               else
+               else if (ivlen > 0)
                        memcpy(ivbuf, iv, ivlen);

That also gives the compiler an additional optimization strategy.

 
Вложения

Re: null iv parameter passed to combo_init()

От
Noah Misch
Дата:
On Sat, Jan 08, 2022 at 06:52:14PM -0800, Zhihong Yu wrote:
> On Sat, Jan 8, 2022 at 5:52 PM Noah Misch <noah@leadboat.com> wrote:
> > On Fri, Jan 07, 2022 at 04:32:01PM -0800, Zhihong Yu wrote:

> > I agree it's time to fix cases like this, given
> > https://postgr.es/m/flat/20200904023648.GB3426768@rfd.leadboat.com.  However,
> > it should be one patch fixing all (or at least many) of them.

> > > --- a/contrib/pgcrypto/px.c
> > > +++ b/contrib/pgcrypto/px.c
> > > @@ -198,10 +198,13 @@ combo_init(PX_Combo *cx, const uint8 *key,
> > unsigned klen,
> > >       if (ivs > 0)
> > >       {
> > >               ivbuf = palloc0(ivs);
> > > -             if (ivlen > ivs)
> > > -                     memcpy(ivbuf, iv, ivs);
> > > -             else
> > > -                     memcpy(ivbuf, iv, ivlen);
> > > +             if (iv != NULL)
> > > +             {
> > > +                     if (ivlen > ivs)
> > > +                             memcpy(ivbuf, iv, ivs);
> > > +                     else
> > > +                             memcpy(ivbuf, iv, ivlen);
> > > +             }
> > >       }
> >
> > If someone were to pass NULL iv with nonzero ivlen, that will silently
> >
> Hi,
> If iv is NULL, none of the memcpy() would be called (based on my patch).
> Can you elaborate your suggestion in more detail ?

On further thought, I would write it this way:

--- a/contrib/pgcrypto/px.c
+++ b/contrib/pgcrypto/px.c
@@ -202,3 +202,3 @@ combo_init(PX_Combo *cx, const uint8 *key, unsigned klen,
             memcpy(ivbuf, iv, ivs);
-        else
+        else if (ivlen != 0)
             memcpy(ivbuf, iv, ivlen);

That helps in two ways.  First, if someone passes iv==NULL and ivlen!=0, my
version will tend to crash, but yours will treat that like ivlen==0.  Since
this would be a programming error, crashing is better.  Second, a compiler can
opt to omit the "ivlen != 0" test from the generated assembly, because the
compiler can know that memcpy(any_value_a, any_value_b, 0) is a no-op.

> Since the referenced email was old, line numbers have changed.
> It would be nice if an up-to-date list is provided in case more places
> should be changed.

To check whether you've gotten them all, configure with CC='gcc
-fsanitize=undefined -fsanitize-undefined-trap-on-error' and run check-world.



Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Sat, Jan 8, 2022 at 7:11 PM Noah Misch <noah@leadboat.com> wrote:
On Sat, Jan 08, 2022 at 06:52:14PM -0800, Zhihong Yu wrote:
> On Sat, Jan 8, 2022 at 5:52 PM Noah Misch <noah@leadboat.com> wrote:
> > On Fri, Jan 07, 2022 at 04:32:01PM -0800, Zhihong Yu wrote:

> > I agree it's time to fix cases like this, given
> > https://postgr.es/m/flat/20200904023648.GB3426768@rfd.leadboat.com.  However,
> > it should be one patch fixing all (or at least many) of them.

> > > --- a/contrib/pgcrypto/px.c
> > > +++ b/contrib/pgcrypto/px.c
> > > @@ -198,10 +198,13 @@ combo_init(PX_Combo *cx, const uint8 *key,
> > unsigned klen,
> > >       if (ivs > 0)
> > >       {
> > >               ivbuf = palloc0(ivs);
> > > -             if (ivlen > ivs)
> > > -                     memcpy(ivbuf, iv, ivs);
> > > -             else
> > > -                     memcpy(ivbuf, iv, ivlen);
> > > +             if (iv != NULL)
> > > +             {
> > > +                     if (ivlen > ivs)
> > > +                             memcpy(ivbuf, iv, ivs);
> > > +                     else
> > > +                             memcpy(ivbuf, iv, ivlen);
> > > +             }
> > >       }
> >
> > If someone were to pass NULL iv with nonzero ivlen, that will silently
> >
> Hi,
> If iv is NULL, none of the memcpy() would be called (based on my patch).
> Can you elaborate your suggestion in more detail ?

On further thought, I would write it this way:

--- a/contrib/pgcrypto/px.c
+++ b/contrib/pgcrypto/px.c
@@ -202,3 +202,3 @@ combo_init(PX_Combo *cx, const uint8 *key, unsigned klen,
                        memcpy(ivbuf, iv, ivs);
-               else
+               else if (ivlen != 0)
                        memcpy(ivbuf, iv, ivlen);

That helps in two ways.  First, if someone passes iv==NULL and ivlen!=0, my
version will tend to crash, but yours will treat that like ivlen==0.  Since
this would be a programming error, crashing is better.  Second, a compiler can
opt to omit the "ivlen != 0" test from the generated assembly, because the
compiler can know that memcpy(any_value_a, any_value_b, 0) is a no-op.

Hi,
Updated patch is attached.
 

> Since the referenced email was old, line numbers have changed.
> It would be nice if an up-to-date list is provided in case more places
> should be changed.

To check whether you've gotten them all, configure with CC='gcc
-fsanitize=undefined -fsanitize-undefined-trap-on-error' and run check-world.
Вложения

Re: null iv parameter passed to combo_init()

От
Tom Lane
Дата:
Noah Misch <noah@leadboat.com> writes:
> On further thought, I would write it this way:

> -        else
> +        else if (ivlen != 0)
>              memcpy(ivbuf, iv, ivlen);

FWIW, I liked the "ivlen > 0" formulation better.  They should be
equivalent, because ivlen is unsigned, but it just seems like "> 0"
is more natural.

            regards, tom lane



Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Sat, Jan 8, 2022 at 11:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Noah Misch <noah@leadboat.com> writes:
> On further thought, I would write it this way:

> -             else
> +             else if (ivlen != 0)
>                       memcpy(ivbuf, iv, ivlen);

FWIW, I liked the "ivlen > 0" formulation better.  They should be
equivalent, because ivlen is unsigned, but it just seems like "> 0"
is more natural.

                        regards, tom lane

Patch v4 is attached.

Cheers
Вложения

Re: null iv parameter passed to combo_init()

От
Noah Misch
Дата:
On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote:
> On Sat, Jan 8, 2022 at 11:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Noah Misch <noah@leadboat.com> writes:
> > > On further thought, I would write it this way:
> >
> > > -             else
> > > +             else if (ivlen != 0)
> > >                       memcpy(ivbuf, iv, ivlen);
> >
> > FWIW, I liked the "ivlen > 0" formulation better.  They should be
> > equivalent, because ivlen is unsigned, but it just seems like "> 0"
> > is more natural.

If I were considering the one code site in isolation, I'd pick "ivlen > 0".
But of the four sites identified so far, three have signed length variables.
Since we're likely to get more examples of this pattern, some signed and some
unsigned, I'd rather use a style that does the optimal thing whether or not
the variable is signed.  What do you think?

> Patch v4 is attached.

Does this pass the test procedure shown upthread?



Re: null iv parameter passed to combo_init()

От
Tom Lane
Дата:
Noah Misch <noah@leadboat.com> writes:
> On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote:
>> On Sat, Jan 8, 2022 at 11:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> FWIW, I liked the "ivlen > 0" formulation better.  They should be
>>> equivalent, because ivlen is unsigned, but it just seems like "> 0"
>>> is more natural.

> If I were considering the one code site in isolation, I'd pick "ivlen > 0".
> But of the four sites identified so far, three have signed length variables.

Oh, hmm.  Unless we want to start changing those to unsigned, I agree
a not-equal test is a safer convention.

            regards, tom lane



Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Sun, Jan 9, 2022 at 8:48 AM Noah Misch <noah@leadboat.com> wrote:
On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote:
> On Sat, Jan 8, 2022 at 11:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Noah Misch <noah@leadboat.com> writes:
> > > On further thought, I would write it this way:
> >
> > > -             else
> > > +             else if (ivlen != 0)
> > >                       memcpy(ivbuf, iv, ivlen);
> >
> > FWIW, I liked the "ivlen > 0" formulation better.  They should be
> > equivalent, because ivlen is unsigned, but it just seems like "> 0"
> > is more natural.

If I were considering the one code site in isolation, I'd pick "ivlen > 0".
But of the four sites identified so far, three have signed length variables.
Since we're likely to get more examples of this pattern, some signed and some
unsigned, I'd rather use a style that does the optimal thing whether or not
the variable is signed.  What do you think?

> Patch v4 is attached.

Does this pass the test procedure shown upthread?
Hi,
I installed gcc 4.9.3

When I ran:
./configure CFLAGS='-fsanitize=undefined -fsanitize-undefined-trap-on-error' 

I saw:

configure:3977: $? = 0
configure:3966: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3966: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3997: checking whether the C compiler works
configure:4019: gcc -fsanitize=undefined -fsanitize-undefined-trap-on-error   conftest.c  >&5
gcc: error: unrecognized command line option '-fsanitize-undefined-trap-on-error'
configure:4023: $? = 1
configure:4061: result: no

I wonder if a higher version gcc is needed.

FYI

Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Sun, Jan 9, 2022 at 12:38 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Sun, Jan 9, 2022 at 8:48 AM Noah Misch <noah@leadboat.com> wrote:
On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote:
> On Sat, Jan 8, 2022 at 11:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Noah Misch <noah@leadboat.com> writes:
> > > On further thought, I would write it this way:
> >
> > > -             else
> > > +             else if (ivlen != 0)
> > >                       memcpy(ivbuf, iv, ivlen);
> >
> > FWIW, I liked the "ivlen > 0" formulation better.  They should be
> > equivalent, because ivlen is unsigned, but it just seems like "> 0"
> > is more natural.

If I were considering the one code site in isolation, I'd pick "ivlen > 0".
But of the four sites identified so far, three have signed length variables.
Since we're likely to get more examples of this pattern, some signed and some
unsigned, I'd rather use a style that does the optimal thing whether or not
the variable is signed.  What do you think?

> Patch v4 is attached.

Does this pass the test procedure shown upthread?
Hi,
I installed gcc 4.9.3

When I ran:
./configure CFLAGS='-fsanitize=undefined -fsanitize-undefined-trap-on-error' 

I saw:

configure:3977: $? = 0
configure:3966: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3966: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3997: checking whether the C compiler works
configure:4019: gcc -fsanitize=undefined -fsanitize-undefined-trap-on-error   conftest.c  >&5
gcc: error: unrecognized command line option '-fsanitize-undefined-trap-on-error'
configure:4023: $? = 1
configure:4061: result: no

I wonder if a higher version gcc is needed.

FYI
 
After installing gcc-11, ./configure passed (with 0003-memcpy-null.patch).
In the output of `make check-world`, I don't see `runtime error`.
Though there was a crash (maybe specific to my machine):

Core was generated by `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/postgres --singl'.
Program terminated with signal SIGILL, Illegal instruction.
#0  0x000000000050642d in write_item.cold ()
Missing separate debuginfos, use: debuginfo-install glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64 sssd-client-1.16.5-10.el7_9.10.x86_64
(gdb) bt
#0  0x000000000050642d in write_item.cold ()
#1  0x0000000000ba9d1b in write_relcache_init_file ()
#2  0x0000000000bb58f7 in RelationCacheInitializePhase3 ()
#3  0x0000000000bd5cb5 in InitPostgres ()
#4  0x0000000000a0a9ea in PostgresMain () 

FYI

Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Sun, Jan 9, 2022 at 12:38 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Sun, Jan 9, 2022 at 8:48 AM Noah Misch <noah@leadboat.com> wrote:
On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote:
> On Sat, Jan 8, 2022 at 11:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Noah Misch <noah@leadboat.com> writes:
> > > On further thought, I would write it this way:
> >
> > > -             else
> > > +             else if (ivlen != 0)
> > >                       memcpy(ivbuf, iv, ivlen);
> >
> > FWIW, I liked the "ivlen > 0" formulation better.  They should be
> > equivalent, because ivlen is unsigned, but it just seems like "> 0"
> > is more natural.

If I were considering the one code site in isolation, I'd pick "ivlen > 0".
But of the four sites identified so far, three have signed length variables.
Since we're likely to get more examples of this pattern, some signed and some
unsigned, I'd rather use a style that does the optimal thing whether or not
the variable is signed.  What do you think?

> Patch v4 is attached.

Does this pass the test procedure shown upthread?
Hi,
I installed gcc 4.9.3

When I ran:
./configure CFLAGS='-fsanitize=undefined -fsanitize-undefined-trap-on-error' 

I saw:

configure:3977: $? = 0
configure:3966: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3966: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3997: checking whether the C compiler works
configure:4019: gcc -fsanitize=undefined -fsanitize-undefined-trap-on-error   conftest.c  >&5
gcc: error: unrecognized command line option '-fsanitize-undefined-trap-on-error'
configure:4023: $? = 1
configure:4061: result: no

I wonder if a higher version gcc is needed.

FYI
 
After installing gcc-11, ./configure passed (with 0003-memcpy-null.patch).
In the output of `make check-world`, I don't see `runtime error`.
Though there was a crash (maybe specific to my machine):

Core was generated by `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/postgres --singl'.
Program terminated with signal SIGILL, Illegal instruction.
#0  0x000000000050642d in write_item.cold ()
Missing separate debuginfos, use: debuginfo-install glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64 sssd-client-1.16.5-10.el7_9.10.x86_64
(gdb) bt
#0  0x000000000050642d in write_item.cold ()
#1  0x0000000000ba9d1b in write_relcache_init_file ()
#2  0x0000000000bb58f7 in RelationCacheInitializePhase3 ()
#3  0x0000000000bd5cb5 in InitPostgres ()
#4  0x0000000000a0a9ea in PostgresMain () 

FYI
Hi,
Earlier I was using devtoolset-11 which had an `Illegal instruction` error.

I compiled / installed gcc-11 from source (which took whole afternoon).
`make check-world` passed with patch v3.
In tmp_install/log/install.log, I saw:

gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
rm -f libpgport.a 

Cheers

Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Sun, Jan 9, 2022 at 6:45 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Sun, Jan 9, 2022 at 12:38 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Sun, Jan 9, 2022 at 8:48 AM Noah Misch <noah@leadboat.com> wrote:
On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote:
> On Sat, Jan 8, 2022 at 11:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Noah Misch <noah@leadboat.com> writes:
> > > On further thought, I would write it this way:
> >
> > > -             else
> > > +             else if (ivlen != 0)
> > >                       memcpy(ivbuf, iv, ivlen);
> >
> > FWIW, I liked the "ivlen > 0" formulation better.  They should be
> > equivalent, because ivlen is unsigned, but it just seems like "> 0"
> > is more natural.

If I were considering the one code site in isolation, I'd pick "ivlen > 0".
But of the four sites identified so far, three have signed length variables.
Since we're likely to get more examples of this pattern, some signed and some
unsigned, I'd rather use a style that does the optimal thing whether or not
the variable is signed.  What do you think?

> Patch v4 is attached.

Does this pass the test procedure shown upthread?
Hi,
I installed gcc 4.9.3

When I ran:
./configure CFLAGS='-fsanitize=undefined -fsanitize-undefined-trap-on-error' 

I saw:

configure:3977: $? = 0
configure:3966: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3966: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3977: $? = 1
configure:3997: checking whether the C compiler works
configure:4019: gcc -fsanitize=undefined -fsanitize-undefined-trap-on-error   conftest.c  >&5
gcc: error: unrecognized command line option '-fsanitize-undefined-trap-on-error'
configure:4023: $? = 1
configure:4061: result: no

I wonder if a higher version gcc is needed.

FYI
 
After installing gcc-11, ./configure passed (with 0003-memcpy-null.patch).
In the output of `make check-world`, I don't see `runtime error`.
Though there was a crash (maybe specific to my machine):

Core was generated by `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/postgres --singl'.
Program terminated with signal SIGILL, Illegal instruction.
#0  0x000000000050642d in write_item.cold ()
Missing separate debuginfos, use: debuginfo-install glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64 sssd-client-1.16.5-10.el7_9.10.x86_64
(gdb) bt
#0  0x000000000050642d in write_item.cold ()
#1  0x0000000000ba9d1b in write_relcache_init_file ()
#2  0x0000000000bb58f7 in RelationCacheInitializePhase3 ()
#3  0x0000000000bd5cb5 in InitPostgres ()
#4  0x0000000000a0a9ea in PostgresMain () 

FYI
Hi,
Earlier I was using devtoolset-11 which had an `Illegal instruction` error.

I compiled / installed gcc-11 from source (which took whole afternoon).
`make check-world` passed with patch v3.
In tmp_install/log/install.log, I saw:

gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
rm -f libpgport.a 

Hi, Noah: 
Patch v3 passes `make check-world`

Can you take another look ?

Re: null iv parameter passed to combo_init()

От
Noah Misch
Дата:
On Mon, Jan 10, 2022 at 03:34:27PM -0800, Zhihong Yu wrote:
> On Sun, Jan 9, 2022 at 6:45 PM Zhihong Yu <zyu@yugabyte.com> wrote:
> > gcc -Wall -Wmissing-prototypes -Wpointer-arith
> > -Wdeclaration-after-statement -Werror=vla -Wendif-labels
> > -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type
> > -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
> > -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined
> > -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND
> > -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
> 
> Patch v3 passes `make check-world`

The patch uses the "LENGTH_VAR != 0" style in px.c, but it uses "POINTER_VAR
!= NULL" style in the other files.  Please use "LENGTH_VAR != 0" style in each
place you're changing.

Assuming the next version looks good, I'll likely back-patch it to v10.  Would
anyone like to argue for a back-patch all the way to 9.2?



Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Wed, Jan 12, 2022 at 6:49 PM Noah Misch <noah@leadboat.com> wrote:
On Mon, Jan 10, 2022 at 03:34:27PM -0800, Zhihong Yu wrote:
> On Sun, Jan 9, 2022 at 6:45 PM Zhihong Yu <zyu@yugabyte.com> wrote:
> > gcc -Wall -Wmissing-prototypes -Wpointer-arith
> > -Wdeclaration-after-statement -Werror=vla -Wendif-labels
> > -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type
> > -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
> > -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined
> > -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND
> > -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
>
> Patch v3 passes `make check-world`

The patch uses the "LENGTH_VAR != 0" style in px.c, but it uses "POINTER_VAR
!= NULL" style in the other files.  Please use "LENGTH_VAR != 0" style in each
place you're changing.

Assuming the next version looks good, I'll likely back-patch it to v10.  Would
anyone like to argue for a back-patch all the way to 9.2?
Hi,
Please take a look at patch v5.

Cheers 
Вложения

Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Wed, Jan 12, 2022 at 7:08 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Wed, Jan 12, 2022 at 6:49 PM Noah Misch <noah@leadboat.com> wrote:
On Mon, Jan 10, 2022 at 03:34:27PM -0800, Zhihong Yu wrote:
> On Sun, Jan 9, 2022 at 6:45 PM Zhihong Yu <zyu@yugabyte.com> wrote:
> > gcc -Wall -Wmissing-prototypes -Wpointer-arith
> > -Wdeclaration-after-statement -Werror=vla -Wendif-labels
> > -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type
> > -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
> > -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined
> > -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND
> > -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
>
> Patch v3 passes `make check-world`

The patch uses the "LENGTH_VAR != 0" style in px.c, but it uses "POINTER_VAR
!= NULL" style in the other files.  Please use "LENGTH_VAR != 0" style in each
place you're changing.

Assuming the next version looks good, I'll likely back-patch it to v10.  Would
anyone like to argue for a back-patch all the way to 9.2?
Hi,
Please take a look at patch v5.

Cheers 
Noah:
Do you have any more review comments ?

Thanks

Re: null iv parameter passed to combo_init()

От
Noah Misch
Дата:
On Sun, Jan 09, 2022 at 06:45:09PM -0800, Zhihong Yu wrote:
> On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu <zyu@yugabyte.com> wrote:
> > After installing gcc-11, ./configure passed (with 0003-memcpy-null.patch).
> > In the output of `make check-world`, I don't see `runtime error`.

That's expected.  With -fsanitize-undefined-trap-on-error, the program will
generate SIGILL when UBSan detects undefined behavior.  To get "runtime error"
messages in the postmaster log, drop -fsanitize-undefined-trap-on-error.  Both
ways of running the tests have uses.  -fsanitize-undefined-trap-on-error is
better when you think the code is clean, because a zero "make check-world"
exit status confirms the code is clean.  Once you know the code is unclean in
some way, -fsanitize-undefined-trap-on-error is better for getting details.

> > Though there was a crash (maybe specific to my machine):
> >
> > Core was generated by
> > `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/postgres
> > --singl'.
> > Program terminated with signal SIGILL, Illegal instruction.
> > #0  0x000000000050642d in write_item.cold ()
> > Missing separate debuginfos, use: debuginfo-install
> > glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64
> > sssd-client-1.16.5-10.el7_9.10.x86_64
> > (gdb) bt
> > #0  0x000000000050642d in write_item.cold ()
> > #1  0x0000000000ba9d1b in write_relcache_init_file ()
> > #2  0x0000000000bb58f7 in RelationCacheInitializePhase3 ()
> > #3  0x0000000000bd5cb5 in InitPostgres ()
> > #4  0x0000000000a0a9ea in PostgresMain ()

That is UBSan detecting undefined behavior.  A successful patch version will
fix write_item(), among many other places that are currently making
check-world fail.  I get the same when testing your v5 under "gcc (Debian
11.2.0-13) 11.2.0".  I used the same host as buildfarm member thorntail, and I
configured like this:

  ./configure -C --with-lz4 --prefix=$HOME/sw/nopath/pghead --enable-tap-tests --enable-debug --enable-depend
--enable-cassertCC='ccache gcc-11 -fsanitize=undefined -fsanitize-undefined-trap-on-error' CFLAGS='-O2
-funwind-tables'

> Earlier I was using devtoolset-11 which had an `Illegal instruction` error.
> 
> I compiled / installed gcc-11 from source (which took whole afternoon).
> `make check-world` passed with patch v3.
> In tmp_install/log/install.log, I saw:
> 
> gcc -Wall -Wmissing-prototypes -Wpointer-arith
> -Wdeclaration-after-statement -Werror=vla -Wendif-labels
> -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type
> -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
> -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined
> -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND
> -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
> rm -f libpgport.a

Perhaps this self-compiled gcc-11 is defective, being unable to detect the
instances of undefined behavior that other builds detect.  If so, use the
"devtoolset-11" gcc instead.  You're also building without optimization; that
might be the problem.



Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Thu, Jan 13, 2022 at 8:09 PM Noah Misch <noah@leadboat.com> wrote:
On Sun, Jan 09, 2022 at 06:45:09PM -0800, Zhihong Yu wrote:
> On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu <zyu@yugabyte.com> wrote:
> > After installing gcc-11, ./configure passed (with 0003-memcpy-null.patch).
> > In the output of `make check-world`, I don't see `runtime error`.

That's expected.  With -fsanitize-undefined-trap-on-error, the program will
generate SIGILL when UBSan detects undefined behavior.  To get "runtime error"
messages in the postmaster log, drop -fsanitize-undefined-trap-on-error.  Both
ways of running the tests have uses.  -fsanitize-undefined-trap-on-error is
better when you think the code is clean, because a zero "make check-world"
exit status confirms the code is clean.  Once you know the code is unclean in
some way, -fsanitize-undefined-trap-on-error is better for getting details.

> > Though there was a crash (maybe specific to my machine):
> >
> > Core was generated by
> > `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/postgres
> > --singl'.
> > Program terminated with signal SIGILL, Illegal instruction.
> > #0  0x000000000050642d in write_item.cold ()
> > Missing separate debuginfos, use: debuginfo-install
> > glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64
> > sssd-client-1.16.5-10.el7_9.10.x86_64
> > (gdb) bt
> > #0  0x000000000050642d in write_item.cold ()
> > #1  0x0000000000ba9d1b in write_relcache_init_file ()
> > #2  0x0000000000bb58f7 in RelationCacheInitializePhase3 ()
> > #3  0x0000000000bd5cb5 in InitPostgres ()
> > #4  0x0000000000a0a9ea in PostgresMain ()

That is UBSan detecting undefined behavior.  A successful patch version will
fix write_item(), among many other places that are currently making
check-world fail.  I get the same when testing your v5 under "gcc (Debian
11.2.0-13) 11.2.0".  I used the same host as buildfarm member thorntail, and I
configured like this:

  ./configure -C --with-lz4 --prefix=$HOME/sw/nopath/pghead --enable-tap-tests --enable-debug --enable-depend --enable-cassert CC='ccache gcc-11 -fsanitize=undefined -fsanitize-undefined-trap-on-error' CFLAGS='-O2 -funwind-tables'

> Earlier I was using devtoolset-11 which had an `Illegal instruction` error.
>
> I compiled / installed gcc-11 from source (which took whole afternoon).
> `make check-world` passed with patch v3.
> In tmp_install/log/install.log, I saw:
>
> gcc -Wall -Wmissing-prototypes -Wpointer-arith
> -Wdeclaration-after-statement -Werror=vla -Wendif-labels
> -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type
> -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
> -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined
> -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND
> -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
> rm -f libpgport.a

Perhaps this self-compiled gcc-11 is defective, being unable to detect the
instances of undefined behavior that other builds detect.  If so, use the
"devtoolset-11" gcc instead.  You're also building without optimization; that
might be the problem.

I tried both locally built gcc-11 and devtoolset-11 with configure command copied from above.
`make world` failed in both cases with:

performing post-bootstrap initialization ... sh: line 1: 24714 Illegal instruction     (core dumped) ".../postgres/tmp_install/.../postgres/bin/postgres" --single -F -O -j -c search_path=pg_catalog -c exit_on_error=true -c log_checkpoints=false template1 > /dev/null
child process exited with exit code 132 

#0  0x000000000050a8d6 in write_item (data=<optimized out>, len=<optimized out>, fp=<optimized out>) at relcache.c:6471
#1  0x0000000000c33273 in write_relcache_init_file (shared=true) at relcache.c:6368
#2  0x0000000000c33c50 in RelationCacheInitializePhase3 () at relcache.c:4220
#3  0x0000000000c55825 in InitPostgres (in_dbname=<optimized out>, dboid=3105442800, username=<optimized out>, useroid=<optimized out>, out_dbname=0x0, override_allow_connections=<optimized out>) at postinit.c:1014

FYI

Re: null iv parameter passed to combo_init()

От
Zhihong Yu
Дата:


On Thu, Jan 13, 2022 at 9:09 PM Zhihong Yu <zyu@yugabyte.com> wrote:


On Thu, Jan 13, 2022 at 8:09 PM Noah Misch <noah@leadboat.com> wrote:
On Sun, Jan 09, 2022 at 06:45:09PM -0800, Zhihong Yu wrote:
> On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu <zyu@yugabyte.com> wrote:
> > After installing gcc-11, ./configure passed (with 0003-memcpy-null.patch).
> > In the output of `make check-world`, I don't see `runtime error`.

That's expected.  With -fsanitize-undefined-trap-on-error, the program will
generate SIGILL when UBSan detects undefined behavior.  To get "runtime error"
messages in the postmaster log, drop -fsanitize-undefined-trap-on-error.  Both
ways of running the tests have uses.  -fsanitize-undefined-trap-on-error is
better when you think the code is clean, because a zero "make check-world"
exit status confirms the code is clean.  Once you know the code is unclean in
some way, -fsanitize-undefined-trap-on-error is better for getting details.

> > Though there was a crash (maybe specific to my machine):
> >
> > Core was generated by
> > `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/postgres
> > --singl'.
> > Program terminated with signal SIGILL, Illegal instruction.
> > #0  0x000000000050642d in write_item.cold ()
> > Missing separate debuginfos, use: debuginfo-install
> > glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64
> > sssd-client-1.16.5-10.el7_9.10.x86_64
> > (gdb) bt
> > #0  0x000000000050642d in write_item.cold ()
> > #1  0x0000000000ba9d1b in write_relcache_init_file ()
> > #2  0x0000000000bb58f7 in RelationCacheInitializePhase3 ()
> > #3  0x0000000000bd5cb5 in InitPostgres ()
> > #4  0x0000000000a0a9ea in PostgresMain ()

That is UBSan detecting undefined behavior.  A successful patch version will
fix write_item(), among many other places that are currently making
check-world fail.  I get the same when testing your v5 under "gcc (Debian
11.2.0-13) 11.2.0".  I used the same host as buildfarm member thorntail, and I
configured like this:

  ./configure -C --with-lz4 --prefix=$HOME/sw/nopath/pghead --enable-tap-tests --enable-debug --enable-depend --enable-cassert CC='ccache gcc-11 -fsanitize=undefined -fsanitize-undefined-trap-on-error' CFLAGS='-O2 -funwind-tables'

> Earlier I was using devtoolset-11 which had an `Illegal instruction` error.
>
> I compiled / installed gcc-11 from source (which took whole afternoon).
> `make check-world` passed with patch v3.
> In tmp_install/log/install.log, I saw:
>
> gcc -Wall -Wmissing-prototypes -Wpointer-arith
> -Wdeclaration-after-statement -Werror=vla -Wendif-labels
> -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type
> -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
> -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined
> -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND
> -I../../src/include  -D_GNU_SOURCE   -c -o path.o path.c
> rm -f libpgport.a

Perhaps this self-compiled gcc-11 is defective, being unable to detect the
instances of undefined behavior that other builds detect.  If so, use the
"devtoolset-11" gcc instead.  You're also building without optimization; that
might be the problem.

I tried both locally built gcc-11 and devtoolset-11 with configure command copied from above.
`make world` failed in both cases with:

performing post-bootstrap initialization ... sh: line 1: 24714 Illegal instruction     (core dumped) ".../postgres/tmp_install/.../postgres/bin/postgres" --single -F -O -j -c search_path=pg_catalog -c exit_on_error=true -c log_checkpoints=false template1 > /dev/null
child process exited with exit code 132 

#0  0x000000000050a8d6 in write_item (data=<optimized out>, len=<optimized out>, fp=<optimized out>) at relcache.c:6471
#1  0x0000000000c33273 in write_relcache_init_file (shared=true) at relcache.c:6368
#2  0x0000000000c33c50 in RelationCacheInitializePhase3 () at relcache.c:4220
#3  0x0000000000c55825 in InitPostgres (in_dbname=<optimized out>, dboid=3105442800, username=<optimized out>, useroid=<optimized out>, out_dbname=0x0, override_allow_connections=<optimized out>) at postinit.c:1014

FYI
Hi,
I forgot to mention that patch v5 was included during the experiment.

Cheers