Обсуждение: ISSTRICT behavior

Hi,

Is there any way of mimicking the IS STRICT behavior
*without* declaring the function as "STRICT"?

E.g., I could use PG_ARGISNULL() to check for invocation
with a NULL argument.  But, the function has already been
invoked at that point.  I.e. I still have to *return*
something -- which might be used as an argument to some
other function, etc.

Or, is there some other way of aborting the function
invocation (e.g., signal an ERROR, etc.)?

Lastly, am I correct in assuming that the STRICT
modifier only protect a function from being invoked
with a NULL _from_the_SQL_interface?  I.e. does it
also protect the function from being invoked from
PL/pgsql?  (I'm sure it does NOT protect from a
deliberate invocation from a C language function, etc.)

(sigh)  Sorry to be picking nits.  I'm just trying to
sort out how to protect against folks making careless
mistakes in the future (e.g., forgetting STRICT in
the function declarations when adding a function to
a database, etc)

Thanks!
--don

Don Y <pgsql@DakotaCom.Net> writes:
> Is there any way of mimicking the IS STRICT behavior
> *without* declaring the function as "STRICT"?

Are you looking for PG_RETURN_NULL()?  If not, this seems a bit
nonsensical.

> Lastly, am I correct in assuming that the STRICT
> modifier only protect a function from being invoked
> with a NULL _from_the_SQL_interface?  I.e. does it
> also protect the function from being invoked from
> PL/pgsql?  (I'm sure it does NOT protect from a
> deliberate invocation from a C language function, etc.)

If a caller fails to check this (directly or indirectly),
it's the caller's bug not the callee's.  To say otherwise
would require introducing ARGISNULL checks into literally
thousands of places.  There are far fewer call sites of
arbitrary functions, so we put the burden on the callers.

            regards, tom lane

On Wed, May 03, 2006 at 11:02:26PM -0700, Don Y wrote:
> (sigh)  Sorry to be picking nits.  I'm just trying to
> sort out how to protect against folks making careless
> mistakes in the future (e.g., forgetting STRICT in
> the function declarations when adding a function to
> a database, etc)

Unfortunatly there is no way to ensure the user declares the function
in SQL in the way your code expects. I remember a discussion once about
allowing you to declare the essential function details in the C file so
that Postgres could complain if the user got it wrong, but nothing ever
came of it.

In short, give them an INSTALL.sql and tell them to use it.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Tom Lane wrote:
> Don Y <pgsql@DakotaCom.Net> writes:
>> Is there any way of mimicking the IS STRICT behavior
>> *without* declaring the function as "STRICT"?
>
> Are you looking for PG_RETURN_NULL()?  If not, this seems a bit
> nonsensical.

No.

First, if the function is defined to return an INT16,
then returning a NULL doesn't make any sense -- since the
caller doesn't know how to deal with a NULL (it expects
an INT16, for example).  It is my understanding that
returning NULL doesn't abort the call chain, etc.
So, the NULL has to be handled as a return value... in
a place that is expecting an INT16 (in this case).

What I am trying to do is make functions more robust.
As it stands currently, the functions get written and
compiled "once".  Thereafter, someone can FAIL to
specify STRICT when creating those functions in SQL
(CREATE FUNCTION...) and leave the server vulnerable
to having those functions invoked with NULL arguments.

I can PG_ARGISNULL(x) in each function and issue an
error if I detect that the function is being invoked
with a NULL (this code would be superfluous if the
function was declared as STRICT -- since it would never
have been invoked at all!).  But, I am assuming that
this just generates the message but doesn't otherwise
prevent/abort the function call (the function wouldn't
have been invoked if it had been STRICT so the whole
process would have never been started).  So, while I
may have explicitly protected this function, I now have
to return something (some VALUE) that remains consistent
with the expectations of anything that invokes this
function (e.g., some bogus value that is at least the
correct *type*).

I.e. there is no way to abort the "computation" (from
within the function that detects the invocation with NULL)

>> Lastly, am I correct in assuming that the STRICT
>> modifier only protect a function from being invoked
>> with a NULL _from_the_SQL_interface?  I.e. does it
>> also protect the function from being invoked from
>> PL/pgsql?  (I'm sure it does NOT protect from a
>> deliberate invocation from a C language function, etc.)
>
> If a caller fails to check this (directly or indirectly),
> it's the caller's bug not the callee's.  To say otherwise
> would require introducing ARGISNULL checks into literally
> thousands of places.  There are far fewer call sites of
> arbitrary functions, so we put the burden on the callers.

But, specifically, *does* the STRICT modifier apply to
PL/pgsql invocations of functions?  Or, just those
"from the command line" (so to speak).  E.g., if I
invoke a STRICT function with a NULL argument from
within a trigger, will the server core/SEGV?  Or, will
it catch the attempt just like it would if I had
tried to invoke the function in a SELECT statement
issued through psql?

Martijn van Oosterhout wrote:
> On Wed, May 03, 2006 at 11:02:26PM -0700, Don Y wrote:
>> (sigh)  Sorry to be picking nits.  I'm just trying to
>> sort out how to protect against folks making careless
>> mistakes in the future (e.g., forgetting STRICT in
>> the function declarations when adding a function to
>> a database, etc)
>
> Unfortunatly there is no way to ensure the user declares the function
> in SQL in the way your code expects. I remember a discussion once about
> allowing you to declare the essential function details in the C file so
> that Postgres could complain if the user got it wrong, but nothing ever
> came of it.
>
> In short, give them an INSTALL.sql and tell them to use it.

Is there any way to prevent them from *adding* these functions
(i.e. build them into template) so they have to use them the
way *I* have already defined them?

Don Y <pgsql@DakotaCom.Net> writes:
> First, if the function is defined to return an INT16,
> then returning a NULL doesn't make any sense -- since the
> caller doesn't know how to deal with a NULL (it expects
> an INT16, for example).

Really?  That would be a caller bug, if it's calling a function
that might return NULL.

> What I am trying to do is make functions more robust.
> As it stands currently, the functions get written and
> compiled "once".  Thereafter, someone can FAIL to
> specify STRICT when creating those functions in SQL
> (CREATE FUNCTION...) and leave the server vulnerable
> to having those functions invoked with NULL arguments.

This would be the error of the person specifying the function's
SQL definition.  Since there are many ways to crash the system by
writing a C function definition wrongly (eg, give the wrong
datatypes), I can't get very excited about this particular one.
We do make this a superuser-only feature for a reason: you're
expected to be competent enough to get it right.

            regards, tom lane

Don Y <pgsql@DakotaCom.Net> writes:
> Is there any way to prevent them from *adding* these functions
> (i.e. build them into template) so they have to use them the
> way *I* have already defined them?

Only if you think you can deny your users superuser privileges on
their own databases.  Good luck selling that one (I sure wouldn't
buy any code that tried to enforce such a thing)

            regards, tom lane

On Wed, May 03, 2006 at 11:45:31PM -0700, Don Y wrote:
> Martijn van Oosterhout wrote:
> >Unfortunatly there is no way to ensure the user declares the function
> >in SQL in the way your code expects. I remember a discussion once about
> >allowing you to declare the essential function details in the C file so
> >that Postgres could complain if the user got it wrong, but nothing ever
> >came of it.
> >
> >In short, give them an INSTALL.sql and tell them to use it.
>
> Is there any way to prevent them from *adding* these functions
> (i.e. build them into template) so they have to use them the
> way *I* have already defined them?

I'm not sure if I'm understanding you correctly, but if you're asking
"can I stop users from creating new SQL functions that use my existing
C function in strange ways" then the answer is no. All you can do is
stop postgresql from seeing the function under any circumstances, by
declaring it "static" or using some other way to strip the symbol from
the resulting library.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Tom Lane wrote:
> Don Y <pgsql@DakotaCom.Net> writes:
>> Is there any way to prevent them from *adding* these functions
>> (i.e. build them into template) so they have to use them the
>> way *I* have already defined them?
>
> Only if you think you can deny your users superuser privileges on
> their own databases.  Good luck selling that one (I sure wouldn't
> buy any code that tried to enforce such a thing)

I'm not designing for the "traditional" role that you're
used to so I can do whatever makes sense for this product
and just *define* that as it's behavior.  Since there are
no other products that compete with it, users don't
really have much choice!  :>

Having said that, I don't want to be arbitrary in the
impositions I make on others.

One potential workaround is to just disable the ability
to add functions altogether.  Those that I opt to build
in can be proven to work properly and other functions
can just be prohibited.  Thus, no need to worry about
the user failing to declare the functions properly.

Another option (for *me*) is to hack the parser so that
it applies the STRICT modifier to all CREATE FUNCTION
declarations implicitly.  And, change the rules for
function definitions so that the function will know
that it will never be invoked with NULL.

But, that's also heavy-handed.  I'd prefer to just
build functions that are robust enough to handle this
"loophole" in the declaration/definition interfaces
and let other folks copy that model if they chose
to write their own functions.  The more I bastardize
the user interface, the less usable my codebase will
be for use by "mainstream" developers.  :-(  I already
have several changes that won't back port due to their
application specific nature; I'm trying to keep the
number of such changes to a minimum...

Martijn van Oosterhout wrote:
> On Wed, May 03, 2006 at 11:45:31PM -0700, Don Y wrote:
>> Martijn van Oosterhout wrote:
>>> Unfortunatly there is no way to ensure the user declares the function
>>> in SQL in the way your code expects. I remember a discussion once about
>>> allowing you to declare the essential function details in the C file so
>>> that Postgres could complain if the user got it wrong, but nothing ever
>>> came of it.
>>>
>>> In short, give them an INSTALL.sql and tell them to use it.
>> Is there any way to prevent them from *adding* these functions
>> (i.e. build them into template) so they have to use them the
>> way *I* have already defined them?
>
> I'm not sure if I'm understanding you correctly, but if you're asking
> "can I stop users from creating new SQL functions that use my existing
> C function in strange ways" then the answer is no. All you can do is

*That* was (one of) the questions.  Or, "can I stop them from
using functions in ways that the parser CAN'T catch but which
could impact the reliability of the server, etc."

> stop postgresql from seeing the function under any circumstances, by
> declaring it "static" or using some other way to strip the symbol from
> the resulting library.

I don't want to hide the function; just ensure that no one
*redefines* the SQL interface to it in a manner that is
inconsistent with its implementation.  If I can make the
implementation robust enough that it could protect itself
against this potential, that would be acceptable (hence
my original question).  Barring that, I need to do whatever
it takes to safeguard the server so that it can't be brought
to its knees by a simple bug like failing to specify STRICT, etc.

On Thu, May 04, 2006 at 12:19:12AM -0700, Don Y wrote:
> I'm not designing for the "traditional" role that you're
> used to so I can do whatever makes sense for this product
> and just *define* that as it's behavior.  Since there are
> no other products that compete with it, users don't
> really have much choice!  :>

You can do what you like, however, it's still not clear to me what you
think the problem is. If you want your functions to be declared STRICT,
provide a .sql file that does that. It you want to program defensivly
and not crash if the user declares the function without STRICT, add:

if( PG_ARGISNULL(...) )
    PG_RETURN_NULL();

Which has exactly the same effect. Of course, users could screw up the
data-types also so you could, if you wanted, add more code to check the
datatypes passed.

Fact is, if the user has superuser priveledges, they can create C
functions any way they like. If you want to protect from that you need
to add stuff to your C function.

Have a ncie day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Tom Lane wrote:
> Don Y <pgsql@DakotaCom.Net> writes:
>> First, if the function is defined to return an INT16,
>> then returning a NULL doesn't make any sense -- since the
>> caller doesn't know how to deal with a NULL (it expects
>> an INT16, for example).
>
> Really?  That would be a caller bug, if it's calling a function
> that might return NULL.

The function wouldn't be expected (nor defined!) to return NULL.
So, if it was INVOKED with a NULL, it could *detect* that
and issue an error message.  But, it could NOT do PG_RETURN_NULL
since the caller(s) wouldn't be expecting a NULL return value
(why complicate every routine by forcing them all to handle
NULLs?).  Hence my comment that I would have to pick some
generic (insignificant) value that was consistent with the
return type of the function and pass that along.

OTOH, if the function could *abort* it's invocation, then
I don't have to worry about return values.  It is a closer
model to the STRICT behavior -- instead of aborting the
function invocation BEFORE (which STRICT essentially does),
I could abort it AFTER invocation (once I had detected
the NULL argument)

>> What I am trying to do is make functions more robust.
>> As it stands currently, the functions get written and
>> compiled "once".  Thereafter, someone can FAIL to
>> specify STRICT when creating those functions in SQL
>> (CREATE FUNCTION...) and leave the server vulnerable
>> to having those functions invoked with NULL arguments.
>
> This would be the error of the person specifying the function's
> SQL definition.  Since there are many ways to crash the system by
> writing a C function definition wrongly (eg, give the wrong
> datatypes), I can't get very excited about this particular one.
> We do make this a superuser-only feature for a reason: you're
> expected to be competent enough to get it right.

So, it might be more prudent to build the functions that
are needed and then just prevent other  functions from being
added at all!  That's a relatively easy parser hack...

On Thu, May 04, 2006 at 12:23:07AM -0700, Don Y wrote:
> I don't want to hide the function; just ensure that no one
> *redefines* the SQL interface to it in a manner that is
> inconsistent with its implementation.  If I can make the
> implementation robust enough that it could protect itself
> against this potential, that would be acceptable (hence
> my original question).  Barring that, I need to do whatever
> it takes to safeguard the server so that it can't be brought
> to its knees by a simple bug like failing to specify STRICT, etc.

Well, if you really want to, you can do this at the beginning of each
function. It makes it completely fool-proof (until someone finds a
better fool ofcourse).

--- cut ---
if( PG_NARGS != exptectedargs )
   die ("bad args" );
for( i=0; i<PG_NARGS; i++ )
{
   if( PG_ARGISNULL(i) )
        PG_RETURN_NULL();
   if( get_fn_expr_argtype( fcinfo->flinfo, i ) != expectedtypes[i] )
        die("bad args" );
}

if( get_fn_expr_rettype(fcinfo->flinfo) )
    die "bad return type";
--- cut ---

--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On Thu, May 04, 2006 at 12:29:30AM -0700, Don Y wrote:
> OTOH, if the function could *abort* it's invocation, then
> I don't have to worry about return values.  It is a closer
> model to the STRICT behavior -- instead of aborting the
> function invocation BEFORE (which STRICT essentially does),
> I could abort it AFTER invocation (once I had detected
> the NULL argument)

Are you sure you understand what STRICT does? STRICT doesn't abort
anything. STRICT means "if this function gets called with any NULL
arguments, the result is NULL". Since this is correct behaviour for the
vast majority of functions, it's implemented as a flag rather than
requiring each and every function to check.

Also, anything that calls a function must be prepared to handle a NULL
return value. Any function can return NULL, even if only because it is
declared strict and you passed a null argument...

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Martijn van Oosterhout wrote:
> On Thu, May 04, 2006 at 12:19:12AM -0700, Don Y wrote:
>> I'm not designing for the "traditional" role that you're
>> used to so I can do whatever makes sense for this product
>> and just *define* that as it's behavior.  Since there are
>> no other products that compete with it, users don't
>> really have much choice!  :>
>
> You can do what you like, however, it's still not clear to me what you
> think the problem is. If you want your functions to be declared STRICT,
> provide a .sql file that does that. It you want to program defensivly
> and not crash if the user declares the function without STRICT, add:
>
> if( PG_ARGISNULL(...) )
>     PG_RETURN_NULL();

But, this means anything that invokes the function has to be
ready to handle a NULL returned *from* this function.

> Which has exactly the same effect.

That was my original question.  I don't think it *is* the same.
If you declare STRICT, the function never gets invoked!
If you use PG_ARGISNULL, the function *has* been invoked
and (the subject of another of my questions), all I can do
is issue a message -- but I can't stop the rest of the
execution:

SELECT nonstrict(another(...)) ...

Nonstrict() has been defined to expect an INT (e.g.).
I.e. that's how it *should* be defined.

If another() is misdeclared as not STRICT (because the
user is not the one who wrote the actual *code* for it)
and happens to be invoked with NULL, then it *can*
detect that it has been invoked with NULL (PG_ARGISNULL)
and it can issue an error message.  But, it will have
to return an INT (*not* NULL) otherwise Nonstrict will
complain (crash?) when it sees the NULL *result* from the
another() invocation.

If, instead, another() could see the NULL passed to *it*,
issue an error and then *abort* the "process" that is
running the SELECT...

This would be more in line with how the SELECT would
operate if another() *had* been properly declared as
STRICT yet invoked with NULL!

(Or, have I misunderstood something?)

Ditto:

SELECT another_function(...) + another_function(...)  ...

> Of course, users could screw up the
> data-types also so you could, if you wanted, add more code to check the
> datatypes passed.
>
> Fact is, if the user has superuser priveledges, they can create C
> functions any way they like. If you want to protect from that you need
> to add stuff to your C function.

I'm not worried about the folks writing the actual C code.
What I am worried about is the folks who use the wrong
.sql to CREATE FUNCTION...

It may be best (for me) to just cut this capability out
completely...

Martijn van Oosterhout wrote:
> On Thu, May 04, 2006 at 12:29:30AM -0700, Don Y wrote:
>> OTOH, if the function could *abort* it's invocation, then
>> I don't have to worry about return values.  It is a closer
>> model to the STRICT behavior -- instead of aborting the
>> function invocation BEFORE (which STRICT essentially does),
>> I could abort it AFTER invocation (once I had detected
>> the NULL argument)
>
> Are you sure you understand what STRICT does? STRICT doesn't abort
> anything. STRICT means "if this function gets called with any NULL
> arguments, the result is NULL". Since this is correct behaviour for the
> vast majority of functions, it's implemented as a flag rather than
> requiring each and every function to check.

Ah, I have misunderstood this!  I saw it as "if invoked with NULL,
don't invoke the function" -- which, I guess, is somewhat consistent
with your comment... the function isn't invoked, BUT 'NULL' IS
SUBSTITUTED FOR IT'S RETURN VALUE.

> Also, anything that calls a function must be prepared to handle a NULL
> return value. Any function can return NULL, even if only because it is
> declared strict and you passed a null argument...

So, even the functions that I am defining that ALWAYS return
particular types have to be treated AS IF they could return NULL?
(in the case you describe -- the opposite case that I have been
describing)

Hmmm... that might make things even easier!

Martijn van Oosterhout wrote:
> On Thu, May 04, 2006 at 12:23:07AM -0700, Don Y wrote:
>> I don't want to hide the function; just ensure that no one
>> *redefines* the SQL interface to it in a manner that is
>> inconsistent with its implementation.  If I can make the
>> implementation robust enough that it could protect itself
>> against this potential, that would be acceptable (hence
>> my original question).  Barring that, I need to do whatever
>> it takes to safeguard the server so that it can't be brought
>> to its knees by a simple bug like failing to specify STRICT, etc.
>
> Well, if you really want to, you can do this at the beginning of each
> function. It makes it completely fool-proof (until someone finds a
> better fool ofcourse).
>
> --- cut ---
> if( PG_NARGS != exptectedargs )
>    die ("bad args" );
> for( i=0; i<PG_NARGS; i++ )
> {
>    if( PG_ARGISNULL(i) )
>         PG_RETURN_NULL();
>    if( get_fn_expr_argtype( fcinfo->flinfo, i ) != expectedtypes[i] )
>         die("bad args" );
> }
>
> if( get_fn_expr_rettype(fcinfo->flinfo) )
>     die "bad return type";

Given your (previous) comment clarifying STRICT, this is
clearly the way to go.  It "tolerates" errors in the
CREATE FUNCTION statements.  Granted, the user can still
screw up the definition so that things don't *work*
properly... but, he can't strip off some protections
that STRICT affords a function *written* with those
expectations.

(I don't care if the software "doesn't work"... I just
have to make sure the server isn't compromised in the
process)

Thanks!
--don