Обсуждение: proposal: type info support functions for functions that use "any" type

Hi,

Tom introduced supported functions for calculation function's selectivity. Still I have similar idea to use supported function for calculation function's parameter's types and function return type.

Motivation:

Reduce a necessity of overloading of functions. My motivation is related primary to Orafce, but this feature should be helpful for anybody with similar goals. The function's overloading is great functionality but it is hard for maintenance.

My idea to enhance a CREATE FUNCTION command to be able do

CREATE FUCNTION foo("any")

RETURNS "any" AS ...

TYPEINFO foo_typeinfo

CREATE FUNCTION decode(VARIADIC "any")

RETURNS "any" AS ...

TYPEINFO decode_typeinfo.

The typeinfo functions returns a pointer tu structure with param types and result type. Only function with "any" parameters or "any" result can use TYPEINFO supported function. This functionality should not be allowed for common functions.

This functionality is limited just for C coders. But I expect so typical application coder doesn't need it. It doesn't replace my proposal of introduction other polymorphic type - now named "commontype" (can be named differently). The commontype is good enough solution for application coders, developers.

Comments, notes?

Regards

Pavel

Pavel Stehule <pavel.stehule@gmail.com> writes:
> so 9. 3. 2019 v 7:22 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
> napsal:
>> Tom introduced supported functions for calculation function's selectivity.
>> Still I have similar idea to use supported function for calculation
>> function's parameter's types and function return type.
>> Motivation:
>> Reduce a necessity of overloading of functions. My motivation is related
>> primary to Orafce, but this feature should be helpful for anybody with
>> similar goals. The function's overloading is great functionality but it is
>> hard for maintenance.

> here is a patch

TBH, I don't like this proposal one bit.  As far as I can see, the idea
is to let a function's support function redefine the function's declared
argument and result types on-the-fly according to no predetermined rules,
and that seems to me like it's a recipe for disaster.  How will anyone
understand which function(s) are candidates to match a query, or why one
particular candidate got selected over others?  It's already hard enough
to understand the behavior of polymorphic functions in complex cases,
and those are much more constrained than this would be.

Moreover, I don't think you've even provided a compelling example
case.  What's this doing that you couldn't do with existing polymorphic
types or the anycompatibletype proposal?

I also strongly suspect that this would break pieces of the system
that expect that the stored pg_proc.prorettype has something to do
with reality.  At minimum, you'd need to fix a number of places you
haven't touched here that have their own knowledge of function type
resolution, such as enforce_generic_type_consistency,
resolve_polymorphic_argtypes, resolve_aggregate_transtype.  Probably
anyplace that treats polymorphics as being any sort of special case
would have to be taught to re-call the support function to find out
what it should think the relevant types are.

(I don't even want to think about what happens if the support function's
behavior changes between original parsing and these re-checking spots.)

Another thing that's very much less than compelling about your example
is that your support function seems to be happy to throw errors
if the argument types don't match what it's expecting.  That seems
quite unacceptable, since it would prevent the parser from moving on
to consider other possibly-matching functions.  Maybe that's just
because it's a quick hack not a polished example, but it doesn't
seem like a good precedent.

In short, I think the added complexity and bug potential outweigh
any possible gain from this.

                        regards, tom lane

I wrote:
> TBH, I don't like this proposal one bit.  As far as I can see, the idea
> is to let a function's support function redefine the function's declared
> argument and result types on-the-fly according to no predetermined rules,
> and that seems to me like it's a recipe for disaster.  How will anyone
> understand which function(s) are candidates to match a query, or why one
> particular candidate got selected over others?  It's already hard enough
> to understand the behavior of polymorphic functions in complex cases,
> and those are much more constrained than this would be.

After thinking about this a bit more, it seems like you could avoid
a lot of problems if you restricted what the support function call
does to be potentially replacing the result type of a function
declared to return ANY with some more-specific type (computed from
examination of the actual arguments).  That would make it act much
more like a traditional polymorphic function.  It'd remove the issues
about interactions among multiple potentially-matching functions,
since we'd only call a single support function for an already-identified
target function.

You'd still need to touch everyplace that knows about polymorphic
type resolution, since this would essentially be another form of
polymorphic function.  And I'm still very dubious that it's worth
the trouble.  But it would be a lot more controllable than the
proposal as it stands.

                        regards, tom lane

On Sat, Jul 27, 2019 at 5:45 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:
> pá 26. 7. 2019 v 22:53 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:
>> I wrote:
>> > TBH, I don't like this proposal one bit.  As far as I can see, the idea
>> > is to let a function's support function redefine the function's declared
>> > argument and result types on-the-fly according to no predetermined rules,
>> > and that seems to me like it's a recipe for disaster.  How will anyone
>> > understand which function(s) are candidates to match a query, or why one
>> > particular candidate got selected over others?  It's already hard enough
>> > to understand the behavior of polymorphic functions in complex cases,
>> > and those are much more constrained than this would be.
>>
>> After thinking about this a bit more, it seems like you could avoid
>> a lot of problems if you restricted what the support function call
>> does to be potentially replacing the result type of a function
>> declared to return ANY with some more-specific type (computed from
>> examination of the actual arguments).  That would make it act much
>> more like a traditional polymorphic function.  It'd remove the issues
>> about interactions among multiple potentially-matching functions,
>> since we'd only call a single support function for an already-identified
>> target function.
>
>
> I am not sure if I understand well - so I repeat it with my words.
>
> So calculation of result type (replace ANY by some specific) can be ok?
>
> I am able to do it if there will be a agreement.
...

Hi Pavel,

I see that this is an active project with an ongoing discussion, but
we have run out of July so I have moved this to the September CF and
set it to "Waiting on Author".

--
Thomas Munro
https://enterprisedb.com

I wrote:
> TBH, I don't like this proposal one bit.  As far as I can see, the idea
> is to let a function's support function redefine the function's declared
> argument and result types on-the-fly according to no predetermined rules,
> and that seems to me like it's a recipe for disaster.  How will anyone
> understand which function(s) are candidates to match a query, or why one
> particular candidate got selected over others?  It's already hard enough
> to understand the behavior of polymorphic functions in complex cases,
> and those are much more constrained than this would be.

After thinking about this a bit more, it seems like you could avoid
a lot of problems if you restricted what the support function call
does to be potentially replacing the result type of a function
declared to return ANY with some more-specific type (computed from
examination of the actual arguments).  That would make it act much
more like a traditional polymorphic function.  It'd remove the issues
about interactions among multiple potentially-matching functions,
since we'd only call a single support function for an already-identified
target function.

You'd still need to touch everyplace that knows about polymorphic
type resolution, since this would essentially be another form of
polymorphic function.  And I'm still very dubious that it's worth
the trouble.  But it would be a lot more controllable than the
proposal as it stands.

                        regards, tom lane

Hi

rebase

Pavel

Pavel Stehule <pavel.stehule@gmail.com> writes:
>  [ parser-support-function-with-demo-20191128.patch ]

TBH, I'm still not convinced that this is a good idea.  Restricting
the support function to only change the function's return type is
safer than the original proposal, but it's still not terribly safe.
If you change the support function's algorithm in any way, how do
you know whether you've broken existing stored queries?  If the
support function consults external resources to make its choice
(perhaps checking the existence of a cast), where could we record
that the query depends on the existence of that cast?  There'd be
no visible trace of that in the query parsetree.

I'm also still not convinced that this idea allows doing anything
that can't be done just as well with polymorphism.  It would be a
really bad idea for the support function to be examining the values
of the arguments (else what happens when they're not constants?).
So all you can do is look at their types, and then it seems like
the things you can usefully do are pretty much like polymorphism,
i.e. select some one of the input types, or a related type such
as an array type or element type.  If there are gaps in what you
can express with polymorphism, I'd much rather spend effort on
improving that facility than in adding something that is only
accessible to advanced C coders.  (Yes, I know I've been slacking
on reviewing [1].)

Lastly, I still think that this patch doesn't begin to address
all the places that would have to know about the feature.  There's
a lot of places that know about polymorphism --- if this is
polymorphism on steroids, which it is, then why don't all of those
places need to be touched?

On the whole I think we should reject this idea.

                        regards, tom lane

[1] https://commitfest.postgresql.org/26/1911/

út 14. 1. 2020 v 22:09 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Pavel Stehule <pavel.stehule@gmail.com> writes:
>  [ parser-support-function-with-demo-20191128.patch ]

TBH, I'm still not convinced that this is a good idea.  Restricting
the support function to only change the function's return type is
safer than the original proposal, but it's still not terribly safe.
If you change the support function's algorithm in any way, how do
you know whether you've broken existing stored queries?  If the
support function consults external resources to make its choice
(perhaps checking the existence of a cast), where could we record
that the query depends on the existence of that cast?  There'd be
no visible trace of that in the query parsetree.

This risk is real and cannot be simply solved without more complications.

Can be solution to limit and enforce this functionality only for extensions that be initialized from shared_preload_libraries or local_preload_libraries?

I'm also still not convinced that this idea allows doing anything
that can't be done just as well with polymorphism.  It would be a
really bad idea for the support function to be examining the values
of the arguments (else what happens when they're not constants?).
So all you can do is look at their types, and then it seems like
the things you can usefully do are pretty much like polymorphism,
i.e. select some one of the input types, or a related type such
as an array type or element type.  If there are gaps in what you
can express with polymorphism, I'd much rather spend effort on
improving that facility than in adding something that is only
accessible to advanced C coders.  (Yes, I know I've been slacking
on reviewing [1].)

For my purpose critical information is type. I don't need to work with constant, but I can imagine, so some API can be nice to work with constant value.

Yes, I can solve lot of things by patch [1], but not all, and this patch shorter, and almost trivial.

Lastly, I still think that this patch doesn't begin to address
all the places that would have to know about the feature.  There's
a lot of places that know about polymorphism --- if this is
polymorphism on steroids, which it is, then why don't all of those
places need to be touched?

I am sorry, I don't understand  last sentence?

On the whole I think we should reject this idea.

                        regards, tom lane

[1] https://commitfest.postgresql.org/26/1911/

Pavel Stehule <pavel.stehule@gmail.com> writes:
>  [ parser-support-function-with-demo-20191128.patch ]

TBH, I'm still not convinced that this is a good idea.  Restricting
the support function to only change the function's return type is
safer than the original proposal, but it's still not terribly safe.
If you change the support function's algorithm in any way, how do
you know whether you've broken existing stored queries?  If the
support function consults external resources to make its choice
(perhaps checking the existence of a cast), where could we record
that the query depends on the existence of that cast?  There'd be
no visible trace of that in the query parsetree.

I'm also still not convinced that this idea allows doing anything
that can't be done just as well with polymorphism.  It would be a
really bad idea for the support function to be examining the values
of the arguments (else what happens when they're not constants?).
So all you can do is look at their types, and then it seems like
the things you can usefully do are pretty much like polymorphism,
i.e. select some one of the input types, or a related type such
as an array type or element type.  If there are gaps in what you
can express with polymorphism, I'd much rather spend effort on
improving that facility than in adding something that is only
accessible to advanced C coders.  (Yes, I know I've been slacking
on reviewing [1].)

Lastly, I still think that this patch doesn't begin to address
all the places that would have to know about the feature.  There's
a lot of places that know about polymorphism --- if this is
polymorphism on steroids, which it is, then why don't all of those
places need to be touched?

On the whole I think we should reject this idea.

                        regards, tom lane

[1] https://commitfest.postgresql.org/26/1911/

Daniel Gustafsson <daniel@yesql.se> writes:
> This patch has been bumped in CFs for the past year, with the thread stalled
> and the last review comment being in support of rejection.  Tom, do you still
> feel it should be rejected in light of Pavel's latest posts?

I have seen no convincing response to the concerns I raised in my
last message in the thread [1], to wit that

1. I think the "flexibility" of letting a support function resolve the
output type in some unspecified way is mostly illusory, because if it
doesn't do it in a way that's morally equivalent to polymorphism, it's
doing it wrong.  Also, I'm not that excited about improving polymorphism
in a way that is only accessible with specialized C code.  The example of
Oracle-like DECODE() could be handled about as well if we had a second set
of anycompatible-style polymorphic types, that is something like

decode(expr anycompatible,
       search1 anycompatible, result1 anycompatible2,
       search2 anycompatible, result2 anycompatible2,
       search3 anycompatible, result3 anycompatible2,
       ...
) returns anycompatible2;

Admittedly, you'd need to write a separate declaration for each number of
arguments you wanted to support, but they could all point at the same C
function --- which'd be a lot simpler than in this patch, since it would
not need to deal with any type coercions, only comparisons.

I also argue that to the extent that the support function is reinventing
polymorphism internally, it's going to be inferior to the parser's
version.  As an example, with Pavel's sample implementation, if a
particular query needs a coercion from type X to type Y, that's nowhere
visible in the parse tree.  So you could drop the cast without being told
that view so-and-so depends on it, leading to a run-time failure next time
you try to use that view.  Doing the same thing with normal polymorphism,
the X-to-Y cast function would be used in the parse tree and so we'd know
about the dependency.

2. I have no faith that the proposed implementation is correct or
complete.  As I complained earlier, a lot of places have special-case
handling for polymorphism, and it seems like every one of them would
need to know about this feature too.  That is, to the extent that
this patch's footprint is smaller than commit 24e2885ee -- which it
is, by a lot -- I think those are bugs of omission.  It will not work
to have a situation where some parts of the backend resolve a function's
result type as one thing and others resolve it as something else thanks to
failure to account for this new feature.  As a concrete example, it looks
like we'd fail pretty hard if someone tried to use this facility in an
aggregate support function.

So my opinion is still what it was in January.

                        regards, tom lane

[1] https://www.postgresql.org/message-id/31501.1579036195%40sss.pgh.pa.us