Обсуждение: [HACKERS] oversight in EphemeralNamedRelation support
Hi, Hugo Mercier (in Cc) reported me an error in a query, failing since pg10. Simple test case to reproduce: CREATE TABLE public.test (id integer); WITH test AS (select 42) INSERT INTO public.test SELECT * FROM test; which will fail with "relation "test" cannot be the target of a modifying statement". IIUC, that's an oversight in 18ce3a4ab22, where setTargetTable() doesn't exclude qualified relation when searching for special relation. PFA a simple patch to fix this issue, with updated regression test. Regards. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Вложения
On Tue, Oct 10, 2017 at 2:35 AM, Julien Rouhaud <rjuju123@gmail.com> wrote: > Hugo Mercier (in Cc) reported me an error in a query, failing since pg10. > > Simple test case to reproduce: > > CREATE TABLE public.test (id integer); > WITH test AS (select 42) INSERT INTO public.test SELECT * FROM test; > > which will fail with "relation "test" cannot be the target of a > modifying statement". > > IIUC, that's an oversight in 18ce3a4ab22, where setTargetTable() > doesn't exclude qualified relation when searching for special > relation. I agree. > PFA a simple patch to fix this issue, with updated regression test. Thanks! I suppose we could consider moving the schemaname check into getRTEForSpecialRelationType(), since otherwise both callers need to do that (and as you discovered, one forgot). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Oct 9, 2017 at 10:43 PM, Thomas Munro <thomas.munro@enterprisedb.com> wrote: > > I suppose we could consider moving the schemaname check into > getRTEForSpecialRelationType(), since otherwise both callers need to > do that (and as you discovered, one forgot). Thanks for the feedback. That was my first idea, but I assumed there could be future use for this function on qualified RangeVar if it wasn't done this way. I agree it'd be much safer, so v2 attached, check moved in getRTEForSpecialRelationType(). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Вложения
Julien Rouhaud <rjuju123@gmail.com> writes:
> On Mon, Oct 9, 2017 at 10:43 PM, Thomas Munro
> <thomas.munro@enterprisedb.com> wrote:
>> I suppose we could consider moving the schemaname check into
>> getRTEForSpecialRelationType(), since otherwise both callers need to
>> do that (and as you discovered, one forgot).
> Thanks for the feedback. That was my first idea, but I assumed there
> could be future use for this function on qualified RangeVar if it
> wasn't done this way.
> I agree it'd be much safer, so v2 attached, check moved in
> getRTEForSpecialRelationType().
Hm. I actually think the bug here is that 18ce3a4ab introduced
anything into setTargetTable at all. There was never previously
any assumption that the target could be anything but a regular
table, so we just ignored CTEs there, and I do not think the
new behavior is an improvement.
So my proposal is to rip out the getRTEForSpecialRelationTypes
check there. I tend to agree that getRTEForSpecialRelationTypes
should probably contain an explicit check for unqualified name
rather than relying on its caller ... but that's a matter of
future-proofing not a bug fix.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Oct 13, 2017 at 8:50 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Julien Rouhaud <rjuju123@gmail.com> writes: >> On Mon, Oct 9, 2017 at 10:43 PM, Thomas Munro >> <thomas.munro@enterprisedb.com> wrote: >>> I suppose we could consider moving the schemaname check into >>> getRTEForSpecialRelationType(), since otherwise both callers need to >>> do that (and as you discovered, one forgot). > >> Thanks for the feedback. That was my first idea, but I assumed there >> could be future use for this function on qualified RangeVar if it >> wasn't done this way. > >> I agree it'd be much safer, so v2 attached, check moved in >> getRTEForSpecialRelationType(). > > Hm. I actually think the bug here is that 18ce3a4ab introduced > anything into setTargetTable at all. There was never previously > any assumption that the target could be anything but a regular > table, so we just ignored CTEs there, and I do not think the > new behavior is an improvement. > > So my proposal is to rip out the getRTEForSpecialRelationTypes > check there. I tend to agree that getRTEForSpecialRelationTypes > should probably contain an explicit check for unqualified name > rather than relying on its caller ... but that's a matter of > future-proofing not a bug fix. That check arrived in v11 revision of the patch: https://www.postgresql.org/message-id/CACjxUsPfUUa813oDvJRx2wuiqHXO3VsCLQzcuy0r%3DUEyS-xOjQ%40mail.gmail.com Before that, CTE used as modify targets produced a different error message: postgres=# WITH d AS (SELECT 42) INSERT INTO d VALUES (1); ERROR: relation "d" does not exist LINE 1: WITH d AS (SELECT 42) INSERT INTO d VALUES (1); ^ ... but ENRs used like that caused a crash. The change to setTargetTable() went in to prevent that (and improved the CTE case's error message semi-incidentally). To take out we'll need a new check somewhere else to prevent that. Where? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Fri, Oct 13, 2017 at 8:50 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Hm. I actually think the bug here is that 18ce3a4ab introduced
>> anything into setTargetTable at all. There was never previously
>> any assumption that the target could be anything but a regular
>> table, so we just ignored CTEs there, and I do not think the
>> new behavior is an improvement.
> Before that, CTE used as modify targets produced a different error message:
> postgres=# WITH d AS (SELECT 42) INSERT INTO d VALUES (1);
> ERROR: relation "d" does not exist
> LINE 1: WITH d AS (SELECT 42) INSERT INTO d VALUES (1);
> ^
Well, I think that is a poorly chosen example. Consider this instead:
pre-v10, you could do this:
regression=# create table mytable (f1 int);
CREATE TABLE
regression=# with mytable as (select 1 as x) insert into mytable values(1);
INSERT 0 1
regression=# select * from mytable;f1
---- 1
(1 row)
The CTE was simply not part of the available namespace for the INSERT's
target, so it found the regular table instead. v10 has thus broken
cases that used to work. I think that's a bug.
There may or may not be a case for allowing ENRs to capture names that
would otherwise refer to ordinary tables; I'm not sure. But I see very
little case for allowing CTEs to capture such references, because surely
we are never going to allow that to do anything useful, and we have
several years of precedent now that they don't capture.
I think we need to either remove that call from setTargetTable entirely,
or maybe adjust it so it can only find ENRs not CTEs.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Oct 13, 2017 at 10:01 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Thomas Munro <thomas.munro@enterprisedb.com> writes: >> Before that, CTE used as modify targets produced a different error message: > >> postgres=# WITH d AS (SELECT 42) INSERT INTO d VALUES (1); >> ERROR: relation "d" does not exist >> LINE 1: WITH d AS (SELECT 42) INSERT INTO d VALUES (1); >> ^ > > Well, I think that is a poorly chosen example. Consider this instead: > pre-v10, you could do this: > > regression=# create table mytable (f1 int); > CREATE TABLE > regression=# with mytable as (select 1 as x) insert into mytable values(1); > INSERT 0 1 > regression=# select * from mytable; > f1 > ---- > 1 > (1 row) > > The CTE was simply not part of the available namespace for the INSERT's > target, so it found the regular table instead. v10 has thus broken > cases that used to work. I think that's a bug. Hmm. Yeah. I have to say it's a bit surprising that the following refers to two different objects: with mytable as (select 1 as x) insert into mytable select * from mytable Obviously the spec is useless here since this is non-standard (at a guess they'd probably require a qualifier there to avoid parsing as a <query name> if they allowed DML after <with clause>). As you said it's worked like that for several releases, so whatever I might think about someone who deliberately writes such queries, the precedent probably trumps naive notions about WITH creating a single consistent lexical scope. > There may or may not be a case for allowing ENRs to capture names that > would otherwise refer to ordinary tables; I'm not sure. But I see very > little case for allowing CTEs to capture such references, because surely > we are never going to allow that to do anything useful, and we have > several years of precedent now that they don't capture. > > I think we need to either remove that call from setTargetTable entirely, > or maybe adjust it so it can only find ENRs not CTEs. I think it'd be better to find and reject ENRs only. The alternative would be to make ENRs invisible to DML, which seems arbitrary and weird (even though it might be more consistent with our traditional treatment of CTEs). One handwavy reason I'd like them to remain visible to DML (and be rejected) is that I think they're a bit like table variables (see SQL Server), and someone might eventually want to teach them, or something like them, how to be writable. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Fri, Oct 13, 2017 at 10:01 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> The CTE was simply not part of the available namespace for the INSERT's
>> target, so it found the regular table instead. v10 has thus broken
>> cases that used to work. I think that's a bug.
> Hmm. Yeah. I have to say it's a bit surprising that the following
> refers to two different objects:
> with mytable as (select 1 as x) insert into mytable select * from mytable
Yeah, I agree --- personally I'd never write a query like that. But
the fact that somebody ran into it when v10 has been out for barely
a week suggests that people are doing it.
>> I think we need to either remove that call from setTargetTable entirely,
>> or maybe adjust it so it can only find ENRs not CTEs.
> I think it'd be better to find and reject ENRs only. The alternative
> would be to make ENRs invisible to DML, which seems arbitrary and
> weird (even though it might be more consistent with our traditional
> treatment of CTEs). One handwavy reason I'd like them to remain
> visible to DML (and be rejected) is that I think they're a bit like
> table variables (see SQL Server), and someone might eventually want to
> teach them, or something like them, how to be writable.
Yeah, that would be the argument for making them visible. I'm not
sure how likely it is that we'll ever actually have writable ENRs,
but I won't stand in the way if you want to do it like that.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Oct 13, 2017 at 12:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Thomas Munro <thomas.munro@enterprisedb.com> writes: >> On Fri, Oct 13, 2017 at 10:01 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> The CTE was simply not part of the available namespace for the INSERT's >>> target, so it found the regular table instead. v10 has thus broken >>> cases that used to work. I think that's a bug. > >> Hmm. Yeah. I have to say it's a bit surprising that the following >> refers to two different objects: >> with mytable as (select 1 as x) insert into mytable select * from mytable > > Yeah, I agree --- personally I'd never write a query like that. But > the fact that somebody ran into it when v10 has been out for barely > a week suggests that people are doing it. Not exactly -- Julien's bug report was about a *qualified* reference being incorrectly rejected. >>> I think we need to either remove that call from setTargetTable entirely, >>> or maybe adjust it so it can only find ENRs not CTEs. > >> I think it'd be better to find and reject ENRs only. The alternative >> would be to make ENRs invisible to DML, which seems arbitrary and >> weird (even though it might be more consistent with our traditional >> treatment of CTEs). One handwavy reason I'd like them to remain >> visible to DML (and be rejected) is that I think they're a bit like >> table variables (see SQL Server), and someone might eventually want to >> teach them, or something like them, how to be writable. > > Yeah, that would be the argument for making them visible. I'm not > sure how likely it is that we'll ever actually have writable ENRs, > but I won't stand in the way if you want to do it like that. I hope so :-) I might be a nice way to get cheap locally scoped temporary tables, among other things. Okay, here's Julien's patch tweaked to reject just the ENR case. This takes us back to the 9.6 behaviour where CTEs don't hide tables in this context. I also removed the schema qualification in his regression test so we don't break that again. This way, his query from the first message in the thread works with the schema qualification (the bug he reported) and without it (the bug or at least incompatible change from 9.6 you discovered). I considered testing for a NULL return from parserOpenTable() instead of the way the patch has it, since parserOpenTable() already had an explicit test for ENRs, but its coding would give preference to an unqualified table of the same name. I considered moving the test for an ENR match higher up in parserOpenTable(), and that might have been OK, but then I realised no code in the tree actually tests for its undocumented NULL return value anyway. I think that NULL-returning branch is dead code, and all tests pass without it. Shouldn't we just remove it, as in the attached? I renamed the ENR used in plpgsql.sql's transition_table_level2_bad_usage_func() and with.sql's "sane response" test, because they both confused matters by using an ENR with the name "d" which is also the name of an existing table. For example, if you start with unpatched master, rename transition_table_level2_bad_usage_func()'s ENR to "dx" and simply remove the check for ENRs from setTargetTable() as you originally suggested, you'll get a segfault because the NULL return from parserOpenTable() wasn't checked. If you leave transition_table_level2_bad_usage_func()'s ENR name as "d" it'll quietly access the wrong table instead, which is misleading. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Вложения
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Fri, Oct 13, 2017 at 12:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Yeah, I agree --- personally I'd never write a query like that. But
>> the fact that somebody ran into it when v10 has been out for barely
>> a week suggests that people are doing it.
> Not exactly -- Julien's bug report was about a *qualified* reference
> being incorrectly rejected.
Nonetheless, he was using a CTE name equivalent to the name of the
query's target table. That's already confusing IMV ... and it does
not seem unreasonable to guess that he only qualified the target
because it stopped working unqualified.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Oct 13, 2017 at 5:22 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Thomas Munro <thomas.munro@enterprisedb.com> writes: >> On Fri, Oct 13, 2017 at 12:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Yeah, I agree --- personally I'd never write a query like that. But >>> the fact that somebody ran into it when v10 has been out for barely >>> a week suggests that people are doing it. > >> Not exactly -- Julien's bug report was about a *qualified* reference >> being incorrectly rejected. > > Nonetheless, he was using a CTE name equivalent to the name of the > query's target table. That's already confusing IMV ... and it does > not seem unreasonable to guess that he only qualified the target > because it stopped working unqualified. FWIW, the original (and much more complex) query Hugo sent me was inserting data in a qualified table name (the schema wasn't public, and I assume not in his search_path). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Oct 13, 2017 at 10:01 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > But I see very > little case for allowing CTEs to capture such references, because surely > we are never going to allow that to do anything useful, and we have > several years of precedent now that they don't capture. For what it's worth, SQL Server allows DML in CTEs like us but went the other way on this. Not only are its CTEs in scope as DML targets, it actually lets you update them in cases where a view would be updatable, rewriting as base table updates. I'm not suggesting that we should do that too (unless of course it shows up in a future standard), just pointing it out as a curiosity. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Fri, Oct 13, 2017 at 10:01 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> But I see very
>> little case for allowing CTEs to capture such references, because surely
>> we are never going to allow that to do anything useful, and we have
>> several years of precedent now that they don't capture.
> For what it's worth, SQL Server allows DML in CTEs like us but went
> the other way on this. Not only are its CTEs in scope as DML targets,
> it actually lets you update them in cases where a view would be
> updatable, rewriting as base table updates. I'm not suggesting that
> we should do that too (unless of course it shows up in a future
> standard), just pointing it out as a curiosity.
Interesting. Still, given that we have quite a few years of precedent
that CTEs aren't in scope as DML targets, I'm disinclined to change
our semantics unless the point does show up in the standard.
I've not heard anyone speaking against the choices you made in your
prior message, so I'll go review your v3 patch, and push unless
I find problems.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers