Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

Поиск
Список
Период
Сортировка
Искать

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Kevin Grittner <kgrittn@ymail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Kevin Grittner <kgrittn@ymail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Alvaro Herrera <alvherre@2ndquadrant.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost  writes:
> > I've uploaded the latest patch, rebased against master, with my changes
> > to here: http://snowman.net/~sfrost/rls_ringerc_sf.patch.gz as I don't
> > believe it'd clear the mailing list (it's 29k).
> 
> Please actually post it, for the archives' sake.  29k is far below the
> list limit.  (Which I don't know exactly what it is ... but certainly
> in the hundreds of KB.)

Huh, thought it was more like 25k.  Well, here goes then...

> > I'll take a look at changing the cache key to include user ID and
> > ripping out the plan invalidation logic from the current patch tomorrow
> > but I seriously doubt I'll be able to get all of that done in the next
> > day or two.
> 
> TBH I think we are up against the deadline.  April 15 was the agreed-to
> drop dead date for pushing new features into 9.4.

Yeah. :/  May be for the best anyway, this should be able to go in early
in the 9.5 cycle and get more testing and refinement.  Still stinks
though as I feel like this patch didn't get the attention it should have
due to a simple misunderstanding, but we do need to stop at some point
to get a release together.

	Thanks,

		Stephen

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Alvaro Herrera <alvherre@2ndquadrant.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Alvaro Herrera <alvherre@2ndquadrant.com>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:
All,

* Brightwell, Adam (adam.brightwell@crunchydatasolutions.com) wrote:
> Attached is a updated patch taking into account the recommendations
> provided.

  Alright, attached is a patch which I've been over in a great deal more
  detail, as we seem to have moved beyond grammar and simple
  functionality.  It's been much reworked and improved (particularly in
  rewrite/rowsecurity.c, but also commands/policy.c).  Other
  improvements of note (not including the improvements made and
  mentioned by Adam previously):

  Lots of additional comments around what's happening
  Improved SGML documentation 
  Better \d and \dp support
  Explicit function for check row-security requirements
  Correct handling for views run under policies
  Simplified changes to copy.c
  Use normal DROP and RENAME processes (eg: DropStmt and friends)
  Default-deny policy implementation, and regression tests
  Handle sub-queries in WITH CHECK
  Avoid duplicate policy application
  Corrected plancache invalidation
  Improved and additional regression tests
  tab completion

  This addresses all of the comments brought up previously, as far as
  I'm aware, along with quite a few other issues which I found while
  doing my review and rework.

  As always- testing, reviews, comments are welcome.  We've done a fair
  bit of testing internally, but it's great to see how others are
  imaginging and trying to use new capabilities like these- especially
  if they run into any problems! :)

  This took quite a bit longer than I had expected, but I think the
  rework, review and additional testing was well worth it.

  I'm planning to break from this for a few days and resume helping with
  the commitfest more-or-less full-time until I have to head out for
  PostgresOpen.

  	Thanks!

		Stephen

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:
* Robert Haas (robertmhaas@gmail.com) wrote:
> On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost  wrote:
> > If we want to be able to disable RLS w/o dropping the policies, then I
> > think we have to completely de-couple the two and users would then have
> > both add policies AND turn on RLS to have RLS actually be enabled for a
> > given table.  I'm on the fence about that.
> >
> > Thoughts?
> 
> A strong +1 for doing just that.

Alright, updated patch attached which does just that (thanks to Adam
for the updates for this and testing pg_dump- I just reviewed it and
added some documentation updates and other minor improvements), and
rebased to master.  Also removed the catversion bump, so it should apply
cleanly for people, for a while anyway.

	Thanks!

		Stephen

Re: RLS Design

От:
Andres Freund <andres@2ndquadrant.com>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Andres Freund <andres@2ndquadrant.com>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: RLS Design

От:
"Erik Rijkers" <er@xs4all.nl>
Дата:
On Wed, September 10, 2014 23:50, Stephen Frost wrote:
>  [rls_9-10-2014.patch]

I can't get this to apply; I attach the complaints of patch.


Erik Rijkers





API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Gregory Smith <gregsmithpgsql@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: RLS Design

От:
Yeb Havinga <yebhavinga@gmail.com>
Дата:

Re: RLS Design

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: RLS Design

От:
Craig Ringer <craig@2ndquadrant.com>
Дата:

Re: RLS Design

От:
Josh Berkus <josh@agliodbs.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: RLS Design

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:

Re: RLS Design

От:
Kouhei Kaigai <kaigai@ak.jp.nec.com>
Дата:

Re: RLS Design

От:
Kouhei Kaigai <kaigai@ak.jp.nec.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Robert Haas <robertmhaas@gmail.com>
Дата:

Re: RLS Design

От:
Thom Brown <thom@linux.com>
Дата:
On 19 September 2014 17:32, Stephen Frost <sfrost@snowman.net> wrote:
Thom,

Thanks!

* Thom Brown (thom@linux.com) wrote:
> On 14 September 2014 16:38, Stephen Frost <sfrost@snowman.net> wrote:
> # create policy visible_colours on colours for all to joe using (visible =
> true);
> CREATE POLICY
[...]
> > insert into colours (name, visible) values ('transparent',false);
> ERROR:  new row violates WITH CHECK OPTION for "colours"
> DETAIL:  Failing row contains (7, transparent, f).
>
> > select * from pg_policies ;
>    policyname    | tablename | roles | cmd |       qual       | with_check
> -----------------+-----------+-------+-----+------------------+------------
>  visible_colours | colours   | {joe} | ALL | (visible = true) |
> (1 row)
>
> There was no WITH CHECK OPTION.

As I hope is clear if you look at the documentation- if the WITH CHECK
clause is omitted, then the USING clause is used for both filtering and
checking new records, otherwise you'd be able to add records which
aren't visible to you.

I can see that now, although I do find the error message somewhat confusing.  Firstly, it looks like "OPTION" is part of the parameter name, which it isn't.

Also, I seem to get an error message with the following:

# create policy nice_colours ON colours for all to joe using (visible = true) with check (name in ('blue','green','yellow'));
CREATE POLICY

\c - joe

> insert into colours (name, visible) values ('blue',false);
ERROR:  function with OID 0 does not exist

And if this did work, but I only violated the USING clause, would this still say the WITH CHECK clause was the cause?

Thom

Re: RLS Design

От:
Thom Brown <thom@linux.com>
Дата:
On 14 September 2014 16:38, Stephen Frost <sfrost@snowman.net> wrote:
* Robert Haas (robertmhaas@gmail.com) wrote:
> On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > If we want to be able to disable RLS w/o dropping the policies, then I
> > think we have to completely de-couple the two and users would then have
> > both add policies AND turn on RLS to have RLS actually be enabled for a
> > given table.  I'm on the fence about that.
> >
> > Thoughts?
>
> A strong +1 for doing just that.

Alright, updated patch attached which does just that (thanks to Adam
for the updates for this and testing pg_dump- I just reviewed it and
added some documentation updates and other minor improvements), and
rebased to master.  Also removed the catversion bump, so it should apply
cleanly for people, for a while anyway.

This is testing what has been committed:

# create table colours (id serial, name text, visible boolean);
CREATE TABLE

# insert into colours (name, visible) values ('blue',true),('yellow',true),('ultraviolet',false),('green',true),('infrared',false);
INSERT 0 5

# create policy visible_colours on colours for all to joe using (visible = true);
CREATE POLICY

# grant all on colours to public;
GRANT

# grant all on sequence colours_id_seq to public;
GRANT

# alter table colours enable row level security ;
ALTER TABLE

\c - joe

> select * from colours;
 id |  name  | visible 
----+--------+---------
  1 | blue   | t
  2 | yellow | t
  4 | green  | t
(3 rows)

> insert into colours (name, visible) values ('purple',true);
INSERT 0 1

> insert into colours (name, visible) values ('transparent',false);
ERROR:  new row violates WITH CHECK OPTION for "colours"
DETAIL:  Failing row contains (7, transparent, f).

> select * from pg_policies ;
   policyname    | tablename | roles | cmd |       qual       | with_check 
-----------------+-----------+-------+-----+------------------+------------
 visible_colours | colours   | {joe} | ALL | (visible = true) | 
(1 row)


There was no WITH CHECK OPTION.

--
Thom

Re: RLS Design

От:
Thom Brown <thom@linux.com>
Дата:
On 19 September 2014 17:54, Stephen Frost  wrote:
>
> Thom,
>
> * Thom Brown (thom@linux.com) wrote:
> > On 19 September 2014 17:32, Stephen Frost  wrote:
> > > * Thom Brown (thom@linux.com) wrote:
> > > > On 14 September 2014 16:38, Stephen Frost  wrote:
> > > > # create policy visible_colours on colours for all to joe using (visible
> > > =
> > > > true);
> > > > CREATE POLICY
> > > [...]
> > > > > insert into colours (name, visible) values ('transparent',false);
> > > > ERROR:  new row violates WITH CHECK OPTION for "colours"
> > > > DETAIL:  Failing row contains (7, transparent, f).
> > > >
> > > > > select * from pg_policies ;
> > > >    policyname    | tablename | roles | cmd |       qual       |
> > > with_check
> > > >
> > > -----------------+-----------+-------+-----+------------------+------------
> > > >  visible_colours | colours   | {joe} | ALL | (visible = true) |
> > > > (1 row)
> > > >
> > > > There was no WITH CHECK OPTION.
> > >
> > > As I hope is clear if you look at the documentation- if the WITH CHECK
> > > clause is omitted, then the USING clause is used for both filtering and
> > > checking new records, otherwise you'd be able to add records which
> > > aren't visible to you.
> >
> > I can see that now, although I do find the error message somewhat
> > confusing.  Firstly, it looks like "OPTION" is part of the parameter name,
> > which it isn't.
>
> Hmm, the notion of 'with check option' is from the SQL standard, which
> is why I felt the error message was appropriate as-is..
>
> > Also, I seem to get an error message with the following:
> >
> > # create policy nice_colours ON colours for all to joe using (visible =
> > true) with check (name in ('blue','green','yellow'));
> > CREATE POLICY
> >
> > \c - joe
> >
> > > insert into colours (name, visible) values ('blue',false);
> > ERROR:  function with OID 0 does not exist
>
> Now *that* one is interesting and I'll definitely go take a look at it.
> We added quite a few regression tests to try and make sure these things
> work.
>
> > And if this did work, but I only violated the USING clause, would this
> > still say the WITH CHECK clause was the cause?
>
> WITH CHECK applies for INSERT and UPDATE for the new records going into
> the table.  You can't actually violate the USING clause for an INSERT
> as USING is for filtering records, not checking that records being added
> to the table are valid.
>
> To try and clarify- by explicitly setting both USING and WITH CHECK, you
> *are* able to INSERT records which are not visible to you.  We felt that
> was an important capability to support.

I find it a bit of a limitation that I can't specify both INSERT and
UPDATE for a policy.  I'd want to be able to specify something like
this:

CREATE POLICY no_greys_allowed
  ON colours
  FOR INSERT, UPDATE
  WITH CHECK (name NOT IN ('grey','gray'));

I would expect this to be rather common to prevent certain values
making their way into a table.  Instead I'd have to create 2 policies
as it stands.



In order to debug issues with accessing table data, perhaps it would
be useful to output the name of the policy that was violated.  If a
table had 20 policies on, it could become time-consuming to debug.



I keep getting tripped up by overlapping policies.  On the one hand, I
created a policy to ensure rows being added or selected have a
"visible" column set to true.  On the other hand, I have a policy that
ensures that the name of a colour doesn't appear in a list.  Policy 1
is violated until policy 2 is added:

(using the table I created in a previous post on this thread...)

# create policy must_be_visible ON colours for all to joe using
(visible = true) with check (visible = true);
CREATE POLICY

\c - joe

> insert into colours (name, visible) values ('pink',false);
ERROR:  new row violates WITH CHECK OPTION for "colours"
DETAIL:  Failing row contains (28, pink, f).

\c - thom

# create policy no_greys_allowed on colours for insert with check
(name not in ('grey','gray'));
CREATE POLICY

\c - joe

# insert into colours (name, visible) values ('pink',false);
INSERT 0 1

I expected this to still trigger an error due to the first policy.  Am
I to infer from this that the policy model is permissive rather than
restrictive?


I've also attached a few corrections for the docs.

Thom

Re: RLS Design

От:
Thom Brown <thom@linux.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Adam Brightwell <adam.brightwell@crunchydatasolutions.com>
Дата:
Hi all,

This is my first post to the mailing list and I am looking forward to working with everyone in the community.

With that said...

I'll take a look at changing the cache key to include user ID and
ripping out the plan invalidation logic from the current patch tomorrow
but I seriously doubt I'll be able to get all of that done in the next
day or two.  If anyone else is able to help out, it'd certainly be
appreciated; I really think that's the main hurdle to address at this
point with this patch- without the plan invalidation complexity, the
the patch is really just building out the catalog, the SQL-level
statements for managing it, and the bit of code required to add the
conditional to statements involving RLS-enabled tables.

I have been collaborating with Stephen on addressing this particular item with RLS.

As a basis, I have been working with Craig's 'rls-9.4-upd-sb-views' branch rebased against master around 9.4beta1.

Through this effort, we have concluded that for RLS the case of invalidating a plan is only necessary when switching between a superuser and a non-superuser.  Obviously, re-planning on every role change would be too costly, but this approach should help minimize that cost.  As well, there were not any cases outside of this one that were immediately apparent with respect to RLS that would require re-planning on a per userid basis.

I have tested this approach with the following patch.


Does this sound like a sane approach?  Thoughts?  Recommendations?

Thanks,
Adam

Re: RLS Design

От:
Kohei KaiGai <kaigai@kaigai.gr.jp>
Дата:

Re: RLS Design

От:
Kohei KaiGai <kaigai@kaigai.gr.jp>
Дата:

Re: RLS Design

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: RLS Design

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: RLS Design

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: RLS Design

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: RLS Design

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Dean Rasheed <dean.a.rasheed@gmail.com>
Дата:

Re: RLS Design

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
All,

Attached is a patch for RLS that incorporates the following changes:

* Syntax:
  - CREATE POLICY <policy_name> ON <table_name> FOR <command> USING ( <qual> )
  - ALTER POLICY <policy_name> ON <table_name> FOR <command> USING ( <qual> )
  - DROP POLICY <policy_name> ON <table_name> FOR <command>

* "row_security" GUC Setting - enable/disable row level security.

* BYPASSRLS and NOBYPASSRLS role attribute - allows user to bypass RLS if row_security GUC is set to OFF.

There are still some remaining issues but we hope to have those resolved soon.

Any comments or suggestions would be greatly appreciated.

Thanks,
Adam


On Mon, Jul 21, 2014 at 11:38 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jul 18, 2014 at 7:01 PM, Brightwell, Adam
<adam.brightwell@crunchydatasolutions.com> wrote:
>> I think we do want a way to modify policies.  However, we tend to
>> avoid syntax that involves unnatural word order, as this certainly
>> does.  Maybe it's better to follow the example of CREATE RULE and
>> CREATE TRIGGER and do something this instead:
>>
>> CREATE POLICY policy_name ON table_name USING quals;
>> ALTER POLICY policy_name ON table_name USING quals;
>> DROP POLICY policy_name ON table_name;
>>
>> The advantage of this is that you can regard "policy_name ON
>> table_name" as the identifier for the policy throughout the system.
>> You need some kind of identifier of that sort anyway to support
>> COMMENT ON, SECURITY LABEL, and ALTER EXTENSION ADD/DROP for policies.
>
> Sounds good.  I certainly think it makes a lot of sense to include the ALTER
> functionality, if for no other reason than ease of use.
>
> Another item to consider, though I believe it can come later, is per-action
> policies.  Following the above suggested syntax, perhaps that might look
> like the following?
>
> CREATE POLICY policy_name ON table_name FOR action USING quals;
> ALTER POLICY policy_name ON table_name FOR action USING quals;
> DROP POLICY policy_name ON table_name FOR action;

That seems reasonable.  You need to give some thought to what happens
if the user types:

CREATE POLICY pol1 ON tab1 FOR SELECT USING q1;
ALTER POLICY pol1 ON tab1 FOR INSERT USING q2;

I guess you end up with q1 as the SELECT policy and q2 as the INSERT
policy.  Similarly, had you typed:

CREATE POLICY pol1 ON tab1 USING q1;
ALTER POLICY pol1 ON tab1 FOR INSERT USING q2;

...then I guess you end up with q2 for INSERTs and q1 for everything
else.  I'm wondering if it might be better, though, not to allow the
quals to be specified in CREATE POLICY, or else to allow multiple
actions.  Otherwise, getting pg_dump to DTRT might be complicated.

Perhaps:

CREATE POLICY pol1 ON tab1 ( [ [ FOR operation [ OR operation ] ... ]
USING quals ] ... );
where operation = SELECT | INSERT | UPDATE | DELETE

So that you can write things like:

CREATE POLICY pol1 ON tab1 (USING a = 1);
CREATE POLICY pol2 ON tab2 (FOR INSERT USING a = 1, FOR UPDATE USING b
= 1, FOR DELETE USING c = 1);

And then, for ALTER, just allow one change at a time, syntax as you
proposed.  That way each policy can be dumped as a single CREATE
statement.

> I was also giving some thought to the use of "POLICY", perhaps I am wrong,
> but it does seem it could be at risk of becoming ambiguous down the road.  I
> can't think of any specific examples at the moment, but my concern is what
> happens if we wanted to add another "type" of policy, whatever that might
> be, later?  Would it make more sense to go ahead and qualify this a little
> more with "ROW SECURITY POLICY"?

I think that's probably over-engineering.  I'm not aware of anything
else we might add that would be likely to be called a policy, and if
we did add something we could probably call it something else instead.
And long command names are annoying.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



--

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
Robert,

However, I believe that
there has been a lack of focus in the development of the patch thus
far in a couple of key areas - first in terms of articulating how it
is different from and better than a writeable security barrier view,
and second on how to manage the security and operational aspects of
having a feature like this.  I think that the discussion subsequent to
my June 10th email has let to some good discussion on both points,
which was my intent, but I still think much more time and thought
needs to be spent on those issues if we are to have a feature which is
up to our usual standards.  I do apologize to anyone who interpreted
that initial as a pure rant, because it really wasn't intended that
way.  Contrariwise, I hope that the people defending this patch will
admit that the issues I am raising are real and focus on whether and
how those concerns can be addressed.

I absolutely appreciate all of the feedback that has been provided.  It has been educational.  To your point above, I started putting together a wiki page, as Stephen has spoken to, that is meant to capture these concerns and considerations as well as to capture ideas around solutions.  


This page is obviously not complete, but I think it is a good start. Hopefully this document will help to continue the conversation and assist in addressing all the concerns that have been brought to the table.  As well, I hope that this document serves to demonstrate our intent and that we *are* taking these concerns seriously.  I assure you that as one of the individuals who is working towards the acceptance of this feature/patch, I am very much concerned about meeting the expected standards of quality and security. 

Thanks,
Adam 

Re: RLS Design

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
All,

Attached is a updated patch taking into account the recommendations provided.

This patch created against master at ad5d46a4494b0b480a3af246bb4227d9bdadca37

The following items have been addressed:

* Add ALTER TABLE <name> { ENABLE | DISABLE } ROW LEVEL SECURITY - set flag on table to allow for a default-deny capability.  If RLS is enabled on a table and has no policies, then a default-deny policy is automatically applied.  If RLS is disabled on table and the table still has policies on it then then an error is raised.  Though if DISABLE is accompanied with CASCADE, then all policies will be removed and no error is raised.

* Update CREATE POLICY to include WITH CHECK ( <expression> ).  Therefore, the syntax is now as follows:
   CREATE POLICY <name> ON <table>
   [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
   [ USING ( <expression> ) ]
   [ WITH CHECK ( <expression> ) ]

A WITH CHECK expression is required for creating an INSERT policy and is optional on UPDATE and ALL.  The intended purpose is to provide a VIEW-like WITH CHECK OPTION functionality to RLS. 

* Add ALTER POLICY <name> ON <table> RENAME TO <new_name> - renames a policy.

* Updated GUC row_security to allow ON | OFF | FORCE.  Each option breaks down as follows:
    - ON - RLS is appled to all roles except the table owner and superusers.
    - OFF - RLS can be bypassed, but only by roles with BYPASSRLS.  If the roles does not have BYPASSRLS, then an error is raised.
    - FORCE - RLS is applied to all roles, regardless of ownership, superuser or BYPASSRLS.

* Removed SET ROW SECURITY { ON | OFF } as requested.

* Removed all GetConfigOption for "row_security" GUC.

* Removed setting row_security GUC to OFF in SET SESSION/SET ROLE for superuser.

* Add psql \dp support.  Displays RLS information in new column "Policies".

* Updated documentation.

* Other cleanup and improvements.

There are still some minor issues being worked through, however, it is expected that those will be resolved soon.  However, any feedback, comments or suggestions on the above and in general would be greatly appreciated.

Thanks,
Adam


On Wed, Sep 3, 2014 at 10:17 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Aug 29, 2014 at 8:16 PM, Brightwell, Adam
<adam.brightwell@crunchydatasolutions.com> wrote:
> Attached is a patch for RLS that was create against master at
> 01363beae52700c7425cb2d2452177133dad3e93 and is ready for review.
>
> Overview:
>
> This patch provides the capability to create multiple named row level
> security policies for a table on a per command basis and assign them to be
> applied to specific roles/users.
>
> It contains the following changes:
>
> * Syntax:
>
> CREATE POLICY <name> ON <table>
>     [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
>     [ TO { PUBLIC | <role> [, <role> ] } ]
>     USING (<condition>)
>
> Creates a RLS policy named <name> on <table>.  Specifying a command is
> optional, but the default is ALL.  Specifying a role is options, but the
> default is PUBLIC.  If PUBLIC and other roles are specified, ONLY PUBLIC is
> applied and a warning is raised.
>
> ALTER POLICY <name> ON <table>
>     [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
>     [ TO { PUBLIC | <role> [, <role> ] } ]
>     USING (<condition>)
>
> Alter a RLS policy named <name> on <table>.  Specifying a command is
> optional, if provided then the policy's command is changed otherwise it is
> left as-is.  Specifying a role is optional, if provided then the policy's
> role is changed otherwise it is left as-is.  The <condition> must always be
> provided and is therefore always replaced.

This is not a full review of this patch; as we're mid-CommitFest, I
assume this will get added to the next CommitFest.

In earlier discussions, it was proposed (and I thought the proposal
was viewed favorably) that when enabling row-level security for a
table (i.e. before doing CREATE POLICY), you'd have to first flip the
table to a default-deny mode:

ALTER TABLE <name> ENABLE ROW LEVEL SECURITY;

In this design, I'm not sure what happens when there are policies for
some but not all users or some but not all actions.  Does creating a
INSERT policy for one particular user cause a default-deny policy to
be turned on for all other users and all other operations?  That might
be OK, but at the very least it should be documented more clearly.
Does dropping the very last policy then instantaneously flip the table
back to default-allow?

As far as I can tell from the patch, and that's not too far since I've
only looked at briefly, there's a default-deny policy only if there is
at least 1 policy that applies to your user ID for this operation.  As
far as making it easy to create a watertight combination of policies,
that seems like a bad plan.

+         elog(ERROR, "Table \"%s\" already has a policy named \"%s\"."
+             " Use a different name for the policy or to modify this policy"
+             " use ALTER POLICY %s ON %s USING (qual)",
+             RelationGetRelationName(target_table), stmt->policy_name,
+             RelationGetRelationName(target_table), stmt->policy_name);
+

That needs to be an ereport, be capitalized properly, and the hint, if
it's to be included at all, needs to go into errhint().

+                          errhint("all roles are considered members
of public")));

Wrong message style for a hint.  Also, not sure that's actually
appropriate for a hint.

+         case EXPR_KIND_ROW_SECURITY:
+             return "ROW SECURITY";

This is quite simply bizarre.  That's not the SQL syntax of anything.

+             | ROW SECURITY row_security_option
+                 {
+                     VariableSetStmt *n = makeNode(VariableSetStmt);
+                     n->kind = VAR_SET_VALUE;
+                     n->name = "row_security";
+                     n->args = list_make1(makeStringConst($3, @3));
+                     $$ = n;
+                 }

I object to this.  There's no reason that we should bloat the parser
to allow SET ROW SECURITY in lieu of SET row_security unless this is a
standard-mandated syntax with standard-mandated semantics, which I bet
it isn't.

  /*
+  * Although only "on" and"off" are documented, we accept all likely
variants of
+  * "on" and "off".
+  */
+ static const struct config_enum_entry row_security_options[] = {
+     {"off", ROW_SECURITY_OFF, false},
+     {"on", ROW_SECURITY_ON, false},
+     {"true", ROW_SECURITY_ON, true},
+     {"false", ROW_SECURITY_OFF, true},
+     {"yes", ROW_SECURITY_ON, true},
+     {"no", ROW_SECURITY_OFF, true},
+     {"1", ROW_SECURITY_ON, true},
+     {"0", ROW_SECURITY_OFF, true},
+     {NULL, 0, false}
+ };

Just make it a bool and you get all this for free.

+ /*
+  * is_rls_enabled -
+  *   determines if row-security is enabled by checking the value of the system
+  *   configuration "row_security".
+  */
+ bool
+ is_rls_enabled()
+ {
+     char const *rls_option;
+
+     rls_option = GetConfigOption("row_security", true, false);
+
+     return (strcmp(rls_option, "on") == 0);
+ }

Words fail me.

+     if (AuthenticatedUserIsSuperuser)
+         SetConfigOption("row_security", "off", PGC_INTERNAL, PGC_S_OVERRIDE);

Injecting this kind of magic into InitializeSessionUserId(),
SetSessionAuthorization(), and SetCurrentRoleId() seems 100%
unacceptable to me.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



--

Re: RLS Design

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
All,

Attached is a patch for RLS that was create against master at 01363beae52700c7425cb2d2452177133dad3e93 and is ready for review.

Overview:

This patch provides the capability to create multiple named row level security policies for a table on a per command basis and assign them to be applied to specific roles/users.

It contains the following changes:

* Syntax:

CREATE POLICY <name> ON <table>
    [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
    [ TO { PUBLIC | <role> [, <role> ] } ]
    USING (<condition>)

Creates a RLS policy named <name> on <table>.  Specifying a command is optional, but the default is ALL.  Specifying a role is options, but the default is PUBLIC.  If PUBLIC and other roles are specified, ONLY PUBLIC is applied and a warning is raised.

ALTER POLICY <name> ON <table>
    [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
    [ TO { PUBLIC | <role> [, <role> ] } ]
    USING (<condition>)

Alter a RLS policy named <name> on <table>.  Specifying a command is optional, if provided then the policy's command is changed otherwise it is left as-is.  Specifying a role is optional, if provided then the policy's role is changed otherwise it is left as-is.  The <condition> must always be provided and is therefore always replaced.

DROP POLICY <name> ON <table>

Drop a RLS policy named <name> on <table>.

* Plancache Invalidation:  If a relation has a row-security policy and row-security is enabled then the invalidation will occur when either the row_security GUC is changed OR when a the current user changes.  This invalidation ONLY takes place for cached plans where the target relation has a row security policy.

* Security Qual Expression:  All row-security policies are OR'ed together.  In the case where another security qual is added, such as in the case of a Security Barrier Views, the the row-security policies are AND'ed with those quals.

Example:

If a table has policies p1 and p2 and a security barrier view is created for that table called rls_sbv, then SELECT * FROM rls_sbv WHERE <some_condition> would result in the following expression: <some_condition> AND (p1 OR p2)

* row_security GUC - enable/disable row level security.

* BYPASSRLS and NOBYPASSRLS role attribute - allows user to bypass RLS if row_security GUC is set to OFF.  If a user sets row_security to OFF and does not have this attribute, then an error is raised when attempting to query a relation with a RLS policy.

* psql \d <table> support: psql describe support for listing policy information per table.

* pg_policies system view: lists all row-security policy information.

Any feedback, comments or suggestions would be greatly appreciated.

Thanks,
Adam


On Mon, Aug 18, 2014 at 10:19 PM, Brightwell, Adam <adam.brightwell@crunchydatasolutions.com> wrote:
All,

Attached is a patch for RLS that incorporates the following changes:

* Syntax:
  - CREATE POLICY <policy_name> ON <table_name> FOR <command> USING ( <qual> )
  - ALTER POLICY <policy_name> ON <table_name> FOR <command> USING ( <qual> )
  - DROP POLICY <policy_name> ON <table_name> FOR <command>

* "row_security" GUC Setting - enable/disable row level security.

* BYPASSRLS and NOBYPASSRLS role attribute - allows user to bypass RLS if row_security GUC is set to OFF.

There are still some remaining issues but we hope to have those resolved soon.

Any comments or suggestions would be greatly appreciated.

Thanks,
Adam


On Mon, Jul 21, 2014 at 11:38 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jul 18, 2014 at 7:01 PM, Brightwell, Adam
<adam.brightwell@crunchydatasolutions.com> wrote:
>> I think we do want a way to modify policies.  However, we tend to
>> avoid syntax that involves unnatural word order, as this certainly
>> does.  Maybe it's better to follow the example of CREATE RULE and
>> CREATE TRIGGER and do something this instead:
>>
>> CREATE POLICY policy_name ON table_name USING quals;
>> ALTER POLICY policy_name ON table_name USING quals;
>> DROP POLICY policy_name ON table_name;
>>
>> The advantage of this is that you can regard "policy_name ON
>> table_name" as the identifier for the policy throughout the system.
>> You need some kind of identifier of that sort anyway to support
>> COMMENT ON, SECURITY LABEL, and ALTER EXTENSION ADD/DROP for policies.
>
> Sounds good.  I certainly think it makes a lot of sense to include the ALTER
> functionality, if for no other reason than ease of use.
>
> Another item to consider, though I believe it can come later, is per-action
> policies.  Following the above suggested syntax, perhaps that might look
> like the following?
>
> CREATE POLICY policy_name ON table_name FOR action USING quals;
> ALTER POLICY policy_name ON table_name FOR action USING quals;
> DROP POLICY policy_name ON table_name FOR action;

That seems reasonable.  You need to give some thought to what happens
if the user types:

CREATE POLICY pol1 ON tab1 FOR SELECT USING q1;
ALTER POLICY pol1 ON tab1 FOR INSERT USING q2;

I guess you end up with q1 as the SELECT policy and q2 as the INSERT
policy.  Similarly, had you typed:

CREATE POLICY pol1 ON tab1 USING q1;
ALTER POLICY pol1 ON tab1 FOR INSERT USING q2;

...then I guess you end up with q2 for INSERTs and q1 for everything
else.  I'm wondering if it might be better, though, not to allow the
quals to be specified in CREATE POLICY, or else to allow multiple
actions.  Otherwise, getting pg_dump to DTRT might be complicated.

Perhaps:

CREATE POLICY pol1 ON tab1 ( [ [ FOR operation [ OR operation ] ... ]
USING quals ] ... );
where operation = SELECT | INSERT | UPDATE | DELETE

So that you can write things like:

CREATE POLICY pol1 ON tab1 (USING a = 1);
CREATE POLICY pol2 ON tab2 (FOR INSERT USING a = 1, FOR UPDATE USING b
= 1, FOR DELETE USING c = 1);

And then, for ALTER, just allow one change at a time, syntax as you
proposed.  That way each policy can be dumped as a single CREATE
statement.

> I was also giving some thought to the use of "POLICY", perhaps I am wrong,
> but it does seem it could be at risk of becoming ambiguous down the road.  I
> can't think of any specific examples at the moment, but my concern is what
> happens if we wanted to add another "type" of policy, whatever that might
> be, later?  Would it make more sense to go ahead and qualify this a little
> more with "ROW SECURITY POLICY"?

I think that's probably over-engineering.  I'm not aware of anything
else we might add that would be likely to be called a policy, and if
we did add something we could probably call it something else instead.
And long command names are annoying.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--

Re: RLS Design

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
I think we do want a way to modify policies.  However, we tend to
avoid syntax that involves unnatural word order, as this certainly
does.  Maybe it's better to follow the example of CREATE RULE and
CREATE TRIGGER and do something this instead:

CREATE POLICY policy_name ON table_name USING quals;
ALTER POLICY policy_name ON table_name USING quals;
DROP POLICY policy_name ON table_name;

The advantage of this is that you can regard "policy_name ON
table_name" as the identifier for the policy throughout the system.
You need some kind of identifier of that sort anyway to support
COMMENT ON, SECURITY LABEL, and ALTER EXTENSION ADD/DROP for policies.

Sounds good.  I certainly think it makes a lot of sense to include the ALTER functionality, if for no other reason than ease of use.

Another item to consider, though I believe it can come later, is per-action policies.  Following the above suggested syntax, perhaps that might look like the following?

CREATE POLICY policy_name ON table_name FOR action USING quals;
ALTER POLICY policy_name ON table_name FOR action USING quals;
DROP POLICY policy_name ON table_name FOR action; 

I was also giving some thought to the use of "POLICY", perhaps I am wrong, but it does seem it could be at risk of becoming ambiguous down the road.  I can't think of any specific examples at the moment, but my concern is what happens if we wanted to add another "type" of policy, whatever that might be, later?  Would it make more sense to go ahead and qualify this a little more with "ROW SECURITY POLICY"?

Thanks,
Adam

--

Re: RLS Design

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
Stephen,

Yeah, now that we're trying to bake this into ALTER TABLE we need to be
a bit more cautious.  I'd think:

ALTER TABLE tab POLICY ADD ...

Would work though?  (note: haven't looked/tested myself)

Yes, I just tested it and the following would work from a grammar perspective:

ALTER TABLE <table_name> POLICY ADD <policy_name> (policy_quals)
ALTER TABLE <table_name> POLICY DROP <policy_name>

Though, it would obviously require the addition of POLICY to the list of unreserved keywords.  I don't suspect that would be a concern, as it is not "reserved", but thought I would point it out just in case.

Another thought I had was, would we also want the following, so that policies could be modified?

ALTER TABLE <table_name> POLICY ALTER <policy_name> (policy_quals)

Thanks,
Adam

--

Re: RLS Design

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
Tom,

Thanks for the feedback.

20MB messages to the list aren't that friendly.  Please don't do that
again, unless asked to.

Apologies, I didn't realize it was so large until after it was sent.  At any rate, it won't happen again.
 
FWIW, the above syntax is a nonstarter, at least unless we're willing to
make POLICY a reserved word (hint: we're not).  The reason is that the
ADD/DROP COLUMN forms consider COLUMN to be optional, meaning that the
column name could directly follow ADD; and the column type name, which
could also be just a plain identifier, would directly follow that.  So
there's no way to resolve the ambiguity with one token of lookahead.
This actually isn't just bison being stupid: in fact, you simply
cannot tell whether

     ALTER TABLE tab ADD POLICY varchar(42);

is an attempt to add a column named "policy" of type varchar(42), or an
attempt to add a policy named "varchar" with quals "42".

Ok.  Make sense and I was afraid that was the case.
 
Thanks,
Adam

--

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
"Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>
Дата:
Hey Tom,
 
Hm ... I'm not following why we'd need a special case for superusers and
not anyone else?  Seems like any useful RLS scheme is going to require
more privilege levels than just superuser and not-superuser.

As it stands right now, superuser is the only case where RLS policies should not be applied/completely ignored.  I suppose it is possible to create RLS policies that are related to other privilege levels, but those would still need to be applied despite user id, excepting superuser.  I'll defer to Stephen or Craig on the usefulness of this scheme.

Could we put the "if superuser then ok" test into the RLS condition test
and thereby not need more than one plan at all?

As I understand it, the application of RLS policies occurs in the rewriter.  Therefore, when switching back and forth between superuser and not-superuser the query must be rewritten, which would ultimately result in the need for a new plan correct?  If that is the case, then I am not sure how one plan is possible.  However, again, I'll have to defer to Stephen or Craig on this one.

Thanks,
Adam

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:
Robert,

On Tuesday, June 17, 2014, Robert Haas <robertmhaas@gmail.com> wrote:
After sending that one (1) email, I was promptly told that "I'm very
disappointed to hear that the mechanical pieces around making RLS easy
for users to use ... is receiving such push-back."  The push-back, at
that point in time, consisted of one (1) email.  Several more emails
have been sent that time, including the above-quoted text, seeming to
me to imply that the people who are concerned about this feature are
being unreasonable.  I don't believe I am the only such person,
although I may be the main one right at the moment, and you may not be
entirely surprised to hear that I don't think I'm being unreasonable.
 
I'm on my phone at the moment but that looks like a quote from me. My email and concern there was regarding the specific suggestion that we could check off the "RLS" capability which users have been asking us to provide nearly since I started with PG by saying that they could use Updatable SB views. I did not intend it as a comment regarding the specific technical concerns raised and have been responding to and trying to address those independently and openly. 

I've expressed elsewhere on this thread my gratitude that the technical concerns are being brought up now, near the beginning of the cycle, so we can address them. I've been working with others who are interested in RLS on a wiki page to outline and understand the options and identify dependencies and priorities. Hopefully the link will be posted shortly (again, not at a computer right now) and we can get comments back. There are some very specific questions which really need to be addressed and which I've mentioned before (in particular the question of what user the functions in a view definition should run as, both for "normal" views, for SB views, and for when an RLS qual is included and run through that framework, and if doing so would address some of the concerns which have been raised regarding selects running code). 

Thanks,

Stephen

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:
Robert,

Alright, I can't help it so I'll try and reply from my phone for a couple of these. :)

On Wednesday, September 3, 2014, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Aug 29, 2014 at 8:16 PM, Brightwell, Adam
<adam.brightwell@crunchydatasolutions.com> wrote:
> Attached is a patch for RLS that was create against master at
> 01363beae52700c7425cb2d2452177133dad3e93 and is ready for review.
>
> Overview:
>
> This patch provides the capability to create multiple named row level
> security policies for a table on a per command basis and assign them to be
> applied to specific roles/users.
>
> It contains the following changes:
>
> * Syntax:
>
> CREATE POLICY <name> ON <table>
>     [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
>     [ TO { PUBLIC | <role> [, <role> ] } ]
>     USING (<condition>)
>
> Creates a RLS policy named <name> on <table>.  Specifying a command is
> optional, but the default is ALL.  Specifying a role is options, but the
> default is PUBLIC.  If PUBLIC and other roles are specified, ONLY PUBLIC is
> applied and a warning is raised.
>
> ALTER POLICY <name> ON <table>
>     [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
>     [ TO { PUBLIC | <role> [, <role> ] } ]
>     USING (<condition>)
>
> Alter a RLS policy named <name> on <table>.  Specifying a command is
> optional, if provided then the policy's command is changed otherwise it is
> left as-is.  Specifying a role is optional, if provided then the policy's
> role is changed otherwise it is left as-is.  The <condition> must always be
> provided and is therefore always replaced.

This is not a full review of this patch; as we're mid-CommitFest, I
assume this will get added to the next CommitFest.

As per usual, the expectation is that the patch is reviewed and updated during the commitfest.  Given that the commitfest isn't even over according to the calendar it seems a bit premature to talk about the next one, but certainly if it's not up to a commitable level before the end of this commitfest then it'll be submitted for the next. 
 
In earlier discussions, it was proposed (and I thought the proposal
was viewed favorably) that when enabling row-level security for a
table (i.e. before doing CREATE POLICY), you'd have to first flip the
table to a default-deny mode:

I do recall that (now that you remind me- clearly it had been lost during the subsequent discussion, from my point of view at least) and agree that it'd be useful. I don't believe it'll be difficult to address. 
 
ALTER TABLE <name> ENABLE ROW LEVEL SECURITY;

Sounds reasonable to me. 
 
+         elog(ERROR, "Table \"%s\" already has a policy named \"%s\"."
+             " Use a different name for the policy or to modify this policy"
+             " use ALTER POLICY %s ON %s USING (qual)",
+             RelationGetRelationName(target_table), stmt->policy_name,
+             RelationGetRelationName(target_table), stmt->policy_name);
+
That needs to be an ereport, be capitalized properly, and the hint, if
it's to be included at all, needs to go into errhint().

Already addressed. 
 
+                          errhint("all roles are considered members
of public")));

Wrong message style for a hint.  Also, not sure that's actually
appropriate for a hint.

Fair enough. Will address.
 
+         case EXPR_KIND_ROW_SECURITY:
+             return "ROW SECURITY";

This is quite simply bizarre.  That's not the SQL syntax of anything.

Will address. 
 
+             | ROW SECURITY row_security_option
+                 {
+                     VariableSetStmt *n = makeNode(VariableSetStmt);
+                     n->kind = VAR_SET_VALUE;
+                     n->name = "row_security";
+                     n->args = list_make1(makeStringConst($3, @3));
+                     $$ = n;
+                 }

I object to this.  There's no reason that we should bloat the parser
to allow SET ROW SECURITY in lieu of SET row_security unless this is a
standard-mandated syntax with standard-mandated semantics, which I bet
it isn't.

Agreed. Seemed like a nice idea but it's not necessary. 
 
  /*
+  * Although only "on" and"off" are documented, we accept all likely
variants of
+  * "on" and "off".
+  */
+ static const struct config_enum_entry row_security_options[] = {
+     {"off", ROW_SECURITY_OFF, false},
+     {"on", ROW_SECURITY_ON, false},
+     {"true", ROW_SECURITY_ON, true},
+     {"false", ROW_SECURITY_OFF, true},
+     {"yes", ROW_SECURITY_ON, true},
+     {"no", ROW_SECURITY_OFF, true},
+     {"1", ROW_SECURITY_ON, true},
+     {"0", ROW_SECURITY_OFF, true},
+     {NULL, 0, false}
+ };

Just make it a bool and you get all this for free.

Right- holdover from an earlier attempt to make it more complicated but now we've simplified it and so it should just be a bool. 

 
+     if (AuthenticatedUserIsSuperuser)
+         SetConfigOption("row_security", "off", PGC_INTERNAL, PGC_S_OVERRIDE);

Injecting this kind of magic into InitializeSessionUserId(),
SetSessionAuthorization(), and SetCurrentRoleId() seems 100%
unacceptable to me.

I was struggling with the right way to address this and welcome suggestions. The primary issue is that I really want to support a superuser turning it on, so we can't simply have it disabled for all superusers all the time. The requirement that it not be enabled by default for superusers makes sense, but how far does that extend and how do we address upgrades?  In particular, can we simply set row_security=off as a custom GUC setting when superusers are created or roles altered to be made superusers?  Would we do that in pg_upgrade?

Thanks!

Stephen

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:


On Thursday, July 10, 2014, Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Jul 9, 2014 at 2:13 AM, Stephen Frost <sfrost@snowman.net> wrote:
> Yes, this would be possible (and is nearly identical to the original
> patch, except that this includes per-role considerations), however, my
> thinking is that it'd be simpler to work with policy names rather than
> sets of quals, to use when mapping to roles, and they would potentially
> be useful later for other things (eg: for setting up which policies
> should be applied when, or which should be OR' or AND"d with other
> policies, or having groups of policies, etc).

Hmm.  I guess that's reasonable.  Should the policy be a per-table
object (like rules, constraints, etc.) instead of a global object?

You could do:

ALTER TABLE table_name ADD POLICY policy_name (quals);
ALTER TABLE table_name POLICY FOR role_name IS policy_name;
ALTER TABLE table_name DROP POLICY policy_name;

Right, I was thinking they would be per table as they would specifically provide a name for a set of quals, and quals are naturally table-specific. I don't see a need to have them be global- that had been brought up before with the notion of applications picking their policy, but we could also add that later through another term (eg: contexts) which would then map to policies or similar. We could even extend policies to be global by mapping existing per-table ones to be global if we really needed to...

My feeling at the moment is that having them be per-table makes sense and we'd still have flexibility to change later if we had some compelling reason to do so. 

Thanks!

Stephen 

Re: API change advice: Passing plan invalidation info from the rewriter into the planner?

От:
Stephen Frost <sfrost@snowman.net>
Дата:
Greg, all,

I will reply to the emails in detail when I get a chance but am out of town at a funeral, so it'll likely be delayed. I did want to echo my agreement for the most part with Greg and in particular...

On Thursday, June 12, 2014, Gregory Smith <gregsmithpgsql@gmail.com> wrote:
On 6/11/14, 10:26 AM, Robert Haas wrote:
Now, as soon as we introduce the concept that selecting from a table might not really mean "read from the table" but "read from the table after applying this owner-specified qual", we're opening up a whole new set of attack surfaces. Every pg_dump is an opportunity to hack somebody else's account, or at least audit their activity.

I'm in full agreement we should clearly communicate the issues around pg_dump in particular, because they can't necessarily be eliminated altogether without some major work that's going to take a while to finish.  And if the work-around is some sort of GUC for killing RLS altogether, that's ugly but not unacceptable to me as a short-term fix.

A GUC which is enable / disable / error-instead may work quiet well, with error-instead for pg_dump default if people really want it (there would have to be a way to disable that though, imv).

Note that enable is default in general, disable would be for superuser only (or on start-up) to disable everything, and error-instead anyone could use but it would error instead of implementing RLS when querying an RLS-enabled table. 

This approach was suggested by an existing user testing out this RLS approach, to be fair, but it looks pretty sane to me as a way to address some of these concerns. Certainly open to other ideas and thoughts though. 

Thanks,

Stephen

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:
Kaigai,

On Thursday, July 3, 2014, Kouhei Kaigai <kaigai@ak.jp.nec.com> wrote:
Sorry for my late responding, now I'm catching up the discussion.

> * Robert Haas (robertmhaas@gmail.com) wrote:
> > On Tue, Jul 1, 2014 at 3:20 PM, Dean Rasheed <dean.a.rasheed@gmail.com>
> wrote:
> > > If RLS quals are instead regarded as constraints on access, and
> > > multiple policies apply, then it seems that the quals should now be
> > > combined with AND rather than OR, right?
>
> I do feel that RLS quals are constraints on access, but I don't see how
> it follows that multiple quals should be AND'd together because of that.
> I view the RLS policies on each table as being independent and "standing
> alone" regarding what can be seen.  If you have access to a table today
> through policy A, and then later policy B is added, using AND would mean
> that the set of rows returned is less than if only policy A existed.
> That doesn't seem correct to me.
>
It seems to me direction of the constraints (RLS-policy) works to is reverse.

In case when we have no RLS-policy, 100% of rows are visible isn't it?

No, as outlined later, the table would appear empty if no policies exist and RLS is enabled for the table. 
 
Addition of a constraint usually reduces the number of rows being visible,
or same number of rows at least. Constraint shall never work to the direction
to increase the number of rows being visible.

Can you clarify where this is coming from..?  It sounds like you're referring to an existing implementation and, if so, it'd be good to get more information on how that works exactly.
 
If multiple RLS-policies are connected with OR-operator, the first policy
works to the direction to reduce number of visible rows, but the second
policy works to the reverse direction.

This isn't accurate, as mentioned. Each policy stands alone to define what is visible through it and if no policy exists then no rows are visible. 
 
If we would have OR'd RLS-policy, how does it merged with user given
qualifiers with?

The RLS quals are all applied together with OR's and the result is AND'd with any user quals provided. This is only when multiple policies are being applied for a given query and seems pretty straight forward to me. 
 
For example, if RLS-policy of t1 is (t1.credential < get_user_credential)
and user's query is:
  SELECT * FROM t1 WHERE t1.x = t1.x;
Do you think RLS-policy shall be merged with OR'd form?

Only the RLS policies are OR'd together, not user provided quals. The above would result in:

Where t1.x = t1.x and (t1.credential < get_user_credential)

If another policy also applies for this query, such as t1.cred2 < get_user_credential then we would have:

Where t1.x = t1.x and (t1.credential < get_user_credential OR t1.cred2 < get_user_credential)
 
This is similar to how roles work- your overall access includes all access granted to any roles you are a member of. You don't need SELECT rights granted to every role you are a member of to select from the table. Additionally, if an admin wants to AND the quals together then they can simply create a policy which does that rather than have 2 policies. 

> > Yeah, maybe.  I intuitively feel that OR would be more useful, so it
> > would be nice to find a design where that makes sense.  But it depends
> > a lot, in my view, on what syntax we end up with.  For example,
> > suppose we add just one command:
> >
> > ALTER TABLE table_name FILTER [ role_name | PUBLIC ] USING qual;
> >
> > If the given role inherits from multiple roles that have different
> > filters, I think the user will naturally expect all of the filters to
> > be applied.
>
> Agreed.
>
> > But you could do it other ways.  For example:
> >
> > ALTER TABLE table_name [ NO ] ROW LEVEL SECURITY; ALTER TABLE
> > table_name GRANT ROW ACCESS TO role_name USING qual;
> >
> > If a table is set to NO ROW LEVEL SECURITY then it behaves just like
> > it does now: anyone who accesses it sees all the rows, restricted to
> > those columns for which they have permission.  If the table is set to
> > ROW LEVEL SECURITY then the default is to show no rows.  The second
> > command then allows access to a subset of the rows for a give role
> > name.  In this case, it is probably logical for access to be combined
> > via OR.
>
> I can see value is having a table-level option to indicate if RLS is applied
> for that table or not, but I had been thinking we'd just automatically manage
> that.  That is to say that once you define an RLS policy for a table, we
> go look and see what policy should be applied in each case.  With the user
> able to control that, what happens if they say "row security" on the table
> and there are no policies?  All access would show the table as empty?  What
> if policies exist and they decide to 'turn off' RLS for the table- suddenly
> everyone can see all the rows?
>
> My answers to the above (which are making me like the idea more,
> actually...) would be:
>
> Yes, if they turn on RLS for the table and there aren't any policies, then
> the table appears empty for anyone with normal SELECT rights (table owner
> and superusers would still see everything).
>
> If policies exist and the user asks to turn off RLS, I'd throw an ERROR
> as there is a security risk there.  We could support a CASCADE option which
> would go and drop the policies from the table first.
>
Hmm... This approach starts from the empty permission then adds permission
to reference a particular range of the configured table. It's one attitude.


Right- just like how our grant system works. 
 
However, I think it has a dark side we cannot ignore. Usually, the purpose
of security mechanism is to ensure which is readable/writable according to
the rules. Once multiple RLS-policies are merged with OR'd form, its results
are unpredicatable.

I don't see how it's unpredictable at all. 
 
Please assume here are two individual applications that use RLS on table-X.
Even if application-1 want only rows being "public" become visible, it may
expose "credential" or "secret" rows by interaction of orthogonal policy
configured by application-2 (that may configure the policy according to the
source ip-address). It seems to me application-2 partially invalidated the
RLS-policy configured by application-1.

 You are suggesting instead that if application 2 sets up policies on the table and then application 1 adds another policy that it should reduce what application 2's users can see?  That doesn't make any sense to me.  I'd actually expect these applications to at least use different roles anyway, which means they could each have a single role specific policy which only returns what that application is allowed to see. 
 
I think, an important characteristic is things to be invisible is invisible
even though multiple rules are configured.

This is addressed through the ability to associate roles to policies. 

Thanks,

Stephen

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:
Hey Robert,

On my phone at the moment but wanted to reply.

I'm working through a few of these issues already actually (noticed as I've been going over it with Adam), but certainly appreciate the additional review. We've not posted another update quite yet but plan to shortly.

Thanks!

Stephen

Re: RLS Design

От:
Stephen Frost <sfrost@snowman.net>
Дата:
Robert,

On Friday, July 11, 2014, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jul 11, 2014 at 4:55 AM, Stephen Frost <sfrost@snowman.net> wrote:
> My feeling at the moment is that having them be per-table makes sense and
> we'd still have flexibility to change later if we had some compelling reason
> to do so.

I don't think you can really change it later.  If policies are
per-table, then you could have a policy p1 on table t1 and also on
table t2; if they become global objects, then you can't have p1 in two
places.  I hope I'm not beating a dead horse here, but changing syntax
after it's been released is very, very hard.

Fair enough. My thinking was we'd come up with a way to map them (eg: table_policy), but I do agree that changing it later would really suck and having them be per-table makes a lot of sense. 
 
But that's not an argument against doing it this way; I think
per-table policies are probably simpler and better here.  It means,
for example, that policies need not have their own permissions and
ownership structure - they're part of the table, just like a
constraint, trigger, or rule, and the table owner's permissions
control.  I like that, and I think our users will, too.

Agreed and I believe this is more-or-less what I had proposed up-thread (not at a computer at the moment). I hope to have a chance to review and update the design and flush out the catalog definition this weekend.

Thanks!

Stephen
FAQ