Re: pgaudit - an auditing extension for PostgreSQL

Simon,

* Simon Riggs (simon@2ndQuadrant.com) wrote:
> On 30 June 2014 16:20, Stephen Frost <sfrost@snowman.net> wrote:
> > eh?  The focus of this patch is to add auditing to PG and having very
> > clearly drawn auditing requirements of a very large customer isn't
> > relevant?  I don't follow that logic at all.  In fact, I thought the
> > point of pgaudit was specifically to help address the issues which are
> > being raised from this section of our user base (perhaps not those who
> > follow NIST, but at least those organizations with similar interests..).
>
> It's not clear to me how NIST 800-53 mandates the use of SQL (or any
> other) language statements for auditing.

800-53 doesn't mandate SQL.  If that was implied by me somewhere then I
apologize for the confusion.  Having SQL support *has* been requested of
multiple users and in some of those cases has been considered a
requirement for their specific case; perhaps that was the confusion.

What 800-53 *does* discuss (under AU-9, control 4, from here:
http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r4.pdf)
is the auditor role as being independent and able to control auditing.
Further, it discusses under AU-2 that organizations must determine the
set of events which are to be audited and then under AU-12 (in general,
but also specifically under AU-12 control 3), that authorized
individuals can change what is being audited.

Another point is a practical matter- it's extremely important to control
where the audit logs go as well, and ideally it'd be possible to send
audit logs for certain RLS policies to one location while other audit
logs are sent to a different (perhaps less secure) location.  That's an
issue which I don't believe has to be addressed immediately, but until
it is addressed the audit log has to be at the highest level of control,
which makes it more difficult to manage and monitor.

> I strongly doubt that it
> does, but I am happy to hear summaries of large documents, or specific
> references to pages and paragraphs, just as we would do for SQL
> Standard. There is no other difference between a function and a
> statement - both may create catalog entries; I agree we need catalog
> entries so that auditing config can itself be audited.

Having functions to control the auditing would work, but it's not
exactly the ideal approach, imv, and the only reason it's being
discussed here is because it might be a way to allow an extension to
provide the auditing- not because it's actually a benefit to anyone.
However, if we have such functions in a contrib extension, I worry what
the pg_upgrade path is from that extension to an in-core solution.

Robert feels that's managable and perhaps he's right but we don't have
either the function-based design with custom reloptions nor a concrete
idea of what the in-core solution would be and so I'm skeptical that
we'd be able to easily provide a path from one to the other with
pg_upgrade.

> The rest of this argument seems to revolve around the idea that
> "in-core" and "extension"/contrib are different things. That is much
> more of a general point and one that is basically about development
> style.

The downside of extensions for this case is the inability to create and
integrate new catalog tables into the pg_catalog environment, and the
inability for extensions to extend the SQL grammar, as you mention.
I'd love to see both of those issues solved in a way which would allow
us to build the auditing system that I've been dreaming about as an
extension or "feature".  I don't feel we should be holding off on
projects like auditing while we wait for that to happen though.

> But if you missed it before, I will say it again: we have a patch now
> and 2ndQuadrant has a limit on further funding. If you want to reject
> pgaudit and move on to something else, lets start discussing that
> design and consider who has the time (and implicitly the money) to
> make that happen. We can pick the best one for inclusion in 9.5 later
> in the cycle.

What I'd love to see is progress on both fronts, really.  pgaudit is
great and it'd be good to have it for the older releases and for those
releases up until we get real auditing in PG- I just don't want to make
adding that in-PG-auditing later more difficult than it already is by
having to address upgrades from pgaudit.  That's not an issue if pgaudit
lives outside of PG, but it could be an issue if it's in -contrib.

If we're willing to say that we won't worry about upgrades from pgaudit
to an in-core solution (or at least that the lack of such an upgrade
path won't prevent adding an in-core auditing system) then my concerns
about that would be addressed, but we don't get to change our minds
about that after pgaudit has been released as part of 9.5..
Thanks,
    Stephen