Обсуждение: PostgreSQL Auditing
So Auditing, it seems that some people want auditing (myself, David Steele, 2nd quadrant, and probably others). I personally love postgresql, but until it can meet my annoying compliance requirements, I can't leverage it fully as my organization spends more time on meeting compliance, than actually doing development and engineering.
Sadly, due to the incumbent solutions in the database arena, we are also wasting idiotic amounts of time, money, and increasing system complexity because we are having to use alternative solutions that provide things like auditing.
If David's auditing patch isn't sufficient, what is? Are we waiting on the holy grail of auditing, which implements an entirely new logging subsystem, and hooks so deeply into the innards of PostgreSQL its perfect? Does this mailing list just not care about the potential customers (and potential financial benefits) of providing a complete database solution? Or does the postgresql community just want to stay a hobbyist database that never broaches the enterprise or compliance arenas?
I've worked with many database vendors, and honestly auditing is fairly bland, its boring, and no one really likes it except for the lawyers, and then only when someone was actually caught doing something wrong, which lets face it is quite infrequent given the number of databases that exist out there.
Just because auditing isn't sexy sharding, parallel partitioning, creative indexing (BRIN), or hundreds of thousands of transactions a second, doesn't make it any less of a requirement to countless organizations that would like to use postgresql, but find the audit requirement a must have.
So, in summary, what would it take to get the core PostgreSQL team to actually let auditing patches into the next version?
P.S., do you know what sucks, having a highly performant PostGIS database that works great, and being told to move to Oracle or SQL Server (because they have auditing). Even though they charge extra for Geospatial support (seriously?) or when they don't even have geospatial support (10 years ago). My customer would prefer to re-engineer software designed around PostgreSQL and pay the overpriced licenses, than not have auditing. I agree that their cost analysis is probably way off, even 10 years later, my only solution would be to move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for their 2 year old version that doesn't have all the cool/modern jsonb support.
On Tue, Feb 02, 2016 at 01:05:46AM +0000, Curtis Ruck wrote: > So Auditing, it seems that some people want auditing (myself, David Steele, > 2nd quadrant, and probably others). I personally love postgresql, but > until it can meet my annoying compliance requirements, I can't leverage it > fully as my organization spends more time on meeting compliance, than > actually doing development and engineering. > So, in summary, what would it take to get the core PostgreSQL team to > actually let auditing patches into the next version? [PostgreSQL has a "core team" (pgsql-core@) that rarely rules on specific features. I'm interpreting "core PostgreSQL team" more broadly as "people who determine whether to accept patches".] Based on the last discussion, I expect it would take resolving these concerns: 1. Does PostgreSQL gain enough by having the extension in core instead of leaving the same extension in Github? (Tom) 2. Will the feature set stand the test of time as the one core auditing feature set, pleasing to most of the PostgreSQLusers who seek auditing? It's fine if the initial patch has a subset of the eventual feature set, but it's badif we later wish to undo UI decisions. (Tom, Heikki, Robert) Your own input would be especially helpful on this point. Would you describe the compliance requirements you have facedlately? References to external standards are fine, and lists of private requirements are also fine. If you can annotatea requirements list with the pgaudit feature used to meet each requirement, that would be even more helpful. 3. Does the implementation do what it claims to do, correctly? Any defects are likely to be security vulnerabilities, whichamplifies the cost of fixing them after release. (Noah, Robert) Points (1) and (2) relax considerably for narrow core changes to support an external auditing system. For example, David's errhidefromclient() proposal faces a simpler journey, because it raises far fewer design questions. > P.S., do you know what sucks, having a highly performant PostGIS database > that works great, and being told to move to Oracle or SQL Server (because > they have auditing). Even though they charge extra for Geospatial support In the same sense that PostgreSQL has geospatial support, PostgreSQL 9.3 and later do have auditing. Get it from https://github.com/2ndQuadrant/pgaudit, just like you get PostGIS separately. I realize some sites forbid extensions like pgaudit and PostGIS, but this particular case you describe is fortunate not to have that problem. Thanks, nm
On Tue, Feb 2, 2016 at 1:05 AM, Curtis Ruck <curtis.ruck+pgsql.hackers@gmail.com> wrote: > or pay > EnterpriseDB for their 2 year old version that doesn't have all the > cool/modern jsonb support. Just for the record, anyone who pays for our "2 year old version that doesn't have all the cool/modern jsonb support" is also entitled to use either our 9.4 or 9.5 versions which do have JSONB support. Our new versions are typically released within a couple of months of PostgreSQL, and in the case of 9.5, the gap was more like 3 weeks. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 02/02/2016 02:05 AM, Curtis Ruck wrote:
> [snip]
>
> P.S., do you know what sucks, having a highly performant PostGIS
> database that works great, and being told to move to Oracle or SQL
> Server (because they have auditing). Even though they charge extra
> for Geospatial support (seriously?) or when they don't even have
> geospatial support (10 years ago). My customer would prefer to
> re-engineer software designed around PostgreSQL and pay the overpriced
> licenses, than not have auditing. I agree that their cost analysis is
> probably way off, even 10 years later, my only solution would be to
> move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for
> their 2 year old version that doesn't have all the cool/modern jsonb
> support.
Huh? PPAS 9.5.0.5 is already out there since at least last week; Before
that PPAS 9.4.5.y or so was there ...
(Not affiliated with EDB, but precision is important)
I agree that auditing is a big selling point and frequently used... But
it's got to be done "the Postgres way", and that takes time (and usually
provides superior results).
Just my .02€
/ J.L.
On 2 February 2016 at 02:05, Curtis Ruck <curtis.ruck+pgsql.hackers@gmail.com> wrote:
--
Just because auditing isn't sexy sharding, parallel partitioning, creative indexing (BRIN), or hundreds of thousands of transactions a second, doesn't make it any less of a requirement to countless organizations that would like to use postgresql, but find the audit requirement a must have.So, in summary, what would it take to get the core PostgreSQL team to actually let auditing patches into the next version?
I appreciate your frustration, though I'd say you're making a few conceptual leaps in what you've said. I can help with a few answers.
For example, 2ndQuadrant developed the original pgAudit extension and currently provide commercial support for users. So whether this gets included into core PostgreSQL or not, is not the gating factor on whether commercial support is available for open source software.
Security is an important thing round here, which also means that we follow a default-deny approach to new features. So it can take some time to include new features in core. The process is the same whether its sexy or not. I agree it can be frustrating at times though overall we maintain a high throughput of new features into PostgreSQL.
The original version of PgAudit sat in the queue unreviewed for about 7 months, which was a huge factor in it not being accepted into 9.5. We are very short of reviewers and detailed reviews are accepted from any source. So yourself or a colleague could make a difference here and I encourage people with specialist knowledge and passion to take part.
P.S., do you know what sucks, having a highly performant PostGIS database that works great, and being told to move to Oracle or SQL Server (because they have auditing). Even though they charge extra for Geospatial support (seriously?) or when they don't even have geospatial support (10 years ago). My customer would prefer to re-engineer software designed around PostgreSQL and pay the overpriced licenses, than not have auditing. I agree that their cost analysis is probably way off, even 10 years later, my only solution would be to move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for their 2 year old version that doesn't have all the cool/modern jsonb support.
I agree it sucks when other people make money and you don't. That limits funds available to allocate people on tasks, even when we see them as important. But there are many companies who would be willing to implement solutions or extend open source code for you, allowing that problem to be solved. We don't usually discuss that option here, since this is an engineering list.
Since you've written the email here, I'd ask that you join our community and use your knowledge and passion to make things happen.
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 02/02/2016 02:47 AM, José Luis Tallón wrote: > On 02/02/2016 02:05 AM, Curtis Ruck wrote: >> [snip] >> >> P.S., do you know what sucks, having a highly performant PostGIS >> database that works great, and being told to move to Oracle or SQL >> Server (because they have auditing). Even though they charge extra >> for Geospatial support (seriously?) or when they don't even have >> geospatial support (10 years ago). My customer would prefer to >> re-engineer software designed around PostgreSQL and pay the overpriced >> licenses, than not have auditing. I agree that their cost analysis is >> probably way off, even 10 years later, my only solution would be to >> move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for >> their 2 year old version that doesn't have all the cool/modern jsonb >> support. PostgreSQL has auditing. It is available now, just not in core. Postgis isn't available in core either and it seems to do just fine. JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On Tue, Feb 02, 2016 at 07:24:23AM -0800, Joshua D. Drake wrote: > PostgreSQL has auditing. It is available now, just not in core. Postgis > isn't available in core either and it seems to do just fine. I don't really buy that argument. For one, PostGIS has a pretty narrow functional use-case (spatial), while auditing is a horizontal use-case that could be required for any kind of database usage. Second, PostGIS had 10+ (?) years to build a reputation so that people say "if I have to choose between PostGIS and buying Oracle Spatial, of course I choose PostGIS", the pgaudit extension does not have that. Auditing is a pretty security/enterprisey-related thing that could do with the "officially considered to of the PostgreSQL project standard and ready for production" rubber-stamp that tends to go along with most end-user/admin-oriented stuff shipped in the tarball. I am aware that 2nd Quadrant, Crunchy Data and EnterpriseDB (different codebase via PPAS) all support their auditing extensions commercially, so that there is certainly some form of credibility, but still. Now, whether or not the currently submitted approach actually meets the above rubber-stamp requirements is a different story, but at least I think it would be quite useful to ship auditing capabilites in the distribution. Michael -- Michael Banck Projektleiter / Berater Tel.: +49 (2161) 4643-171 Fax: +49 (2161) 4643-100 Email: michael.banck@credativ.de credativ GmbH, HRB Mönchengladbach 12080 USt-ID-Nummer: DE204566209 Hohenzollernstr. 133, 41061 Mönchengladbach Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer
On 02/02/2016 07:31 AM, curtis.ruck@gmail.com wrote: > Its not available in rpm or premade binary forms in its current instantiation, not a big deal for me, but it raises thebarrier to entry. > > If it was packaged into an RPM, what would be the process to get it added to postgresql's yum repositories? Assuming it is not placed into core, we would need to work with the deb and rpm projects. Which I am sure we could (and would) do. Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 02/02/2016 08:13 AM, Michael Banck wrote: > On Tue, Feb 02, 2016 at 07:24:23AM -0800, Joshua D. Drake wrote: >> PostgreSQL has auditing. It is available now, just not in core. Postgis >> isn't available in core either and it seems to do just fine. > > I don't really buy that argument. For one, PostGIS has a pretty narrow > functional use-case (spatial), while auditing is a horizontal use-case > that could be required for any kind of database usage. The argument was made specifically for the user because they were using PostGIS. > > Second, PostGIS had 10+ (?) years to build a reputation so that people > say "if I have to choose between PostGIS and buying Oracle Spatial, of > course I choose PostGIS", the pgaudit extension does not have that. True enough but so what? At some point, someone has to use it. Just because it doesn't have 10 years of experience doesn't mean we should shove it into core. Those that need it, will use it. My customers (for example) use what I tell them to use. > Auditing is a pretty security/enterprisey-related thing that could do > with the "officially considered to of the PostgreSQL project standard > and ready for production" rubber-stamp that tends to go along with most > end-user/admin-oriented stuff shipped in the tarball. Which is exactly why I think .Org needs an official "Extensions" project which would completely eliminate these arguments. A project team explicitly for vetting extensions. > I am aware that > 2nd Quadrant, Crunchy Data and EnterpriseDB (different codebase via > PPAS) all support their auditing extensions commercially, so that there > is certainly some form of credibility, but still. Meh, commercial solutions aren't a consideration here. This is PostgreSQL not EDB or Crunchy. Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On Tue, Feb 2, 2016 at 11:13 AM, Michael Banck <michael.banck@credativ.de> wrote: > On Tue, Feb 02, 2016 at 07:24:23AM -0800, Joshua D. Drake wrote: >> PostgreSQL has auditing. It is available now, just not in core. Postgis >> isn't available in core either and it seems to do just fine. > > I don't really buy that argument. For one, PostGIS has a pretty narrow > functional use-case (spatial), while auditing is a horizontal use-case > that could be required for any kind of database usage. If you're saying you think the user base for pgaudit will be larger than the user base for PostGIS, color me doubtful. > Now, whether or not the currently submitted approach actually meets the > above rubber-stamp requirements is a different story, but at least I > think it would be quite useful to ship auditing capabilites in the > distribution. That is a sensible position. :-) -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 2/2/16 10:34 AM, Joshua D. Drake wrote: >> Auditing is a pretty security/enterprisey-related thing that could do >> with the "officially considered to of the PostgreSQL project standard >> and ready for production" rubber-stamp that tends to go along with most >> end-user/admin-oriented stuff shipped in the tarball. > > Which is exactly why I think .Org needs an official "Extensions" project > which would completely eliminate these arguments. A project team > explicitly for vetting extensions. Yeah, it's disappointing that PGXN doesn't seem to have really taken off. I'm sure a big part of that is the need for even SQL extensions to have server access, but I suspect part of it is because it's a separate project. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com
On 2/2/16 5:00 AM, Simon Riggs wrote: > Since you've written the email here, I'd ask that you join our community > and use your knowledge and passion to make things happen. +1. Kudos for speaking up in the first place, but it's clear that right now the biggest thing holding Postgres back is lack of reviewers, followed by lack of developers. If your company put even 10% of what it would pay for Oracle or MSSQL licensing back into the community (either via direct employee involvement or by funding development) then auditing could have moved a lot faster than it did. It would also help if the community better publicized ways that companies could give back. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com
On Tue, Feb 02, 2016 at 08:28:54AM -0800, Joshua D. Drake wrote: > On 02/02/2016 07:31 AM, curtis.ruck@gmail.com wrote: > >Its not available in rpm or premade binary forms in its current > >instantiation, not a big deal for me, but it raises the barrier to > >entry. > > > >If it was packaged into an RPM, what would be the process to get it > >added to postgresql's yum repositories? > > Assuming it is not placed into core, we would need to work with the > deb and rpm projects. Which I am sure we could (and would) do. We are looking into packaging pgaudit for Debian. However, then another question comes up: Should the 2nd Quadrant or the Crunchy Data codebase be added to the distribution? Who gets to decide? Alternatively, both could be added, but that will likely confuse users. Michael -- Michael Banck Projektleiter / Berater Tel.: +49 (2161) 4643-171 Fax: +49 (2161) 4643-100 Email: michael.banck@credativ.de credativ GmbH, HRB Mönchengladbach 12080 USt-ID-Nummer: DE204566209 Hohenzollernstr. 133, 41061 Mönchengladbach Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer
On 2/2/16 4:50 PM, Michael Banck wrote: > > We are looking into packaging pgaudit for Debian. > > However, then another question comes up: Should the 2nd Quadrant or the > Crunchy Data codebase be added to the distribution? Who gets to decide? For my 2 cents I think that the version I submitted recently is better for PostgreSQL >= 9.5: https://github.com/pgaudit/pgaudit For this version I started with 2ndQuadrant's excellent work then spent months refactoring, testing and adding additional configuration options.In addition a number of reviewers found issues whichwere fixed and in general it went through a trial by fire. This version includes more comprehensive documentation and extensive regression tests. However, the original 2ndQuadrant version will happily work with PostgreSQL 9.3 or 9.4: https://github.com/2ndQuadrant/pgaudit > Alternatively, both could be added, but that will likely confuse users. This sort of confusion is one of the main reasons I pursued inclusion in core. -- -David david@pgmasters.net
On Tue, Feb 2, 2016 at 5:16 PM, David Steele <david@pgmasters.net> wrote: > This sort of confusion is one of the main reasons I pursued inclusion in > core. But that's exactly wrong. When there is not agreement on one code base over another, that's the time it is most important not to pick one of them arbitrarily privilege it over the others. The *only* time it's appropriate to move something that could just as well as an extension into core is when (1) we think it's highly likely that users will want that particular thing rather than some other thing and (2) we think it's worth the burden of maintaining that code forever. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Its not available in rpm or premade binary forms in its current instantiation, not a big deal for me, but it raises the barrierto entry. If it was packaged into an RPM, what would be the process to get it added to postgresql's yum repositories? February 2 2016 10:24 AM, "Joshua D. Drake" <jd@commandprompt.com> wrote: > On 02/02/2016 02:47 AM, José Luis Tallón wrote: > >> On 02/02/2016 02:05 AM, Curtis Ruck wrote: >>> [snip] >>> >>> P.S., do you know what sucks, having a highly performant PostGIS >>> database that works great, and being told to move to Oracle or SQL >>> Server (because they have auditing). Even though they charge extra >>> for Geospatial support (seriously?) or when they don't even have >>> geospatial support (10 years ago). My customer would prefer to >>> re-engineer software designed around PostgreSQL and pay the overpriced >>> licenses, than not have auditing. I agree that their cost analysis is >>> probably way off, even 10 years later, my only solution would be to >>> move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for >>> their 2 year old version that doesn't have all the cool/modern jsonb >>> support. > > PostgreSQL has auditing. It is available now, just not in core. Postgis isn't available in core > either and it seems to do just fine. > > JD > > -- Command Prompt, Inc. http://the.postgres.company > +1-503-667-4564 > PostgreSQL Centered full stack support, consulting and development. > Everyone appreciates your honesty, until you are honest with them.
Robert,
This isn't wrong. I don't see anyone else trying to submit anything in reference to auditing. Just because there have been multiple implementations in the past, based on commit histories, there have only been a few that tried getting into core or contrib (that i can find in mailing list history). Currently, in 9.6, there is one version that is trying to make it in. If Crunchy, or 2nd Quadrant, tried to get their versions incorporated we would have a disagreement in implementation, but either they are lying in wait, or they passively concur, by not actively disagreeing.
I think if there was a valid commit path laid out for getting auditing into core, or into contrib at least, most users would probably find that sufficient. But it seems that every time someone tries to incorporate auditing, there are countless and varied reasons to deny its inclusion.
David Steele's version of auditing builds on most of the prior pgaudit code and incorporates a large amount of the feedback from 9.5's attempt.
I'm opening to testing and evaluating to see if it meets our compliance requirements, but I am no where close to being a C developer, or having C developers that could actually provide a meaningful review. One issue along this thread already pops up, concerning the client_min_messages, and how other patches in the pipeline for 9.6 would be required to enable the auditing to meet compliance requirements.
It just seems after reading the mailing list history, that there is a lack of interest by people with commit authority, even though there is a decent interest in it from the community, and honestly, no one really likes auditing, but its one of those damned if you do (in performance) and damned if you don't (in legal) things.
Additionally Robert, given your professional status, you are by no means an unbiased contributor in this discussion. Your stance on this matter shows that you don't necessarily want the open source solution to succeed in the commercial/compliance required space. Instead of arguing blankly against inclusion can you at least provide actionable based feedback that if met would allow patches of this magnitude in?
I'm personally fine with fiscally rewarding organizations that assist my customer in succeeding, but its hard to convince my customer to fund open source, even though they wouldn't be able to do 75% of what they do without it. Based on past experience this is the same most open source organizations face, especially when they don't have the marketing engine that the large commercial players have.
Curtis
…
On Tue, Feb 2, 2016 at 5:23 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Feb 2, 2016 at 5:16 PM, David Steele <david@pgmasters.net> wrote:
> This sort of confusion is one of the main reasons I pursued inclusion in
> core.
But that's exactly wrong. When there is not agreement on one code
base over another, that's the time it is most important not to pick
one of them arbitrarily privilege it over the others. The *only* time
it's appropriate to move something that could just as well as an
extension into core is when (1) we think it's highly likely that users
will want that particular thing rather than some other thing and (2)
we think it's worth the burden of maintaining that code forever.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 2/2/16 7:25 PM, Curtis Ruck wrote: > I'm opening to testing and evaluating to see if it meets our compliance > requirements, but I am no where close to being a C developer, or having > C developers that could actually provide a meaningful review. One issue > along this thread already pops up, concerning the client_min_messages, > and how other patches in the pipeline for 9.6 would be required to > enable the auditing to meet compliance requirements. There's other ways you can help besides reviewing. Providing real-world use cases helps. Even better is maintaining things on the wiki that assist with moving things forward (use cases, discussion/decision highlights, really anything that helps move the discussion). > It just seems after reading the mailing list history, that there is a > lack of interest by people with commit authority, even though there is a > decent interest in it from the community, and honestly, no one really > likes auditing, but its one of those damned if you do (in performance) > and damned if you don't (in legal) things. Yeah, no one that's volunteering time (as opposed to being paid to work on PG) is going to pick up something as unsexy and painful to deal with as auditing. > Additionally Robert, given your professional status, you are by no means > an unbiased contributor in this discussion. Your stance on this matter > shows that you don't necessarily want the open source solution to > succeed in the commercial/compliance required space. Instead of arguing I'm sorry, but that's just ridiculous, and I completely agree with Robert's initial sentiment: there needs to be a damn good reason for the community to pick one specific implementation of something when there are competing solutions. > blankly against inclusion can you at least provide actionable based > feedback that if met would allow patches of this magnitude in? It works just like any other patch does: the community has to come to a *consensus* that not only is the feature desired and well designed, but that the implementation is high quality. I haven't followed the auditing discussions closely, but it seems that there are still questions around how the feature should work. > I'm personally fine with fiscally rewarding organizations that assist my > customer in succeeding, but its hard to convince my customer to fund > open source, even though they wouldn't be able to do 75% of what they do > without it. Based on past experience this is the same most open source > organizations face, especially when they don't have the marketing engine > that the large commercial players have. I really don't understand that, given what most of the alternative solutions cost. If they balk at putting money towards developing Postgres they really need to get a quote for running the same amount of MSSQL (let alone Oracle, which is even more expensive). I do think the community could do a better job of at least encouraging companies to fund development. Unfortunately there's always going to be some amount of friction here though, because of the question of how to allocate funds to the different companies that are involved. Another problem is no commercial company can actually guarantee anything will make it into community Postgres, and it's very difficult to even estimate the amount of effort (read as: what to charge) for getting a feature committed. Commercial development is certainly possible though. 2nd Quadrant was able to raise a good amount of money to fund the development of hot standby. IIRC that was before sites like kickstarter existed too, so it would probably be even easier to do today. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com
So, Noah's excellent response has been ignored (from what my threaded Gmail view tells me) at this point...
On Tue, Feb 2, 2016 at 6:25 PM, Curtis Ruck <curtis.ruck+pgsql.hackers@gmail.com> wrote:
Robert,This isn't wrong. I don't see anyone else trying to submit anything in reference to auditing. Just because there have been multiple implementations in the past, based on commit histories, there have only been a few that tried getting into core or contrib (that i can find in mailing list history).
And so if pgaudit is such an improvement then by this logic a poor implementation would have been committed in the past just because someone wrote something and proposed it for commit. Noah summarized his quite weighty point-of-view on the outcome of the patch that has been put forth. One dynamic of which is whether it would be easier - and overall more productive - to make installing auditing as an extension easier compared to getting it into core.
I think if there was a valid commit path laid out for getting auditing into core, or into contrib at least, most users would probably find that sufficient. But it seems that every time someone tries to incorporate auditing, there are countless and varied reasons to deny its inclusion.
You pre-suppose that people are willing and able to spend their time trying to predict the future when in reality they would rather be convinced that what has been put in front of them is worth committing. For what I presume are myriad of both technical and political reasons the people who have the final say have not been so convinced.
It just seems after reading the mailing list history, that there is a lack of interest by people with commit authority, even though there is a decent interest in it from the community, and honestly, no one really likes auditing, but its one of those damned if you do (in performance) and damned if you don't (in legal) things.
So either through persuasion or compensation you need to increase interest...
Additionally Robert, given your professional status, you are by no means an unbiased contributor in this discussion. Your stance on this matter shows that you don't necessarily want the open source solution to succeed in the commercial/compliance required space. Instead of arguing blankly against inclusion can you at least provide actionable based feedback that if met would allow patches of this magnitude in?
Even if that were to be true that would not prevent others involved from moving things forward - an objection like "I don't want this in -core because it will eat into my employer's profits" won't persuade many people. Nit-picking patches might get a bit further but if there is sufficient buy-in from the community as a whole it won't last long. So maybe Robert doesn't actively help as much as he could but a lack of helping is not the same as hindering.
I'm personally fine with fiscally rewarding organizations that assist my customer in succeeding, but its hard to convince my customer to fund open source, even though they wouldn't be able to do 75% of what they do without it. Based on past experience this is the same most open source organizations face, especially when they don't have the marketing engine that the large commercial players have.
Which is why the model seems to be for the service provides who leverage PostgreSQL to charge their customers enough so that part of the profit can flow back into the community that they have based their livelihood upon. Otherwise, maybe features such as this should end up in commercial offerings and those customers who won't charitably contribute will instead have an easier time seeing that at least they are spending less for PostgreSQL licensing than they would for similar features in competitors offerings.
In my estimation this whole thread started poorly and thus likely will not garner much if any forward progress on the issue. Noah's response was spot-on despite that fact and a proper response to it would likely move things forward in a more positive manner. A separate thread could be started to discuss what can be done to improve the scenario where excellent patches and/or extensions are available but are not in core. These two dynamics are separate and trying to hit both in one thread is just going to dilute things so that likely neither topic gets progressed.
David J.
On Tue, Feb 2, 2016 at 8:25 PM, Curtis Ruck <curtis.ruck+pgsql.hackers@gmail.com> wrote: > Additionally Robert, given your professional status, you are by no means an > unbiased contributor in this discussion. Your stance on this matter shows > that you don't necessarily want the open source solution to succeed in the > commercial/compliance required space. Instead of arguing blankly against > inclusion can you at least provide actionable based feedback that if met > would allow patches of this magnitude in? I feel the need to respond to this. I don't appreciate being called a liar. I do not block patches because having them included in PostgreSQL would be to the detriment of my employer. Ever. That would be dishonest and a betrayal of the community's trust. Full stop. I have a track record of committing multiple large patches that were developed by people working for EnterpriseDB's competitors (like logical decoding) or that competed with proprietary features EnterpriseDB already had (like foreign tables). I spend countless hours working on countless patches written by people who do not work for, and have no commercial relationship with, my employer: and who are sometimes working for a competitor. I have worked hard to ensure that EnterpriseDB makes major contributions to the PostgreSQL community, such as parallel query. Even if all of the above were not true, EnterpriseDB does not currently have a feature that competes with pgaudit, and has no current plans to try to develop one. EDBAS does have an auditing feature, but that feature is radically different from what pgaudit does; arguing that I am trying to block pgaudit from going into core because that feature exists is like arguing that I don't want PostgreSQL to get a new frying pan because EnterpriseDB has a toy boat. Furthermore, to the extent that EnterpriseDB does have an interest in having a feature like pgaudit, it would be to my advantage for that feature to go *into* core. After all, everything in PostgreSQL is in EDBAS, but things on PGXN generally aren't. In short, your accusations are both false and illogical. I'm going to go ahead and make a suggestion: instead of showing up on this mailing list and accusing the people who spend their time and energy here trying to make PostgreSQL a better of being pigheaded liars, I think that you should try to really understand how this community works, how it makes decisions, what it does well, and what it does poorly. Then, I think you should argue for your positions in a respectful way, carefully avoiding accusations of bad faith even (and perhaps especially) in cases where you believe it may be present. You will find that almost everyone here behaves in that way, and that is what enables us to get along as well as we do and create a great piece of software together. Every single person who has responded to your emails - and there have been a bunch - has done so with courtesy and integrity, and yet you seem convinced (without a shred of evidence, at least that you've presented) that anyone who doesn't think pgaudit should go into core is either an idiot or part of some sort of cabal. Yet, if that were really true, there would be little point in arguing, because the cabalists won't listen to you anyway, and the idiots will make stupid decisions no matter what. Perhaps you should try starting from a different premise. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 02/02/2016 07:26 PM, Robert Haas wrote: > On Tue, Feb 2, 2016 at 8:25 PM, Curtis Ruck > <curtis.ruck+pgsql.hackers@gmail.com> wrote: >> Additionally Robert, given your professional status, you are by no means an >> unbiased contributor in this discussion. Your stance on this matter shows >> that you don't necessarily want the open source solution to succeed in the >> commercial/compliance required space. Instead of arguing blankly against >> inclusion can you at least provide actionable based feedback that if met >> would allow patches of this magnitude in? I am *the* first person to criticise EDB. There isn't a person in EDB that would find that surprising. That said, Robert, Dave, and all the other contributors that work for EDB are stand up people. They deserve our respect. Period. I appreciate the thought process here but if you want any kind of legitimacy in this community I suggest you do your homework. JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.