Re: Overhauling GUCS

Simon, Tom, Greg,

> Now, that will get a lot easier just by
> virtue of having a smaller config file, but I think that adding something
> into pg_settings that allows saving user-added commentary would be a nice
> step toward some useful standardization on that side of things.  It would
> make future automated tools aimed at parsing and generating new files, as
> part of things like version upgrades, a lot easier if there was a standard
> way such comments were handled in addition to the raw data itself.

Hmmm.  What about a COMMENT ON SETTING?  That seems like it would serve the 
purpose ... and make preserving user comments easier than the current 
file-conversion approach.

> The other thing I'd like to see make its way into pg_settings, so that
> tools can operate on it just by querying the database, is noting what file
> the setting came from so that you can track things like include file
> usage.  I think with those two additions (comments and source file
> tracking) it would even be concievable to clone a working facsimile of
> even a complicated postgresql.conf file set remotely just by reading
> pg_settings.

Hmmm.  Any idea how to do this?  It sounds like a good idea to me.

> So my viewpoint is that we should be aggressively adding new features,
> yet be conservative in changing existing behaviour: provide options for
> behaves-like-previous-release and keep the administrative interfaces as
> similar as possible between releases.

It's my viewpoint based on a lot of user feedback that the current 
postgresql.conf is fundamentally broken and a major roadblock to PostgreSQL 
adoption. This was a point with which there was pretty much universal 
agreement when I talked with people at pgCon.  That is, I beleive that you're 
arguing for keeping .conf stable for the 5% of users who understand it and 
ignoring the 95% of users who are baffled by it.

At this point, I think we are the only major SQL database without some form of 
basic autotuning for the most significant settings (certainly MySQL, Oracle, 
and SQL Server have it); we've been able to coast through that because 
autotuning features are new in other DBs, but it's going to start to hurt us 
pretty soon.

Now, I do believe that we should plan any GUCS overhaul to happen in one 
postgresql version rather than phasing it in over multiple versions so that 
administrators only need to get used to a new way once.

> > Currently, PostgreSQL,conf and our set of GUC configurations suffer from
> > 4 large problems:
> >
> > 1. Most people have no idea how to set these.
> > 2. The current postgresql.conf file is a huge mess of 194 options, the
> > vast majority of which most users will never touch.
> > 3. GUCS lists are kept in 3 different places (guc.c, postgresql.conf,
> > and the settings.sgml), which are only synched with each other manually.
> > 4. We don't seem to be getting any closer to autotuning.
>
> The proposal doesn't actually solve any of those problems.

OK, let me draw some lines:
1 & 2) by not having the settings be defined in a 500-line file, new users 
would no longer be baffled by scores of settings which probably don't concern 
them, trying to find the handful of settings which do.

3) We'd consolidate the GUC lists down from 3 places to 2, which is one less 
area to synchronize.  Magnus and I looked to see if it might be possible to 
generate the docs from the same list, but it's not practical.

4) By shifting from a model where postgresql.conf is document-formatted and 
hand-edited to one where it's machine generated, it becomes vastly easier to 
write simple utilities to manage these settings.  Right now, the big 
"obstacle" to things like SET PERSISTENT is "how to we preseve the 
hand-edited comments in the file" -- and the answer is we *don't.*  

> I disagree with doing any of this.  It doesn't result in any useful
> reduction in maintenance effort, and what it does do is make it
> impossible to keep control over the detailed layout, formatting,
> commenting etc in a sample postgresql.conf.  

Have you *looked* at postgresql.conf.sample lately, Tom?  It's a disaster.  
Maintenance is already difficult, and becoming more so as we add settings.

Further, you and Simon seem to think that the current "narrative docs inside 
the conf file" format has some kind of value which makes things easier for 
DBAs.  I don't believe it does, and I have yet to meet a *single* new 
PostgreSQL user who wasn't confused and intimidated by the file.

> Nor do I think that 
> "generate a whole config file from scratch" is going to be a useful
> behavior for tuning problems --- how will you merge it with what
> you had before?

Who's merging?  I don't think you get the proposal.  The whole "hand-edited" 
approach to postgresql.conf should go away.  It's not useful, it's not 
educational, and it doesn't make PostgreSQL easy to manage.

Further, the lack of an easy way to manage settings via port access to 
PostgreSQL is a significant inhibitor to adopting PostgreSQL in environments 
with large numbers of servers.  See prior arguments by the CPANEL folks about 
why they don't support PostgreSQL, which is in turn a major reason why 
PostgreSQL web hosting is hard to find.  

I agree that editing the data about settings in the guc.c file is not ideal; 
Magnus and I discussed that mainly because we wanted to preserve the 
translation framework with gettext strings.  If someone can think of a better 
way to do this part, I'm all ears.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco