Re: Bootstrap DATA is a pita

On 3/4/15 9:07 AM, Stephen Frost wrote:
> * Robert Haas (robertmhaas@gmail.com) wrote:
>> On Wed, Mar 4, 2015 at 9:42 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>>> and make it harder to compare entries by grepping out some common
>>>>> substring.
>>>
>>>> Could you give an example of the sort of thing you wish to do?
>>>
>>> On that angle, I'm dubious that a format that allows omission of fields is
>>> going to be easy for editing scripts to modify, no matter what the layout
>>> convention is.  I've found it relatively easy to write sed or even Emacs
>>> macros to add new column values to old-school pg_proc.h ... but in this
>>> brave new world, I'm going to be really hoping that the column default
>>> works for 99.9% of pg_proc entries when we add a new pg_proc column,
>>> because slipping a value into a desired position is gonna be hard for
>>> a script when you don't know whether the adjacent existing fields are
>>> present or not.
>>
>> I wonder if we should have a tool in our repository to help people
>> edit the file.  So instead of going in there yourself and changing
>> things by hand, or writing your own script, you can do:
>>
>> updatepgproc.pl --oid 5678 provolatile=v
>>
>> or
>>
>> updatepgpproc.pl --name='.*xact.*' prowhatever=someval
>>
>> Regardless of what format we end up with, that seems like it would
>> make things easier.
>
> Alright, I'll bite on this- we have this really neat tool for editing
> data in bulk, or individual values, or pulling out data to look at based
> on particular values or even functions...  It's called PostgreSQL.
>
> What if we had an easy way to export an existing table into whatever
> format we decide to use for initdb to use?  For that matter, what if
> that file was simple to import into PG?
>
> What about having a way to load all the catalog tables from their git
> repo files into a "pg_dev" schema?  Maybe even include a make target or
> initdb option which does that?  (the point here being to provide a way
> to modify the tables and compare the results to the existing tables
> without breaking the instance one is using for this)
>
> I have to admit that I've never tried to do that with the existing
> format, but seems like an interesting idea to consider.  I further
> wonder if it'd be possible to generate the table structures too..

Semi-related... if we put some special handling in some places for 
bootstrap mode, couldn't most catalog objects be created using SQL, once 
we got pg_class, pg_attributes and pg_type created? That would 
theoretically allow us to drive much more of initdb with plain SQL 
(possibly created via pg_dump).
-- 
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com