Re: Merged Model for libpq

On Mon, Apr 4, 2011 at 9:31 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Annamalai Gurusami <annamalai.gurusami@gmail.com> writes:
>> On 2 April 2011 11:17, John R Pierce <pierce@hogranch.com> wrote:
>>> what you describe is neither postgres nor SQL
>>> perhaps you should look at a storage engine like BerkeleyDB
>
>> I hope that not everybody dismisses this mail thread because of the
>> above response.  We are deriving our product from pgsql.  And since we
>> are customizing pgsql to our proprietary telecom products, we are
>> using things that are not designed for that purpose.  For example, we
>> are using SPI to come up with an embedded client.  I was basically
>> trying to find out if there are better alternatives.  Have the pgsql
>> development team thought about embedded clients and is SPI the way to
>> go?
>
> I don't think you've entirely grasped the seriousness of that response.
> The PG development team *has* thought about embedded scenarios, and
> explicitly rejected them.  There is no interest at all here in that line
> of development, and we are unlikely to even consider patches that might
> make it easier.  We don't like the reliability implications of having
> random client code in the same address space as the database code.
> Moreover, the general trend of recent development has been towards
> making the database more, not less, dependent on auxiliary processes
> such as autovacuum and bgwriter.  There's no way to manage that in an
> embedded scenario ... at least not without resorting to threads, which
> is another thing that we are unprepared to support.
>
> So really you should be looking at some other DBMS if you want an
> embedded implementation.  It'd be nice if PG could be all things to all
> people, but it can't; and this is one of the things it can't be.

That's a perhaps overly strong statement.  First of all, we already
support user provided code (in C no less) in the database.  It is raw
and problematic for most people but it's also pretty cool.

True embedding where the user application is in direct control of the
process is of course not practical, but that doesn't mean a tighter
coupling of user code and the database is not possible.  Stored
procedures (I know I'm way into broken record mode on this) would
likely cover what Annamalai is looking to do IMSNO, even if they were
limited to a high level language like plpgsql, since you could still
dip into C appropriately using classic methods.  Getting there isn't
easy, of course.

In the current state of affairs you can kinda sorta emulate this by
having a client side 'ticker' that dials in every period of time and
executes a control function which kicks off your server side logic and
maintains this state.  That way the bulk of your code and data
manipulation is database side and you more or less completely bypass
the overhead of streaming information through the protocol to the
client unless you want to pay that cost.  There are a lot of reasons
not to do this...it's a 'wrong tool' type of thing, but people want to
do it and it's interesting to think about what doors you could open if
it could be taken further.

merlin