Re: Plugins redux (was Re: [PATCHES] PL instrumentation plugin

<br /><blockquote type="CITE"><pre>
<font color="#000000">ISTM there are two separate bits of functionality that the core backend</font>
<font color="#000000">needs to provide in support of PL plugins.  The first is that we need a</font>
<font color="#000000">"rendezvous" facility whereby two separately-loaded shared libraries can</font>
<font color="#000000">connect and exchange data (a struct of function pointers for the immediate</font>
<font color="#000000">plans for plpgsql, but in general it might be anything).  The exchanged</font>
<font color="#000000">data mustn't be limited to things known in advance to the core backend.</font>
<font color="#000000">The facility should support loading of communicating libraries in either</font>
<font color="#000000">order, and it should also allow for unloading of libraries.  This seems</font>
<font color="#000000">easy enough to design: the core backend should basically provide a</font>
<font color="#000000">hashtable that maps names to "void *" pointer variables, and then two</font>
<font color="#000000">libraries that want to communicate just agree on a name and the data</font>
<font color="#000000">structure to be pointed to.  (More details below.</font>)
</pre></blockquote><br /> I like it so far...<br /><br /><blockquote type="CITE"><pre>
<font color="#000000">The other, probably more controversial bit of functionality is that there</font>
<font color="#000000">needs to be a way to cause a backend to load a PL plugin shared library</font>
<font color="#000000">without any cooperation from the connected client application.  For</font>
<font color="#000000">interactive use of a PL debugger it might be sufficient to tell people to</font>
<font color="#000000">do "LOAD 'plpgsql_debugger'" before running their function-to-be-tested,</font>
<font color="#000000">but if you're trying to debug some behavior that only manifests in a large</font>
<font color="#000000">complicated application, it may be impractical to get the application to</font>
<font color="#000000">issue such a command. </font> 
</pre></blockquote><br /> Ok, but you should know that the PL/pgSQL debugger already handles this without any
cooperationfrom the client.  <br /><br /> Loading the PL debugger means that your backend becomes debuggable, not that
itactually starts debugging. <br /><br /> If you want to debug a PL/pgSQL function invoked by some other process (such
asa Web server), you create a "global breakpoint".  When a debuggable process (that is, a process that has loaded the
debuggerplugin) trips across the breakpoint, that's when it starts debugging.  (Note: a global breakpoint can identify
aparticular backend, by process ID, or it can omit the process ID and trap the first debuggable backend that trips
acrossthe breakpoint).<br /><br /> So, other than the (negligible) performance penalty, its safe to load the debugger
plugininto every backend.  (If you've loaded the debugger, the extra overhead is a single, lightweight-lock protected,
shared-memoryhash lookup for each PL function invocation).<br /><br /><blockquote type="CITE"><pre>
 
<font color="#000000">We agreed while discussing this at the PG</font>
<font color="#000000">Anniversary Conference that providing a GUC variable for this purpose is</font>
<font color="#000000">probably sufficient, as there are multiple ways that such a variable can</font>
<font color="#000000">be set on behalf of an application: ALTER USER SET, ALTER DATABASE SET,</font>
<font color="#000000">"export PGOPTIONS=--var=val" before starting the app, etc.  As submitted,</font>
<font color="#000000">the plugin patch envisions defining a custom GUC variable for each PL and</font>
<font color="#000000">having each PL add logic to force loading of any shared library mentioned</font>
<font color="#000000">in its specific variable.  I claim however that that is just duplicative,</font>
<font color="#000000">and that a single backend-wide GUC variable should be sufficient.</font>
</pre></blockquote><br /> Ok, so the rendevous variables replace the per-language GUC variables (from my patch).  In
effect,a GUC variable is something that a user would manage, a rendevous variable is something that two (or more)
piecesof code manage.  <br /><br /> GUC variables are visible to the user, rendevous variables are visible to
dynamicallyloaded libraries.  <br /><br /> Oh, and rendevous variables are insensitve to load order too.<br /><br />
Nicedesign.<br /><br /><blockquote type="CITE"><pre>
 
<font color="#000000">The principal difference between this proposal and the original patch</font>
<font color="#000000">is that this way supports unloading of PL plugins; AFAICS, the original</font>
<font color="#000000">patch would just crash if the plugin were unloaded and then use of plpgsql</font>
<font color="#000000">continued. </font>
</pre></blockquote><br /> I don't think it could crash because there's no way to unload a plugin (there is no UNLOAD
statement,is there?).<br /><br /> Which reminds me, you haven't proposed a way to unload a shared-library.<br /><br
/><blockquotetype="CITE"><pre>
 
<font color="#000000">We need to think about security issues in defining such a variable:</font>
<font color="#000000">we don't want unprivileged users to be able to load arbitrary code</font>
<font color="#000000">into the server.  One solution to this is to make the variable SUSET,</font>
<font color="#000000">ie, you must be superuser to set it.  This would make it impossible</font>
<font color="#000000">for people to debug their own functions without superuser privileges</font>
<font color="#000000">... but IIRC actual use of the proposed plpgsql debugger requires</font>
<font color="#000000">superuser privileges too, so this isn't necessarily a fatal objection.</font>
</pre></blockquote><br /> Correct (the debugger currently requires superuser privileges).  We'll probably want to relax
thatlater.<br /><br /><blockquote type="CITE"><pre>
 
<font color="#000000">Still, it'd be nice if the PGOPTIONS way of setting the variable could</font>
<font color="#000000">be used without requiring that the client app itself run as superuser.</font>
<font color="#000000">Plan B is to restrict which shared libraries can be loaded using the</font>
<font color="#000000">variable --- there are different ways we could do that, but my suggestion</font>
<font color="#000000">would be to insist that the shared library come from "$libdir/plugins/".</font>
<font color="#000000">The DBA is then responsible for allowing only "safe" shared libraries</font>
<font color="#000000">to be installed in that directory, and therefore we can allow unprivileged</font>
<font color="#000000">users to set "backend_load_libraries".  (We might as well also allow them</font>
<font color="#000000">to execute LOAD for these libraries, so that plugins could be switched</font>
<font color="#000000">intra-session without superuser privileges.)</font>
</pre></blockquote><br /> How about a combination of plan A and plan B?  Make backend_load_libraries a USERSET
variable,but you can't *add* libraries outside of $libdir/plugins/ unless you are a superuser.<br /><br /><blockquote
type="CITE"><pre>
<font color="#000000">BTW, is anyone up for renaming the existing "preload_libraries" variable</font>
<font color="#000000">to "postmaster_load_libraries"?  This would be more symmetrical with</font>
<font color="#000000">"backend_load_libraries", and so perhaps clearer about which does what</font>
</pre></blockquote><br /> Makes sense to me, of course that breaks existing postgresql.conf files.<br /><br /> Do you
wantme to do any of this coding?<br /><table cellpadding="0" cellspacing="0" width="100%"><tr><td><br /><br /> --<br />
 Korry Douglas    <a href="mailto:korryd@enterprisedb.com">korryd@enterprisedb.com</a><br />   EnterpriseDB      <a
href="http://www.enterprisedb.com">http://www.enterprisedb.com</a></td></tr></table>