Re: dynloader.h missing in prebuilt package for Windows?

On Fri, Dec  4, 2015 at 07:09:03PM -0500, Chapman Flack wrote:
> On 12/04/15 12:56, Bruce Momjian wrote:
> > 
> > OK, good logical reason to install dynloader.h on Windows.
> 
> Ah!  Thanks.  I was starting to wonder whether I had said something wrong
> and been sent to the bit bucket within my first two -hackers posts. :)

No, sometimes people just don't know how to respond, particuarly related
to technology we don't use regularly.

> > What do we need to do to close this item?  What versions of Windows
> > installers are missing dynloader.h?  Mingw?  MSVC?  EDB's?  OpenSCG?
> > Postgres Pro (Russian)?
> 
> I am too new around here to be able to supply good answers myself to
> those questions.  I keep learning though.  For example, I have just
> learned that OpenSCG and Postgres Pro (Russian) are things, and (by
> inference) that they run on Windows. :)

Yes, there are several binary Postgres distributions.  However, in
researching, I think there is a central way to fix them all.

> In working on PL/Java, as a non-Windows dev myself, I have been
> blessed to find Ken Olson willing to build my WIP branches in MSVC
> and tell me what I've busted. I think he's building against an EDB
> installation of PostgreSQL, so that's the combination I am least
> ignorant about. I know that mingw has also been used, but I haven't
> yet made a connection with someone who can tell me what I'm breaking
> for that build....

OK, that sounds good.  I assume he is using MSVC to build PL/Java, and
then using the EDB server binaries, which should work fine.

> > Is this a change we need to make on the server end and then all the
> > installers will automatically install the file?  It is present in all
> > Unix-like installs?
> 
> AFAICS, it must at one time have been envisioned that sometimes
> extension code might like a nice dynloader; the whole way that
> backend/port/dynloader contains a version for every platform, and
> the configure script links the appropriate one into src/include with
> all the other .h files that would go into a -devel package, makes me
> _think_ that it'd be present in any install that used configure in
> the build. Anyone building a -devel package would have to go out
> of their way to exclude it but still package the other .h files.

Yes, it should.  Looking at the 'install' Makefile rule in
include/Makefile I see:
       cp $(srcdir)/*.h '$(DESTDIR)$(includedir_server)'/ || exit; \       chmod $(INSTALL_DATA_MODE)
'$(DESTDIR)$(includedir_server)'/*.h || exit; \       for dir in $(SUBDIRS); do \         cp $(srcdir)/$$dir/*.h
'$(DESTDIR)$(includedir_server)'/$$dir/|| exit; \         chmod $(INSTALL_DATA_MODE)
'$(DESTDIR)$(includedir_server)'/$$dir/*.h || exit; \       done
 

This copies all the *.h files in src/include during install.  If I look
at my Debian source install, I see the dynloader.h installed as
include/server/dynloader.h, which is what you want.  In fact, there are
many include files installed in include/server:
c.h            pg_config_ext.h     pgtime.h     postgres_ext.hdynloader.h    pg_config.h         pg_trace.h
postgres_fe.hfmgr.h        pg_config_manual.h  plperl.h     postgres.hfuncapi.h      pg_config_os.h      plpgsql.h
ppport.hgetaddrinfo.h pg_getopt.h         plpython.h   rusagestub.hgetopt_long.h  pgstat.h            plpy_util.h
windowapi.hmiscadmin.h   pgtar.h             port.h
 

> So I'd guess that Windows builds that don't use configure are probably
> the odd players out. But I don't have any contacts or name recognition
> to approach { EDB, OpenSCG, Postgres Pro } and find out how their
> internal build processes work, or whether they all end up lacking
> the file, or whether there is any single change that could be made
> in the PG source tree so that their builds would then all include it.

In fact, there is a single file that affects all the MSVC-based Windows
installers.  All the MSVC build stuff happens in src/tools/msvc, and it
is mostly written in Perl.  This is the corresponding line in the MSVC Perl 
file Install.pm:
CopySetOfFiles(        '',        [ glob("src\\include\\*.h") ],        $target . '/include/server/');

So, for me, the big question is why dynloader.h isn't getting copied
into /include/server.  (There is a mention the 'Makefile' about vpath
builds needing to copy dynloader.h manually --- is vpath being used for
the Windows installers somehow?)

Can you show me what files you have in /include/server, without your
copying the dynloader.h file manually?  Where did you get that Windows
installer?

> >  Also, I am very glad you are working on PL/Java.  :-)
> 
> Thanks!  It has been interesting trying to get up to speed, both on
> how it technically works, and also on the development history, why it
> lives out-of-tree, and so on. (That question had puzzled me for a long
> time, and when I finally found the long 2006 -hackers thread,
> http://www.postgresql.org/message-id/44B3952B.2080401@tada.se
> I was like your genealogy-obsessed aunt finding a trunk in the attic. :)

Yes, it took us a long time to work out the logic of what should be
external.

> I can see that a lot of the considerations raised in that thread, both
> ways, are still pertinent today, plus with nine more years behind us
> to see how things have actually played out. Tom Lane was concerned about
> what would happen if Thomas's time for maintenance became scarce, and
> what seems to happen is someone like Johann Oskarsson, or yours truly,
> will step up, flounder a bit to get over the learning curve, and then
> carry the ball some distance. There is a bit of a barrier to entry,
> because PostgreSQL and Java are each large and deep and PL/Java has
> both, and for the future vigor of the project it seems important to
> find the ways to minimize that barrier. (I know I'm getting OT from
> dynloader here, but this other stuff has been on my mind a while.)
> 
> That doesn't require reopening the in-tree question necessarily
> (I saw that Tom Lane was concerned about a buildfarm going all red
> because of a problem too few people could easily fix), but it would
> probably be very helpful to at least have some kind of _associated
> buildfarm_ so the main board might not go all red, but it would still
> be easy to see right away if a change was going to affect important
> out-of-tree components.
>
> That seems to have been a worthwhile idea nine years ago (I read that
> Andrew Dunstan _almost_ found the time to set it up then), and
> still today something like that would be very helpful. PL/Java still needs
> work on an easily-automatable suite of tests (that much, I'm working on),
> and once that's in place, the next obvious and valuable step would be to get
> it building continuously on the several major platforms, so it can turn red
> on some board that the team can easily see. I might ask for some help or
> suggestions on what are the currently most-favored ways to do that.

Yes, you can either add a buildfarm member of your own, or get someone
who is already on the buildfarm to build PL/Java as part of their build
process:
http://buildfarm.postgresql.org/

> I think that with more automated testing and CI, the barrier to entry on
> PL/Java contributions will be a lot lower, because it is much less
> intimidating to start poking at something when it is easy to see what
> happens.

Agreed.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription                             +