PG vs macOS Mojave

Well, macOS 10.14 (Mojave) is out, so I installed it on a spare machine,
and naturally the first thing I tried was to build PG with it.  Our
core code seems fine, but:

* --with-perl fails in configure, complaining that it can't find perl.h.

* --with-tcl fails in configure, complaining that it can't find
tclConfig.sh.  Furthermore, the historical workaround for that
(--with-tclconfig=/System/Library/Frameworks/Tcl.framework) doesn't fix it.

After some investigation, it seems that Apple has been busy moving
header files (not libraries) under SDK-specific "sysroot" directories,
with the expectation that you'd compile using "-isysroot $SYSROOT".
There's some mention of that here, for example:


https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/cross_development/Configuring/configuring.html#//apple_ref/doc/uid/10000163i-CH1-SW7

The sysroot seems to contain only headers; stuff you need at runtime,
such as shared libraries, is still where it used to be.

The recommended way to get the appropriate sysroot path seems to be

SYSROOT=`xcodebuild -version -sdk macosx Path`

Attached is a draft patch to fix things up.  The core ideas are

(1) Stop assuming that the Perl headers and library are necessarily
in the same place; create a perl_includedir variable to represent the
path to the former.

(2) Tweak src/template/darwin to inject the appropriate -isysroot
option into CPPFLAGS.

(3) Remove the need to manually specify the path to tclConfig.sh,
which has gotten even more painful than before because now it's
somewhere under the sysroot.  You can still specify --with-tclconfig
if you really want to, but it's not necessary anymore to build pltcl
under recent macOS.

Note that (3) alone is not sufficient to fix pltcl; we must do (2)
as well because tclConfig.sh now reports the Tcl include flags as
TCL_INCLUDE_SPEC        = -iwithsysroot /System/Library/Frameworks/Tcl.framework/Versions/8.5/Headers
so unless we also set -isysroot this doesn't work.

It's a bit scary to be adding -isysroot globally.  I thought
briefly about using it only while building pltcl, but that seems
even more dangerous: if there were any discrepancies between the
headers in the sysroot and those in the normal include directories,
building pltcl with different headers from the rest of the system
would surely be disastrous.  In any case, I suspect that the handwriting
is on the wall, and before very much longer it's going to be impossible
to build meaningful code on macOS without -isysroot anyway.

I've tested this on all the macOS versions I have at hand, and it
doesn't seem to break anything.  Only part (1) could possibly
affect other platforms, and that seems safe enough.

I'd like to commit and backpatch this, because otherwise longfin
is going to start falling over when I upgrade its host to Mojave.

Thoughts?

            regards, tom lane

diff --git a/configure b/configure
index 21ecd29..879eee4 100755
--- a/configure
+++ b/configure
@@ -668,6 +668,7 @@ python_majorversion
 PYTHON
 perl_embed_ldflags
 perl_embed_ccflags
+perl_includedir
 perl_useshrplib
 perl_privlibexp
 perl_archlibexp
@@ -9773,6 +9774,14 @@ You might have to rebuild your Perl installation.  Refer to the
 documentation for details.  Use --without-perl to disable building
 PL/Perl." "$LINENO" 5
   fi
+  # On most platforms, archlibexp is also where the Perl include files live ...
+  perl_includedir="$perl_archlibexp"
+  # ... but on some macOS versions, we must look under $PG_SYSROOT instead
+  if test x"$PG_SYSROOT" != x"" ; then
+    if test -d "$PG_SYSROOT$perl_archlibexp" ; then
+      perl_includedir="$PG_SYSROOT$perl_archlibexp"
+    fi
+  fi

 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for CFLAGS recommended by Perl" >&5
 $as_echo_n "checking for CFLAGS recommended by Perl... " >&6; }
@@ -18355,7 +18364,7 @@ fi
 # check for <perl.h>
 if test "$with_perl" = yes; then
   ac_save_CPPFLAGS=$CPPFLAGS
-  CPPFLAGS="$CPPFLAGS -I$perl_archlibexp/CORE"
+  CPPFLAGS="$CPPFLAGS -I$perl_includedir/CORE"
   ac_fn_c_check_header_compile "$LINENO" "perl.h" "ac_cv_header_perl_h" "#include <EXTERN.h>
 "
 if test "x$ac_cv_header_perl_h" = xyes; then :
diff --git a/configure.in b/configure.in
index 8fe6894..530f275 100644
--- a/configure.in
+++ b/configure.in
@@ -1044,6 +1044,15 @@ You might have to rebuild your Perl installation.  Refer to the
 documentation for details.  Use --without-perl to disable building
 PL/Perl.])
   fi
+  # On most platforms, archlibexp is also where the Perl include files live ...
+  perl_includedir="$perl_archlibexp"
+  # ... but on some macOS versions, we must look under $PG_SYSROOT instead
+  if test x"$PG_SYSROOT" != x"" ; then
+    if test -d "$PG_SYSROOT$perl_archlibexp" ; then
+      perl_includedir="$PG_SYSROOT$perl_archlibexp"
+    fi
+  fi
+  AC_SUBST(perl_includedir)dnl
   PGAC_CHECK_PERL_EMBED_CCFLAGS
   PGAC_CHECK_PERL_EMBED_LDFLAGS
 fi
@@ -2229,7 +2238,7 @@ fi
 # check for <perl.h>
 if test "$with_perl" = yes; then
   ac_save_CPPFLAGS=$CPPFLAGS
-  CPPFLAGS="$CPPFLAGS -I$perl_archlibexp/CORE"
+  CPPFLAGS="$CPPFLAGS -I$perl_includedir/CORE"
   AC_CHECK_HEADER(perl.h, [], [AC_MSG_ERROR([header file <perl.h> is required for Perl])],
                   [#include <EXTERN.h>])
   # While we're at it, check that we can link to libperl.
diff --git a/contrib/hstore_plperl/Makefile b/contrib/hstore_plperl/Makefile
index 32ecaa4..d0a3916 100644
--- a/contrib/hstore_plperl/Makefile
+++ b/contrib/hstore_plperl/Makefile
@@ -39,4 +39,4 @@ endif
 # last, probably because it sometimes contains some header files with names
 # that clash with some of ours, or with some that we include, notably on
 # Windows.
-override CPPFLAGS := $(CPPFLAGS) $(perl_embed_ccflags) -I$(perl_archlibexp)/CORE
+override CPPFLAGS := $(CPPFLAGS) $(perl_embed_ccflags) -I$(perl_includedir)/CORE
diff --git a/contrib/jsonb_plperl/Makefile b/contrib/jsonb_plperl/Makefile
index eb6d1de..019fc82 100644
--- a/contrib/jsonb_plperl/Makefile
+++ b/contrib/jsonb_plperl/Makefile
@@ -39,4 +39,4 @@ endif
 # last, probably because it sometimes contains some header files with names
 # that clash with some of ours, or with some that we include, notably on
 # Windows.
-override CPPFLAGS := $(CPPFLAGS) $(perl_embed_ccflags) -I$(perl_archlibexp)/CORE
+override CPPFLAGS := $(CPPFLAGS) $(perl_embed_ccflags) -I$(perl_includedir)/CORE
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 91d7cb8..9cf0c35 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -331,6 +331,7 @@ else
 endif
 perl_archlibexp        = @perl_archlibexp@
 perl_privlibexp        = @perl_privlibexp@
+perl_includedir        = @perl_includedir@
 perl_embed_ccflags    = @perl_embed_ccflags@
 perl_embed_ldflags    = @perl_embed_ldflags@

diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile
index 39dacf8..baf09b4 100644
--- a/src/pl/plperl/GNUmakefile
+++ b/src/pl/plperl/GNUmakefile
@@ -16,7 +16,7 @@ endif
 # probably because it sometimes contains some header files with names
 # that clash with some of ours, or with some that we include, notably on
 # Windows.
-override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS) $(perl_embed_ccflags) -I$(perl_archlibexp)/CORE
+override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS) $(perl_embed_ccflags) -I$(perl_includedir)/CORE

 rpathdir = $(perl_archlibexp)/CORE

diff --git a/src/template/darwin b/src/template/darwin
index ea6d3b0..ed0ebcc 100644
--- a/src/template/darwin
+++ b/src/template/darwin
@@ -3,6 +3,18 @@
 # Note: Darwin is the original code name for macOS, also known as OS X.
 # We still use "darwin" as the port name, partly because config.guess does.

+# Select where system include files should be sought.
+if test x"$PG_SYSROOT" = x"" ; then
+  PG_SYSROOT=`xcodebuild -version -sdk macosx Path 2>/dev/null`
+fi
+if test x"$PG_SYSROOT" != x"" ; then
+  if test -d "$PG_SYSROOT" ; then
+    CPPFLAGS="$CPPFLAGS -isysroot $PG_SYSROOT"
+  else
+    PG_SYSROOT=""
+  fi
+fi
+
 # Select appropriate semaphore support.  Darwin 6.0 (macOS 10.2) and up
 # support System V semaphores; before that we have to use named POSIX
 # semaphores, which are less good for our purposes because they eat a
@@ -15,3 +27,11 @@ case $host_os in
     USE_SYSV_SEMAPHORES=1
     ;;
 esac
+
+# If user didn't specify something else, expect that tclConfig.sh can be
+# found here:
+if test x"$with_tclconfig" = x"" ; then
+  if test -f "$PG_SYSROOT/System/Library/Frameworks/Tcl.framework/tclConfig.sh" ; then
+    with_tclconfig="$PG_SYSROOT/System/Library/Frameworks/Tcl.framework"
+  fi
+fi