Обсуждение: Building PostgreSQL in external directory is broken?

Hackers,

I usually build the PostgreSQL from the sources directory.  But I've
heard a complaint that PostgreSQL can't be built in the external
directory.

Assuming postgres sources located in postgresql directory, the
following sequence of commands

mkdir -p pgbld
cd pgbld
../postgresql/configure --disable-debug --disable-cassert --enable-tap-tests
make -j4

results in an error

.../src/pgbld/../postgresql/src/include/utils/elog.h:73:10: fatal
error: utils/errcodes.h: No such file or directory
   73 | #include "utils/errcodes.h"

      |          ^~~~~~~~~~~~~~~~~~

I've discovered this a bit.  It appears that Makefile generates
src/backend/utils/errcodes.h in the build directory, but symlinks
src/include/utils/errcodes.h to the sources directory.  That is
src/include/utils/errcodes.h appears to be a broken symlink.  I've
written a short patch to fix that.

It seems strange to me that I'm the first one discovering this.  Am I
missing something?

------
Regards,
Alexander Korotkov

Hi Alexander,

> Assuming postgres sources located in postgresql directory, the
> following sequence of commands
>
> mkdir -p pgbld
> cd pgbld
> ../postgresql/configure --disable-debug --disable-cassert --enable-tap-tests
> make -j4
>
> ...
>
> It seems strange to me that I'm the first one discovering this.  Am I
> missing something?

To be honest, this is the first time I see anyone trying to build a
project that is using Autotools from an external directory :)  I
checked the documentation [1] and it doesn't seem that we claim to
support this.

Also I tried your patch on MacOS Monterey 12.4 and it didn't work. I
get the following error:

```
...
ar: cryptohash.o: No such file or directory
ar: hmac.o: No such file or directory
ar: sha1.o: No such file or directory
ar: sha2.o: No such file or directory
```

... with or without the patch.

My guess would be that the reason no one discovered this before is
that this is in fact not supported and/or tested on CI.

Could you give an example of when this can be useful?

[1]: https://www.postgresql.org/docs/current/install-short.html

-- 
Best regards,
Aleksander Alekseev

Aleksander Alekseev <aleksander@timescale.com> writes:

> Hi Alexander,
>
> To be honest, this is the first time I see anyone trying to build a
> project that is using Autotools from an external directory :)  I
> checked the documentation [1] and it doesn't seem that we claim to
> support this.
>
> [1]: https://www.postgresql.org/docs/current/install-short.html

That's the short version. The longer version² does claim it's supported:

    You can also run configure in a directory outside the source tree,
    and then build there, if you want to keep the build directory
    separate from the original source files. This procedure is called a
    VPATH build. Here's how:

    mkdir build_dir
    cd build_dir
    /path/to/source/tree/configure [options go here]
    make

- ilmari

[2] https://www.postgresql.org/docs/current/install-procedure.html

Hi Ilmari,

> That's the short version. The longer version² does claim it's supported:

You are right, I missed this. Thanks!

Regarding these errors:

> ar: cryptohash.o: No such file or directory
> ar: hmac.o: No such file or directory
> ...

This has something to do with the particular choice of the ./configure
flags on a given platform. If I omit:

> --disable-debug --disable-cassert --enable-tap-tests

I get completely different errors:

> clang: error: no such file or directory: 'replication/backup_manifest.o'
> clang: error: no such file or directory: 'replication/basebackup.o'

Apparently this should be checked carefully with different configure
flags combinations if we are going to continue maintaining this.

--
Best regards,
Aleksander Alekseev

On 2022-Jul-13, Alexander Korotkov wrote:

> results in an error
> 
> .../src/pgbld/../postgresql/src/include/utils/elog.h:73:10: fatal
> error: utils/errcodes.h: No such file or directory
>    73 | #include "utils/errcodes.h"
> 
>       |          ^~~~~~~~~~~~~~~~~~

Probably what is happening here is that you have build artifacts in the
source tree after having built there, and that confuses make so not
everything is rebuilt correctly when you call it from the external build
dit.  I suggest to "git clean -dfx" your source tree, then you can rerun
configure/make from the external builddir.

FWIW building in external dirs works fine.  I use it all the time.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Once again, thank you and all of the developers for your hard work on
PostgreSQL.  This is by far the most pleasant management experience of
any database I've worked on."                             (Dan Harris)
http://archives.postgresql.org/pgsql-performance/2006-04/msg00247.php

On Wed, Jul 13, 2022 at 3:19 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2022-Jul-13, Alexander Korotkov wrote:
>
> > results in an error
> >
> > .../src/pgbld/../postgresql/src/include/utils/elog.h:73:10: fatal
> > error: utils/errcodes.h: No such file or directory
> >    73 | #include "utils/errcodes.h"
> >
> >       |          ^~~~~~~~~~~~~~~~~~
>
> Probably what is happening here is that you have build artifacts in the
> source tree after having built there, and that confuses make so not
> everything is rebuilt correctly when you call it from the external build
> dit.  I suggest to "git clean -dfx" your source tree, then you can rerun
> configure/make from the external builddir.
>
> FWIW building in external dirs works fine.  I use it all the time.

You are right.  I made "make distclean" which appears to be not
enough.  With "git clean -dfx" correct symlink is generated.  Sorry
for the noise.

------
Regards,
Alexander Korotkov

On Wed, Jul 13, 2022 at 3:12 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
> > That's the short version. The longer version² does claim it's supported:
>
> You are right, I missed this. Thanks!
>
> Regarding these errors:
>
> > ar: cryptohash.o: No such file or directory
> > ar: hmac.o: No such file or directory
> > ...
>
> This has something to do with the particular choice of the ./configure
> flags on a given platform. If I omit:
>
> > --disable-debug --disable-cassert --enable-tap-tests
>
> I get completely different errors:
>
> > clang: error: no such file or directory: 'replication/backup_manifest.o'
> > clang: error: no such file or directory: 'replication/basebackup.o'
>
> Apparently this should be checked carefully with different configure
> flags combinations if we are going to continue maintaining this.

Please, check Alvaro's advise to run "git clean -dfx".  Helped to me.

------
Regards,
Alexander Korotkov

Alvaro, Alexander,

> Please, check Alvaro's advise to run "git clean -dfx".  Helped to me.

Thanks, `git clean -dfx` did the trick!

> Could you give an example of when this can be useful?

And now I can answer my own question. I can move all shell scripts I
typically use for the development from the repository and be sure they
are not going to be deleted by accident (by `git clean`, for
instance).

Very convenient.

-- 
Best regards,
Aleksander Alekseev

Aleksander Alekseev <aleksander@timescale.com> writes:
>> Could you give an example of when this can be useful?

> And now I can answer my own question. I can move all shell scripts I
> typically use for the development from the repository and be sure they
> are not going to be deleted by accident (by `git clean`, for
> instance).

FWIW, I gather that the upcoming switch to the meson build system
will result in *requiring* use of outside-the-source-tree builds.

            regards, tom lane