Re: Speed up clean meson builds by ~25%

On Sat, May 18, 2024 at 6:09 PM Andres Freund <andres@anarazel.de> wrote:
> A few tests with ccache disabled:

These tests seem to show no difference between the two releases, so I
wonder what you're intending to demonstrate here.

Locally, I have ninja 1.11.1. I'm sure Andres will be absolutely
shocked, shocked I say, to hear that I haven't upgraded to the very
latest.

Anyway, I tried a clean build with CC=clang without the patch and then
with the patch and got:

unpatched - real 1m9.872s
patched - real 1m6.130s

That's the time to run my script that first calls meson setup and then
afterward runs ninja. I tried ninja -v and put the output to a file.
With the patch:

[292/2402] /opt/local/bin/perl
../pgsql/src/interfaces/ecpg/preproc/parse.pl --srcdir
../pgsql/src/interfaces/ecpg/preproc --parser
../pgsql/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y
--output src/interfaces/ecpg/preproc/preproc.y

And without:

[1854/2402] /opt/local/bin/perl
../pgsql/src/interfaces/ecpg/preproc/parse.pl --srcdir
../pgsql/src/interfaces/ecpg/preproc --parser
../pgsql/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y
--output src/interfaces/ecpg/preproc/preproc.y

With my usual CC='ccache clang', I get real 0m37.500s unpatched and
real 0m37.786s patched. I also tried this with the original v1 patch
(not to be confused with the revised, still-v1 patch) and got real
37.950s.

So I guess I'm now feeling pretty unexcited about this. If the patch
really did what the subject line says, that would be nice to have.
However, since Tom will patch the underlying problem in v18, this will
only help people building the back-branches. Of those, it will only
help the people using a slow compiler; I read the thread as suggesting
that you need to be using clang rather than gcc and also not using
ccache. Plus, it sounds like you also need an old ninja version. Even
then, the highest reported savings is 10s and I only save 3s. I think
that's a small enough savings with enough conditionals that we should
not bother. It seems fine to say that people who need the fastest
possible builds of back-branches should use at least one of gcc,
ccache, and ninja >= 1.12.

I'll go mark this patch Rejected in the CommitFest. Incidentally, this
thread is an excellent object lesson in why it's so darn hard to cut
down the size of the CF. I mean, this is a 7-line patch and we've now
got a 30+ email thread about it. In my role as temporary
self-appointed wielder of the CommitFest mace, I have now spent
probably a good 2 hours trying to figure out what to do about it.
There are more than 230 active patches in the CommitFest. That means
CommitFest management would be more than a full-time job for an
experienced committer even if every patch were as simple as this one,
which is definitely not the case. If we want to restore some sanity
here, we're going to have to find some way of distributing the burden
across more people, including the patch authors. Note that Jelte's
last update to the thread is over a month ago. I'm not saying that
he's done something wrong, but I do strongly suspect that the time
that the community as whole has spent on this patch is a pretty
significant multiple of the time that he personally spent on it, and
such dynamics are bound to create scaling issues. I don't know how we
do better: I just want to highlight the problem.

--
Robert Haas
EDB: http://www.enterprisedb.com