Hi,
We have some issues with CI on macos and windows being too expensive (more on
that soon in a separate email). For macos most of the obviously wasted time is
spent installing packages with homebrew. Even with the package downloads being
cached, it takes about 1m20s to install them. We can't just cache the whole
homebrew installation, because it contains a lot of pre-installed packages.
After a bunch of experimenting, I found a way to do this a lot faster: The
attached patch installs macports and installs our dependencies from
that. Because there aren't pre-existing macports packages, we can just cache
the whole thing. Doing so naively wouldn't yield that much of a speedup,
because it takes a while to unpack a tarball (or whatnot) with as many files
as our dependencies have - that's a lot of filesystem metadata
operations. Instead the script creates a HFS+ filesystem in a file and caches
that - that's mounted within a few seconds. To further keep the size in check,
that file is compressed with zstd in the cache.
As macports has a package for IPC::Run and IO::Pty, we can use those instead
of the separate cache we had for the perl installation.
After the patch, the cached case takes ~5s to "install" and the cache is half
the size than the one for homebrew.
The comparison isn't entirely fair, because I used the occasion to not install
'make' (since meson is used for building) and llvm (we didn't enable it for
the build anyway). That gain is a bit smaller without that, but still
significant.
An alternative implementation would be to create the "base" .dmg file outside
of CI and download it onto the CI instances. But I didn't want to figure out
the hosting situation for such files, so I thought this was the best
near-medium term path.
Greetings,
Andres Freund