Обсуждение: improving on pgrminclude / pgcompinclude ?

Amit's complaint about unnecessary headers got me thinking about
src/tools/pginclude.  I think the the stuff in this directory is
trying to do something useful, because I think extra #include
directives that we don't need are a useful thing to eliminate.
However, in practice the contents are very hard for anybody to use
except maybe Bruce, who wrote it with his own environment in mind.  On
my system, any attempt to rerun pgcompinclude -- which is described as
a prerequisite to running pgrminclude -- produces enormous numbers of
bogus errors and warnings, both because of the attempt to call every
defined macro with made-up arguments and because I configure with
stuff like --with-includes=/opt/local/include which pgcompinclude
knows nothing about.

I wonder if we can do better.  The attached patch is the rest of a
couple of hours of hacking on a "see whether all of our includes
compile separately" project.  Directions:

1. configure
2. go to src/tools/pginclude
3. make test-compile-include

It strikes me that if we could make this robust enough to include in
the buidlfarm, that would be good.  And maybe we could then have a
pgrminclude tool which is a bit smarter also, and something that
anyone can easily run.  I don't think pgrminclude should be part of
the buildfarm, but being able to run it every release cycle with
relatively minimal pain instead of every 5 years with extensive
fallout seems like it would be a win.

Thoughts?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:
> I wonder if we can do better.  The attached patch is the rest of a
> couple of hours of hacking on a "see whether all of our includes
> compile separately" project.  Directions:

Don't we have more or less that already in src/tools/pginclude/cpluspluscheck?

> It strikes me that if we could make this robust enough to include in
> the buidlfarm, that would be good.  And maybe we could then have a
> pgrminclude tool which is a bit smarter also, and something that
> anyone can easily run.  I don't think pgrminclude should be part of
> the buildfarm, but being able to run it every release cycle with
> relatively minimal pain instead of every 5 years with extensive
> fallout seems like it would be a win.

I do not think that pgrminclude will *ever* be something that can be
quasi-automatic or work without close adult supervision.  In the
first place, whether a given reference is required or not can easily
vary depending on platform and compile options.  In the second place,
when there is something removable it's often because there are multiple
inclusion pathways reaching the same header file, and figuring out
which one is most appropriate to keep is a judgment call that requires
some understanding of what we think the system structure ought to be.

In fact, undoing the mess that the last pgrminclude run created was
sufficiently painful that I'm not sure I want to see it run again ever.
A few unnecessary includes isn't that big a problem.

Having said that, part of the reason for the carnage was that someone
had previously created a circular #include loop.  It would be good to
have a tool that would whine about that; or at least fix pgrminclude
so it would refuse to do anything if it found one.
        regards, tom lane

On Wed, Oct 5, 2016 at 12:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> I wonder if we can do better.  The attached patch is the rest of a
>> couple of hours of hacking on a "see whether all of our includes
>> compile separately" project.  Directions:
>
> Don't we have more or less that already in src/tools/pginclude/cpluspluscheck?

Hmm, yeah, similar.  I didn't know about that.  But mine tries to get
the Makefile system to run the build, so that you compile it the same
way you are actually compiling the source tree, which is pretty darn
important.  cpluspluscheck fails miserably on my system, because it
doesn't know about the additional include directories I've configured.

>> It strikes me that if we could make this robust enough to include in
>> the buidlfarm, that would be good.  And maybe we could then have a
>> pgrminclude tool which is a bit smarter also, and something that
>> anyone can easily run.  I don't think pgrminclude should be part of
>> the buildfarm, but being able to run it every release cycle with
>> relatively minimal pain instead of every 5 years with extensive
>> fallout seems like it would be a win.
>
> I do not think that pgrminclude will *ever* be something that can be
> quasi-automatic or work without close adult supervision.  In the
> first place, whether a given reference is required or not can easily
> vary depending on platform and compile options.  In the second place,
> when there is something removable it's often because there are multiple
> inclusion pathways reaching the same header file, and figuring out
> which one is most appropriate to keep is a judgment call that requires
> some understanding of what we think the system structure ought to be.
>
> In fact, undoing the mess that the last pgrminclude run created was
> sufficiently painful that I'm not sure I want to see it run again ever.
> A few unnecessary includes isn't that big a problem.

No, but a lot of unnecessary includes eventually sucks.

> Having said that, part of the reason for the carnage was that someone
> had previously created a circular #include loop.  It would be good to
> have a tool that would whine about that; or at least fix pgrminclude
> so it would refuse to do anything if it found one.

I think there's basically no hope of making pgrminclude very smart as
long as it is a shell script.  If we can use perl and integrate with
the Makefiles, I think it could be made much smarter.  I agree that
there's not much chance of running it without close human supervision
and the application of human judgement, but I also think that the
current tool is pretty much hopeless.  If we improve the tool so that
it isn't so difficult to run and doesn't do so many silly things, we
can probably reduce the pain considerably.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company