Обсуждение: Having problems generating a code coverage report

Поиск
Список
Период
Сортировка

Having problems generating a code coverage report

От
Aleksander Alekseev
Дата:
Hi hackers,

Recently I tried to build a code coverage report according to the
documentation [1]. When using the following command:

```
time sh -c 'git clean -dfx && meson setup --buildtype debug
-DPG_TEST_EXTRA="kerberos ldap ssl" -Db_coverage=true -Dldap=disabled
-Dssl=openssl -Dcassert=true -Dtap_tests=enabled
-Dprefix=/home/eax/pginstall build && ninja -C build &&
PG_TEST_EXTRA=1 meson test -C build && ninja -C build coverage-html'
```

... I get:

```
geninfo: ERROR: Unexpected negative count '-3' for
/home/eax/projects/c/postgresql/src/port/snprintf.c:532.
    Perhaps you need to compile with '-fprofile-update=atomic
    (use "geninfo --ignore-errors negative ..." to bypass this error)
Traceback (most recent call last):
  File "/usr/bin/meson", line 40, in <module>
    sys.exit(mesonmain.main())
             ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
294, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
282, in run
    return run_script_command(args[1], args[2:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
223, in run_script_command
    return module.run(script_args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 206, in run
    return coverage(options.outputs, options.source_root,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 123, in coverage
    subprocess.check_call([lcov_exe,
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['lcov', '--directory',
'/home/eax/projects/c/postgresql/build', '--capture', '--output-file',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info.run',
'--no-checksum', '--rc', 'branch_coverage=1']' returned non-zero exit
status 1.
ninja: build stopped: subcommand failed.
```

If I add -fprofile-update=atomic as suggested:

```
time CFLAGS="-fprofile-update=atomic"
CXXFLAGS="-fprofile-update=atomic" sh -c 'git clean -dfx && meson
setup --buildtype debug -DPG_TEST_EXTRA="kerberos ldap ssl"
-Db_coverage=true -Dldap=disabled -Dssl=openssl -Dcassert=true
-Dtap_tests=enabled -Dprefix=/home/eax/pginstall build && ninja -C
build && PG_TEST_EXTRA=1 meson test -C build && ninja -C build
coverage-html'
```

... I get:

```
genhtml: ERROR: duplicate merge record src/include/catalog
Traceback (most recent call last):
  File "/usr/bin/meson", line 40, in <module>
    sys.exit(mesonmain.main())
             ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
294, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
282, in run
    return run_script_command(args[1], args[2:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
223, in run_script_command
    return module.run(script_args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 206, in run
    return coverage(options.outputs, options.source_root,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 150, in coverage
    subprocess.check_call([genhtml_exe,
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['genhtml', '--prefix',
'/home/eax/projects/c/postgresql/build', '--prefix',
'/home/eax/projects/c/postgresql', '--output-directory',
'/home/eax/projects/c/postgresql/build/meson-logs/coveragereport',
'--title', 'Code coverage', '--legend', '--show-details',
'--branch-coverage',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info']'
returned non-zero exit status 1.
ninja: build stopped: subcommand failed.
```

I'm using Xubuntu 24.04 LTS and my lcov version is:

```
$ lcov --version
lcov: LCOV version 2.0-1
```

I tried using Autotools with the same results. Pretty confident it
worked before. I'm wondering if anyone else experienced this lately
and/or knows a workaround.

[1]: https://www.postgresql.org/docs/current/regress-coverage.html

--
Best regards,
Aleksander Alekseev



Re: Having problems generating a code coverage report

От
Peter Geoghegan
Дата:
On Wed, Oct 30, 2024 at 5:21 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
> I'm using Xubuntu 24.04 LTS and my lcov version is:
>
> ```
> $ lcov --version
> lcov: LCOV version 2.0-1
> ```

I use Debian unstable for most of my day to day work. Apparently
Debian unstable has exactly the same version of lcov as Ubuntu 24.04.

I've also been unable to generate coverage reports for some time (at
least on Debian, with LCOV version 2.0-1).

--
Peter Geoghegan



Re: Having problems generating a code coverage report

От
Aleksander Alekseev
Дата:
Hi,

> On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote:
> > I use Debian unstable for most of my day to day work. Apparently
> > Debian unstable has exactly the same version of lcov as Ubuntu 24.04.
> >
> > I've also been unable to generate coverage reports for some time (at
> > least on Debian, with LCOV version 2.0-1).
>
> So do I, for both the debian SID and the lcov parts.

I downgraded to lcov 1.16 [1] and it helped. This is merely a
workaround of course, not a long-time solution.

[1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16

-- 
Best regards,
Aleksander Alekseev



Re: Having problems generating a code coverage report

От
jian he
Дата:
On Thu, Oct 31, 2024 at 10:02 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
>
> Hi,
>
> > On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote:
> > > I use Debian unstable for most of my day to day work. Apparently
> > > Debian unstable has exactly the same version of lcov as Ubuntu 24.04.
> > >
> > > I've also been unable to generate coverage reports for some time (at
> > > least on Debian, with LCOV version 2.0-1).
> >
> > So do I, for both the debian SID and the lcov parts.
>
> I downgraded to lcov 1.16 [1] and it helped. This is merely a
> workaround of course, not a long-time solution.
>
> [1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16
>
> --

my ubuntu meson version: 0.61.2, which also fails.
with similar errors you've posted.


from these two posts,
https://github.com/mesonbuild/meson/pull/12345
https://github.com/mesonbuild/meson/issues/11995

Maybe upgrading meson can solve this problem.



Re: Having problems generating a code coverage report

От
Aleksander Alekseev
Дата:
Hi Jian,

> > I downgraded to lcov 1.16 [1] and it helped. This is merely a
> > workaround of course, not a long-time solution.
> >
> > [1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16
> >
> > --
>
> my ubuntu meson version: 0.61.2, which also fails.
> with similar errors you've posted.
>
>
> from these two posts,
> https://github.com/mesonbuild/meson/pull/12345
> https://github.com/mesonbuild/meson/issues/11995
>
> Maybe upgrading meson can solve this problem.

Thanks for the hint. I'm using Meson 1.3.2. Although it's not ancient
(Feb 2024) there is 1.6.0 available. I'll try upgrading and let you
know the results.

-- 
Best regards,
Aleksander Alekseev



Re: Having problems generating a code coverage report

От
Aleksander Alekseev
Дата:
Hi,

> Thanks for the hint. I'm using Meson 1.3.2. Although it's not ancient
> (Feb 2024) there is 1.6.0 available. I'll try upgrading and let you
> know the results.

I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work:

```
genhtml: ERROR: no data for line:864, TLA:UNC,
file:/home/eax/projects/c/postgresql/src/bin/psql/psqlscanslash.l
    (use "genhtml --ignore-errors unmapped ..." to bypass this error)
Traceback (most recent call last):
  File "/home/eax/.venv/bin/meson", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 293, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 281, in run
    return run_script_command(args[1], args[2:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 222, in run_script_command
    return module.run(script_args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/scripts/coverage.py",
line 208, in run
    return coverage(options.outputs, options.source_root,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/scripts/coverage.py",
line 148, in coverage
    subprocess.check_call([genhtml_exe,
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['genhtml', '--prefix',
'/home/eax/projects/c/postgresql/build', '--prefix',
'/home/eax/projects/c/postgresql', '--output-directory',
'/home/eax/projects/c/postgresql/build/meson-logs/coveragereport',
'--title', 'Code coverage', '--legend', '--show-details',
'--branch-coverage',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info']'
returned non-zero exit status 1.
ninja: build stopped: subcommand failed.
```

If I add `--ignore-errors unmapped` as suggested and execute:

```
genhtml --ignore-errors unmapped --prefix
/home/eax/projects/c/postgresql/build --prefix
/home/eax/projects/c/postgresql --output-directory
/home/eax/projects/c/postgresql/build/meson-logs/coveragereport
--title 'Code coverage' --legend --show-details --branch-coverage
/home/eax/projects/c/postgresql/build/meson-logs/coverage.info
```

... I get:

```
...
Processing file src/pl/plpython/plpy_util.c
  lines=33 hit=22 functions=4 hit=4 branches=14 hit=5
Processing file src/include/catalog/index.h
  lines=10 hit=10 functions=2 hit=2
genhtml: ERROR: duplicate merge record src/include/catalog
```

I didn't investigate further.

-- 
Best regards,
Aleksander Alekseev



Re: Having problems generating a code coverage report

От
Aleksander Alekseev
Дата:
Hi everyone,

> I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work:
> [...]
> I didn't investigate further.

Lcov 2.1 and 2.2beta don't work either.

-- 
Best regards,
Aleksander Alekseev



Re: Having problems generating a code coverage report

От
Peter Geoghegan
Дата:
On Wed, Oct 30, 2024 at 5:26 PM Peter Geoghegan <pg@bowt.ie> wrote:
> I use Debian unstable for most of my day to day work. Apparently
> Debian unstable has exactly the same version of lcov as Ubuntu 24.04.
>
> I've also been unable to generate coverage reports for some time (at
> least on Debian, with LCOV version 2.0-1).

I found a temporary workaround. I'm now once again able to produce
html coverage reports on my Debian unstable workstation. Here's what I
did:

I found that a local installation of lcov 1.16 worked. I cloned the
lcov git repo, and checked out the tag 'v1.16'. I then installed this
older lcov version to a directory under my home directory.

It was also necessary to convince the build system to use my
known-good version of lcov. I had to use autoconf here, since (if I'm
not mistaken) only autoconf will accept specific instructions as to
which gcov, lcov, and genhtml to use (via environment variables). I
also used a relatively old version of GCC, taken from the "gcc-11"
package (not sure if that part was really necessary).

I adding something along these lines to the zsh function that runs
"configure" for me, to automate the process going forward:

  export CC="gcc-11"  # Provided by gcc-11 package
  export GCOV="gcov-11" # Provided by gcc-11 package
  export LCOV="/home/pg/.../lcov"  # Provided by local lcov 1.16 installation
  export GENHTML="/home/pg/.../genhtml"  # Provided by local lcov 1.16
installation

From here I ran the standard, documented procedure: I ran configure
(through the zsh function), built Postgres in the usual way, ran the
tests in the usual way, and finally ran "make coverage-html". The
final html report looks very similar to the one from
coverage.postgresql.org.

(There were some geninfo warnings about certain files not having any
coverage whatsoever, but I'm pretty sure that that's normal.)

--
Peter Geoghegan



Re: Having problems generating a code coverage report

От
Tom Lane
Дата:
Peter Geoghegan <pg@bowt.ie> writes:
> On Wed, Oct 30, 2024 at 5:26 PM Peter Geoghegan <pg@bowt.ie> wrote:
>> I've also been unable to generate coverage reports for some time (at
>> least on Debian, with LCOV version 2.0-1).

> I found a temporary workaround. I'm now once again able to produce
> html coverage reports on my Debian unstable workstation.

I got around to poking at this on a shiny new Fedora 41 image,
with gcc-14.2.1 and lcov-2.0-4.  I see errors and warnings aplenty
there too, but after reading the suppression hints that appear in
the error messages I was able to make them all go away:

$ make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors unmapped,unmapped"
Overall coverage rate:
  lines......: 60.7% (282865 of 466310 lines)
  functions......: 67.4% (16114 of 23895 functions)
$

The "-q --legend" part is our default value of GENHTML_FLAGS,
the new magic is the --ignore-errors bit.  As far as I can tell,
specifying "unmapped" once reduces that class of errors to
warnings and specifying it again silences them entirely.

With this, I got plausible-looking html output (I didn't vet it
in detail, but a couple of spot checks looked sane).

I'm curious whether a similar workaround will help with the
Debian toolchain.

            regards, tom lane



Re: Having problems generating a code coverage report

От
Peter Geoghegan
Дата:
On Wed, Jan 15, 2025 at 4:44 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> With this, I got plausible-looking html output (I didn't vet it
> in detail, but a couple of spot checks looked sane).
>
> I'm curious whether a similar workaround will help with the
> Debian toolchain.

Looks like it will.

I was able to get the lcov git master branch's tip to produce a
Postgres coverage report (same compiler version and OS as before).
Unfortunately, I had to use an even grottier incantation to get this
to work:

make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors
unmapped,unmapped,empty,empty,inconsistent,inconsistent,corrupt,corrupt,range,range"
LCOVFLAGS="--ignore-errors
empty,empty,negative,negative,inconsistent,inconsistent"

(As you can imagine, I came up with this through pure trial and error.)

The final report seems fine, though -- at least on a quick look. The
number of times that we actually relied on the suppressions seems to
have been fairly limited, in the end:

Message summary:
  11 ignore messages:
    empty: 1
    inconsistent: 3
    negative: 7
Overall coverage rate:
  source files: 1527
  lines.......: 78.7% (396815 of 504471 lines)
  functions...: 86.3% (23710 of 27474 functions)

Note also: I had to install gcov to /usr/local this time. I couldn't
get away with installing it to my home directory (something about Perl
not being able to see lcovutil.pm in @INC).

--
Peter Geoghegan



Re: Having problems generating a code coverage report

От
Andres Freund
Дата:
Hi,

On 2024-11-01 18:42:37 +0300, Aleksander Alekseev wrote:
> Hi everyone,
> 
> > I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work:
> > [...]
> > I didn't investigate further.
> 
> Lcov 2.1 and 2.2beta don't work either.

I just encountered this. The problem seems to only happen when the build
directory is inside the source directory.

Not that that's a great solution, but just having the build dir be in a
separate directory fixed the issue for me.

Greetings,

Andres Freund



Re: Having problems generating a code coverage report

От
Michael Paquier
Дата:
On Wed, Jan 21, 2026 at 01:57:58PM -0500, Andres Freund wrote:
> I just encountered this. The problem seems to only happen when the build
> directory is inside the source directory.
>
> Not that that's a great solution, but just having the build dir be in a
> separate directory fixed the issue for me.

Even with a VPATH build on Debien GID?  FWIW, I am unable to run a
coverage-html even in this case from the root of a build, finishing
with stuff like that:
/usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i -d . -d
/home/user/git/postgres/ -o lcov_base.info
lcov: ERROR: (empty) no .gcno files found in /home/user/git/postgres
(use "lcov --ignore-errors empty ..." to bypass this error)

Passing "--ignore-errors empty" to LCOVFLAGS bypasses the first step
but genhtml complains, like:
genhtml: ERROR: (corrupt) unable to read trace file 'lcov_base.info':

meson is no better as far as I've tried, so I have pretty much given
up with coverage reports in this environment at this stage.  If there
is a magic recipe, I am not really aware of one..
--
Michael

Вложения

Re: Having problems generating a code coverage report

От
Andres Freund
Дата:
Hi,

On 2026-01-22 09:48:35 +0900, Michael Paquier wrote:
> On Wed, Jan 21, 2026 at 01:57:58PM -0500, Andres Freund wrote:
> > I just encountered this. The problem seems to only happen when the build
> > directory is inside the source directory.
> > 
> > Not that that's a great solution, but just having the build dir be in a
> > separate directory fixed the issue for me.
> 
> Even with a VPATH build on Debien GID?

Well, mine was a meson build.


> meson is no better as far as I've tried, so I have pretty much given
> up with coverage reports in this environment at this stage.  If there
> is a magic recipe, I am not really aware of one..

I have the following .lcovrc in my postgres source dir:

ignore_errors=inconsistent,gcov,range
check_data_consistency=0
stop_on_error=0
genhtml_hierarchical=1
genhtml_show_navigation=1
parallel=16
geninfo_gcov_tool=/usr/bin/gcov-15

(assuming you build with gcc 15)

With that meson coverage information works (i.e. ninja coverage-html) for both
lcov 2.0 and 2.4 as long as the build directory is outside of the source
directory. Both on Debian Sid and macos.

Greetings,

Andres Freund



Re: Having problems generating a code coverage report

От
Michael Paquier
Дата:
On Wed, Jan 21, 2026 at 08:59:45PM -0500, Andres Freund wrote:
> I have the following .lcovrc in my postgres source dir:
>
> ignore_errors=inconsistent,gcov,range
> check_data_consistency=0
> stop_on_error=0
> genhtml_hierarchical=1
> genhtml_show_navigation=1
> parallel=16
> geninfo_gcov_tool=/usr/bin/gcov-15
>
> (assuming you build with gcc 15)
>
> With that meson coverage information works (i.e. ninja coverage-html) for both
> lcov 2.0 and 2.4 as long as the build directory is outside of the source
> directory. Both on Debian Sid and macos.

Thanks for sharing.  Yes, I have a matching gcc version at 15.2
currently.  I have been playing with this area of the configuration
for some time without finding the correct combination before giving
up, and these suggestions are actually helping for meson.  So now I
can get a report offline.  Yeah at least it's one thing working.

Hmm.  configure with and without VPATH still burp at me on HEAD.  If I
tweak the lcov commands in src/Makefile.global.in like in the attached
patch then the non-VPATH build can work, where I remove the srcdir
part (aka reverting c3d9a66024a9).  For the VPATH build, this patch
does not work at all, it generates an incomplete report, ignoring
entirely contrib/, with its base index.html starting inside src/,
missing a bunch of subdirectories.  I can live with this custom patch
for now, just wondering if anybody else sees what I am seeing for a
few months now.

Thoughts?  At least two out of the three methods are working with the
attached instead of one out of three.  Not sure what to do with VPATH,
but well.
--
Michael

Вложения

Re: Having problems generating a code coverage report

От
Andres Freund
Дата:
Hi,

On 2026-01-22 14:35:40 +0900, Michael Paquier wrote:
> On Wed, Jan 21, 2026 at 08:59:45PM -0500, Andres Freund wrote:
> > With that meson coverage information works (i.e. ninja coverage-html) for both
> > lcov 2.0 and 2.4 as long as the build directory is outside of the source
> > directory. Both on Debian Sid and macos.
>
> Thanks for sharing.  Yes, I have a matching gcc version at 15.2
> currently.  I have been playing with this area of the configuration
> for some time without finding the correct combination before giving
> up, and these suggestions are actually helping for meson.  So now I
> can get a report offline.  Yeah at least it's one thing working.

Cool.


> Hmm.  configure with and without VPATH still burp at me on HEAD.  If I
> tweak the lcov commands in src/Makefile.global.in like in the attached
> patch then the non-VPATH build can work, where I remove the srcdir
> part (aka reverting c3d9a66024a9).  For the VPATH build, this patch
> does not work at all, it generates an incomplete report, ignoring
> entirely contrib/, with its base index.html starting inside src/,
> missing a bunch of subdirectories.  I can live with this custom patch
> for now, just wondering if anybody else sees what I am seeing for a
> few months now.

If I suppress the empty errors & warnings with --ignore-errors empty,empty the
current "-d . -d $(srcdir)" kinda works.  Unfortunately, in VPATH builds, the
directory structure is quite messed up. It seems to get a bit better if I add
-p $(srcdir) to the genhtml invocation, but a few generated files still show
up bogus.

The bogus build files unfortunately aren't trivial to fix, if I additionally
add a "-p $(pwd)" (i.e. add the build dir as a prefix), I get errors about
"duplicate merge record".   But it's not too bad to see that stupid path for
the generated files.


Greetings,

Andres Freund



Re: Having problems generating a code coverage report

От
Michael Paquier
Дата:
On Thu, Jan 22, 2026 at 01:19:43PM -0500, Andres Freund wrote:
> On 2026-01-22 14:35:40 +0900, Michael Paquier wrote:
> If I suppress the empty errors & warnings with --ignore-errors empty,empty the
> current "-d . -d $(srcdir)" kinda works.  Unfortunately, in VPATH builds, the
> directory structure is quite messed up. It seems to get a bit better if I add
> -p $(srcdir) to the genhtml invocation, but a few generated files still show
> up bogus.

With a bit more information to skip more error patterns, I actually
got an unpatched HEAD build able to work properly as well, with:
ignore_errors=inconsistent,gcov,range,empty,usage,usage,empty,empty,path

The VPATH build is indeed looking pretty broken here, with the
top-level including the contents of src/ and /home/, /home/ pointing
to contrib/.  I'll live with that, that's so much better than nothing
working locally.  Thanks for all the pointers, Andres.
--
Michael

Вложения

Re: Having problems generating a code coverage report

От
Stefan Kaltenbrunner
Дата:
On 1/29/25 02:00, Michael Paquier wrote:
> On Wed, Jan 15, 2025 at 07:51:39PM -0500, Peter Geoghegan wrote:
>> I was able to get the lcov git master branch's tip to produce a
>> Postgres coverage report (same compiler version and OS as before).
>> Unfortunately, I had to use an even grottier incantation to get this
>> to work:
>>
>> make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors
>> unmapped,unmapped,empty,empty,inconsistent,inconsistent,corrupt,corrupt,range,range"
>> LCOVFLAGS="--ignore-errors
>> empty,empty,negative,negative,inconsistent,inconsistent"
>>
>> (As you can imagine, I came up with this through pure trial and error.)
> 
> Interesting.  These tricks do not seem to work here with a Debian GID
> host, unfortunately.  I'll try to dig a bit more, this is getting very
> annoying :(

hmm this is now also breaking coverage.postgresql.org (after an upgrade 
to the current debian trixie):


make: Leaving directory 
'/home/coverage/pgsrc/pgsql/src/interfaces/ecpg/test'
/usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i -d . -d . 
-o lcov_base.info
lcov: WARNING: (deprecated) RC option 'lcov_branch_coverage' is 
deprecated.  Consider using 'branch_coverage'. instead. 
(Backward-compatible support will be removed in the future.)
         (use "lcov --ignore-errors deprecated,deprecated ..." to 
suppress this warning)
Message summary:
   1 error message:
     usage: 1
   1 warning message:
     deprecated: 1
lcov: ERROR: (usage) duplicate file ./src/fe_utils/astreamer_gzip.gcno 
in both . and .
         (use "lcov --ignore-errors usage ..." to bypass this error)
make: *** [src/Makefile.global:1064: lcov_base.info] Error 1


This seems to be basically the same issue that Andres patch in this 
thread tries to address by removing one of the -d - applying a similar 
hack locally seems to help a bit until we are running into one of the 
other reported issues:

lcov: WARNING: (inconsistent) 
/home/coverage/pgsrc/pgsql/contrib/bloom/blinsert.c:274: unexecuted 
block on non-branch line with non-zero hit count.  Use "geninfo --rc 
geninfo_unexecuted_blocks=1 to set count to zero.
    (use "lcov --ignore-errors inconsistent,inconsistent ..." to suppress 
this warning)
lcov: ERROR: (negative) Unexpected negative hit count '-5' for line 
/home/coverage/pgsrc/pgsql/src/port/snprintf.c:529" while capturing from 
./src/port/snprintf_shlib.gcda.
    (use "lcov --ignore-errors negative ..." to bypass this error)
Message summary:
   1 error message:
     negative: 1
   2 warning messages:
     deprecated: 1
     inconsistent: 1
make: *** [src/Makefile.global:1069: lcov_test.info] Error 1


While I certainly can try to locally hack around that further to get it 
build it feels wrong to post process/patch our own tree to get 
"official" coverage reporting...



Stefan



Re: Having problems generating a code coverage report

От
Michael Paquier
Дата:
On Sat, Feb 14, 2026 at 09:04:44AM +0100, Stefan Kaltenbrunner wrote:
> While I certainly can try to locally hack around that further to get it
> build it feels wrong to post process/patch our own tree to get "official"
> coverage reporting...

Are you using a VPATH build?  That was the only build option that was
causing me problems on Debian GID even with the tweaks proposed by
Andres in a .lcovrc.  Non-vpath under configure and meson worked
unpatched.  With vpath under configure, that worked but the
directories were in a weird state (contrib/ missing from the root
patch, $HOME included).  meson was much slower than the two others.

I have also tried a few more experiments on Fedora 34 lately, that
points to even newer versions of all these tools.  Things are even
more broken moving forward for meson, vpath and non-vpath.  This is
testing things with an unpatched PG tree, just mentioning that it
does not seem to get better as time goes..
--
Michael

Вложения

Re: Having problems generating a code coverage report

От
Álvaro Herrera
Дата:
On 2026-Feb-16, Michael Paquier wrote:

> On Sat, Feb 14, 2026 at 09:04:44AM +0100, Stefan Kaltenbrunner wrote:
> > While I certainly can try to locally hack around that further to get it
> > build it feels wrong to post process/patch our own tree to get "official"
> > coverage reporting...
> 
> Are you using a VPATH build?

Nope, it's a straight in-tree build.  I made it work after a few more
failed tries with a call like

GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,inconsistent,range"
LCOVFLAGS="-q --legend --ignore-errors usage"
make coverage-html

... and we have a nice (??) trace of what errors are being ignored.  I
think this is absolutely bonkers, but who knows when or if lcov and its
tools are going to be fixed in a way that allows our tree to be
processed correctly.

However, it appears that there are problems with the CSS, because the
lines are not colored anymore.  I have no idea how to get this to work,
but you can see the result running lcov 2.0 in
https://coverage.postgresql.org/

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"You're _really_ hosed if the person doing the hiring doesn't understand
relational systems: you end up with a whole raft of programmers, none of
whom has had a Date with the clue stick."              (Andrew Sullivan)
https://postgr.es/m/20050809113420.GD2768@phlogiston.dyndns.org



Re: Having problems generating a code coverage report

От
Álvaro Herrera
Дата:
On 2026-Feb-17, Álvaro Herrera wrote:

> Nope, it's a straight in-tree build.  I made it work after a few more
> failed tries with a call like
> 
> GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,inconsistent,range"
> LCOVFLAGS="-q --legend --ignore-errors usage"
> make coverage-html

Correction.  These variables are not taken into account if given as
environment variables, but they are if given as make arguments.
Moreover, --legend makes lcov die.  So, apparently this one works:

make coverage-html LCOVFLAGS="-q --ignore-errors usage" GENHTML_FLAGS="-q --legend --ignore-errors
unmapped,corrupt,inconsistent,range"

I also removed a file called .lcovrc that only had this line
  lcov_branch_coverage = 0
though that doesn't appear to have automatically added branch coverage
to the report.  However, I'll wait until the next run to see if anything
changes.

The CSS problem appears to have been a caching glitch, and now it works,
so I guess the cached file must have expired.  It looks Christmas-treeish
again now.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"No renuncies a nada. No te aferres a nada."



Re: Having problems generating a code coverage report

От
Michael Paquier
Дата:
On Tue, Feb 17, 2026 at 06:28:24PM +0100, Alvaro Herrera wrote:
> Correction.  These variables are not taken into account if given as
> environment variables, but they are if given as make arguments.
> Moreover, --legend makes lcov die.  So, apparently this one works:
>
> make coverage-html LCOVFLAGS="-q --ignore-errors usage" GENHTML_FLAGS="-q --legend --ignore-errors
unmapped,corrupt,inconsistent,range"
>
> I also removed a file called .lcovrc that only had this line
>   lcov_branch_coverage = 0
> though that doesn't appear to have automatically added branch coverage
> to the report.  However, I'll wait until the next run to see if anything
> changes.

Based on your ideas, I have been able to get my way through today with
this modified artistic flavor, for a non-VPATH build, in a Fedora
environment:
make coverage-html \
    LCOVFLAGS="-q --ignore-errors gcov,gcov" \
    GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,unmapped,inconsistent"
--
Michael

Вложения