Обсуждение: Help with failures on HEAD

"Todd A. Cook" <ToddA.Cook@synopsys.com> writes:
> Hi,
> Last Saturday morning, builds of HEAD on my animal mantid[1] spontaneously started failing;
> see [2] for an example.  The log shown there is

> Missing checked out branch bf_HEAD:
> * [32mbf_HEAD[m
>   bf_REL_11_STABLE[m
>   bf_REL_12_STABLE[m
>   bf_REL_13_STABLE[m
>   bf_REL_14_STABLE[m
>   bf_REL_15_STABLE[m
>   master[m

In the past, I've been able to resolve issues similar to this one
by flushing the animal's git repo (rm -rf pgmirror.git) and letting
it pull that down fresh on the next run.  Not clear if you included
that when you "removed the buildroot"?

            regards, tom lane

    On 2/9/23, 1:41 PM, "Tom Lane" <tgl@sss.pgh.pa.us <mailto:tgl@sss.pgh.pa.us>> wrote:
    "Todd A. Cook" <ToddA.Cook@synopsys.com <mailto:ToddA.Cook@synopsys.com>> writes:
    > Hi,
    > Last Saturday morning, builds of HEAD on my animal mantid[1] spontaneously started failing;
    > see [2] for an example. The log shown there is

    > Missing checked out branch bf_HEAD:
    > * [32mbf_HEAD[m
    > bf_REL_11_STABLE[m
    > bf_REL_12_STABLE[m
    > bf_REL_13_STABLE[m
    > bf_REL_14_STABLE[m
    > bf_REL_15_STABLE[m
    > master[m

    In the past, I've been able to resolve issues similar to this one
    by flushing the animal's git repo (rm -rf pgmirror.git) and letting
    it pull that down fresh on the next run. Not clear if you included
    that when you "removed the buildroot"?

Yes, I did "rm -rf buildroot".  After that, I verified (with "diff -r") that I had no changes
vs. the distribution tarball other than having my build-farm.conf present.

-- todd

    On 2/9/23, 1:41 PM, "Tom Lane" <tgl@sss.pgh.pa.us <mailto:tgl@sss.pgh.pa.us>> wrote:    "Todd A. Cook" <ToddA.Cook@synopsys.com <mailto:ToddA.Cook@synopsys.com>> writes:    > Hi,    > Last Saturday morning, builds of HEAD on my animal mantid[1] spontaneously started failing;    > see [2] for an example. The log shown there is
    > Missing checked out branch bf_HEAD:    > * [32mbf_HEAD[m    > bf_REL_11_STABLE[m    > bf_REL_12_STABLE[m    > bf_REL_13_STABLE[m    > bf_REL_14_STABLE[m    > bf_REL_15_STABLE[m    > master[m
    In the past, I've been able to resolve issues similar to this one    by flushing the animal's git repo (rm -rf pgmirror.git) and letting    it pull that down fresh on the next run. Not clear if you included    that when you "removed the buildroot"?

Yes, I did "rm -rf buildroot".  After that, I verified (with "diff -r") that I had no changes
vs. the distribution tarball other than having my build-farm.conf present.

Don't do that. All the state is stored under the buildroot, and you'd be throwing that away. 999 times out of 1000 this should do the trick:

rm -rf buildroot/[RH]*/pgsql

The other 1 time out of 1000 you could also remove the mirror like Tom suggests as well.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

"Todd A. Cook" <ToddA.Cook@synopsys.com> writes:
> Thanks, I’ll keep that in mind if this ever happens again.  However, removing those things didn’t fix the problem.  I
suspectthere 
> might be something wrong with the Perl setup on that animal, even though it hasn’t changed recently.  Unfortunately,
myPerl 
> skills are non-existent, and I have no clue how to proceed.

Hmph.  Wondering if something broke recently in the buildfarm code,
I tried rm -rf'ing the whole buildroot locally ... but run_branches.pl
got through fine.

A couple of ideas:

* Run through the setup process documented at

https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto

particularly "Check that required perl modules are present" ...
maybe something went missing?

* Diff your build-farm.conf against the current sample file
and make sure there are no unintentional discrepancies.

            regards, tom lane

    On 2/9/23, 1:41 PM, "Tom Lane" <tgl@sss.pgh.pa.us <mailto:tgl@sss.pgh.pa.us>> wrote:    "Todd A. Cook" <ToddA.Cook@synopsys.com <mailto:ToddA.Cook@synopsys.com>> writes:    > Hi,    > Last Saturday morning, builds of HEAD on my animal mantid[1] spontaneously started failing;    > see [2] for an example. The log shown there is
    > Missing checked out branch bf_HEAD:    > * [32mbf_HEAD[m    > bf_REL_11_STABLE[m    > bf_REL_12_STABLE[m    > bf_REL_13_STABLE[m    > bf_REL_14_STABLE[m    > bf_REL_15_STABLE[m    > master[m
    In the past, I've been able to resolve issues similar to this one    by flushing the animal's git repo (rm -rf pgmirror.git) and letting    it pull that down fresh on the next run. Not clear if you included    that when you "removed the buildroot"?

Yes, I did "rm -rf buildroot".  After that, I verified (with "diff -r") that I had no changes
vs. the distribution tarball other than having my build-farm.conf present.

It looks like you're getting some color output, probably because of some unwise git configuration or something that makes git think it's talking to a terminal. Please see if this patch works to fix things:

diff --git a/PGBuild/SCM.pm b/PGBuild/SCM.pm
index dcfd180..6dc1232 100644
--- a/PGBuild/SCM.pm
+++ b/PGBuild/SCM.pm
@@ -983,7 +983,7 @@ sub _setup_new_workdir
    # doesn't yet know about
    my @fetchlog = run_log('git fetch --prune');
 
-   my @branches = `git branch`;
+   my @branches = `git branch --no-color`;
    chomp @branches;
    my @colog;
    if (grep { /\bbf_$branch\b/ } @branches)

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com