Обсуждение: [HACKERS] Running make check-world in buildfarm (was Re: [COMMITTERS] pgsql:Use SASLprep to normalize passwords for SCRAM authentication.)
On 04/08/2017 04:23 AM, Tom Lane wrote: > Heikki Linnakangas <heikki.linnakangas@iki.fi> writes: >> Use SASLprep to normalize passwords for SCRAM authentication. > > The test script that this adds appears to fail unless the environment > selects a UTF8-based locale. On my RHEL6 machine, I see: > > LANG=C make check fail > LANG=en_US.iso88591 make check fail > LANG=en_US.utf8 make check ok Fixed, thanks! > I'm surprised that more of the buildfarm hasn't fallen over. Hmm. It looks like none of the buildfarm members are running the authentication tests. Nor recovery tests, nor subscription tests. We're missing a trick here, at least some of the buildfarm members really ought to run "make check-world", we're missing a lot of coverage otherwise. - Heikki
On Sat, Apr 8, 2017 at 7:33 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote: > Hmm. It looks like none of the buildfarm members are running the > authentication tests. Nor recovery tests, nor subscription tests. We're > missing a trick here, at least some of the buildfarm members really ought to > run "make check-world", we're missing a lot of coverage otherwise. I recall that Andrew has been favoring as much as possible one folder path per test series in the buildfarm client (perhaps to keep the tests separated and have the logs easier to analyze?). I would not mind much if this is replaced by a pure make check-world, which is what most of the serious hackers do, or at least a make check running from src/test to save us a lot of maintenance pain. -- Michael
On 2017-04-08 23:01:06 +0900, Michael Paquier wrote: > On Sat, Apr 8, 2017 at 7:33 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote: > > Hmm. It looks like none of the buildfarm members are running the > > authentication tests. Nor recovery tests, nor subscription tests. We're > > missing a trick here, at least some of the buildfarm members really ought to > > run "make check-world", we're missing a lot of coverage otherwise. > > I recall that Andrew has been favoring as much as possible one folder > path per test series in the buildfarm client (perhaps to keep the > tests separated and have the logs easier to analyze?). I would not > mind much if this is replaced by a pure make check-world, which is > what most of the serious hackers do, or at least a make check running > from src/test to save us a lot of maintenance pain. I think it's partially knowing which target failed, and which regression.diffs to display. If we were able to revamp check-world so it outputs a list of targets the regression machinery were able to run individually, it'd probably help? - Andres
On 04/08/2017 10:11 AM, Andrew Dunstan wrote: > > > -------- Forwarded Message -------- > Subject: Re: [HACKERS] Running make check-world in buildfarm (was Re: > [COMMITTERS] pgsql: Use SASLprep to normalize passwords for SCRAM > authentication.) > Date: Sat, 8 Apr 2017 07:05:54 -0700 > From: Andres Freund <andres@anarazel.de> > To: Michael Paquier <michael.paquier@gmail.com>, Andrew Dunstan > <andrew@dunslane.net> > CC: Heikki Linnakangas <hlinnaka@iki.fi>, Tom Lane > <tgl@sss.pgh.pa.us>, pgsql-hackers <pgsql-hackers@postgresql.org> > > > > On 2017-04-08 23:01:06 +0900, Michael Paquier wrote: > > On Sat, Apr 8, 2017 at 7:33 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote: > > > Hmm. It looks like none of the buildfarm members are running the > > > authentication tests. Nor recovery tests, nor subscription tests. We're > > > missing a trick here, at least some of the buildfarm members really ought to > > > run "make check-world", we're missing a lot of coverage otherwise. > > > > I recall that Andrew has been favoring as much as possible one folder > > path per test series in the buildfarm client (perhaps to keep the > > tests separated and have the logs easier to analyze?). I would not > > mind much if this is replaced by a pure make check-world, which is > > what most of the serious hackers do, or at least a make check running > > from src/test to save us a lot of maintenance pain. > > I think it's partially knowing which target failed, and which > regression.diffs to display. If we were able to revamp check-world so > it outputs a list of targets the regression machinery were able to run > individually, it'd probably help? > Yes, I don't want just to run check-world. I am aware of a few test sets that need to be added, and I'm planning on doing that this weekend, in fact. Specifically: recovery, subscription, authentication and SSL. Peter Eisentraut raised this with me about a week ago. Instead of just adding targets to check-world, perhaps we need to look at what we can do so that the buildfarm client can discover what checks it might run and run them, just as we specify test schedules for pg_regress. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> I think it's partially knowing which target failed, and which
>> regression.diffs to display. If we were able to revamp check-world so
>> it outputs a list of targets the regression machinery were able to run
>> individually, it'd probably help?
> Yes, I don't want just to run check-world.
Yup. The situation with the TAP tests (bin-check step) is already a
usability fail: when there's a failure, your first problem is to root
through megabytes of poorly-differentiated logs just to figure out
what actually failed. Treating all of check-world as a single buildfarm
step would be a disaster.
> Instead of just adding targets to check-world, perhaps we need to look
> at what we can do so that the buildfarm client can discover what checks
> it might run and run them, just as we specify test schedules for pg_regress.
+1. In the meantime, is there any chance of breaking down bin-check into
a separate step per src/bin/ subdirectory?
regards, tom lane
On 04/08/2017 05:26 PM, Andrew Dunstan wrote: > Yes, I don't want just to run check-world. > > I am aware of a few test sets that need to be added, and I'm planning on > doing that this weekend, in fact. Specifically: recovery, subscription, > authentication and SSL. Peter Eisentraut raised this with me about a > week ago. Thanks, that'd be great! Beware that src/test/ssl is not safe to run on a multi-user system, because it allows connections from localhost with well-known user certificates. I've been running it on chipmunk for a while, with the attached module script. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Вложения
On 04/08/2017 12:11 PM, Tom Lane wrote: > Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes: >>> I think it's partially knowing which target failed, and which >>> regression.diffs to display. If we were able to revamp check-world so >>> it outputs a list of targets the regression machinery were able to run >>> individually, it'd probably help? >> Yes, I don't want just to run check-world. > Yup. The situation with the TAP tests (bin-check step) is already a > usability fail: when there's a failure, your first problem is to root > through megabytes of poorly-differentiated logs just to figure out > what actually failed. Treating all of check-world as a single buildfarm > step would be a disaster. > >> Instead of just adding targets to check-world, perhaps we need to look >> at what we can do so that the buildfarm client can discover what checks >> it might run and run them, just as we specify test schedules for pg_regress. > +1. In the meantime, is there any chance of breaking down bin-check into > a separate step per src/bin/ subdirectory? > > Possibly. I will look when I go to do the missing checks, later today or tomorrow. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 04/08/2017 02:49 PM, Andrew Dunstan wrote: > > On 04/08/2017 12:11 PM, Tom Lane wrote: >> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes: >>>> I think it's partially knowing which target failed, and which >>>> regression.diffs to display. If we were able to revamp check-world so >>>> it outputs a list of targets the regression machinery were able to run >>>> individually, it'd probably help? >>> Yes, I don't want just to run check-world. >> Yup. The situation with the TAP tests (bin-check step) is already a >> usability fail: when there's a failure, your first problem is to root >> through megabytes of poorly-differentiated logs just to figure out >> what actually failed. Treating all of check-world as a single buildfarm >> step would be a disaster. >> >>> Instead of just adding targets to check-world, perhaps we need to look >>> at what we can do so that the buildfarm client can discover what checks >>> it might run and run them, just as we specify test schedules for pg_regress. >> +1. In the meantime, is there any chance of breaking down bin-check into >> a separate step per src/bin/ subdirectory? >> >> > Possibly. I will look when I go to do the missing checks, later today or > tomorrow. > OK, crake is running this code now. See <https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2017-04-09%2001%3A58%3A15? I've left off the SSL tests for now. We should look into how we can do that more safely. Meanwhile Heikki is running the tests. Note that some of these tests are quite expensive in terms of time, particularly recover, subscription and pg_rewind. This who want to play along can get the bleeding edge code from git, either by cloning or grabbing a zip of the latest code. I have one or two things I want to do before wrapping up another client release. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services