Обсуждение: Build farm coverage for isolation tests
I'm not sure what the right thing is to do here. Heikki added a new testing methodology under src/test/isolation which allows intermingling a series of statements on multiple connections in desired permutations. (By default each test defined for it runs all permutations of how the statement sequences can be interleaved, but you can specify one or more particular permutations if needed.) I filled it out with the SSI tests which had been run under dtester during initial development. To run it you change to the src/test/isolation directory and run `make check` or `make installcheck`. It takes too long to run this to think about including it in the main `make check`, but I'd feel better about things if this set of tests was normally run on the buildfarm. On my so-so development machine, the installcheck takes about 20 seconds. Setting up and tearing down a temp installation for the `make check` adds another 10 seconds. That seems unlikely to be a deal-breaker for the buildfarm, even if we were to add a little to the test set. Would it make sense to add this to `make check-world` and `make installcheck-world`? Would something else be a good idea to get buildfarm coverage? -Kevin
On 04/19/2011 11:16 AM, Kevin Grittner wrote: > I'm not sure what the right thing is to do here. > > Heikki added a new testing methodology under src/test/isolation > which allows intermingling a series of statements on multiple > connections in desired permutations. (By default each test defined > for it runs all permutations of how the statement sequences can be > interleaved, but you can specify one or more particular permutations > if needed.) I filled it out with the SSI tests which had been run > under dtester during initial development. To run it you change to > the src/test/isolation directory and run `make check` or `make > installcheck`. > > It takes too long to run this to think about including it in the > main `make check`, but I'd feel better about things if this set of > tests was normally run on the buildfarm. On my so-so development > machine, the installcheck takes about 20 seconds. Setting up and > tearing down a temp installation for the `make check` adds another > 10 seconds. That seems unlikely to be a deal-breaker for the > buildfarm, even if we were to add a little to the test set. > > Would it make sense to add this to `make check-world` and `make > installcheck-world`? Would something else be a good idea to get > buildfarm coverage? > > The buildfarm doesn't run either of these. I think the best thing might be to add a new step which runs the isolation check or installcheck. It would only need a small amount of perl code adde3d to the client to accomplish, and nothing on the server side. cheers andrew
Andrew Dunstan <andrew@dunslane.net> wrote: > I think the best thing might be to add a new step which runs the > isolation check or installcheck. It would only need a small amount > of perl code adde3d to the client to accomplish, and nothing on > the server side. Since I'm unskilled at perl and have never looked at the buildfarm scripts, could someone with the appropriate skills and knowledge add this for me? In bash, when I'm in the normal base directory for a build from source, my usual incantation is: pushd src/test/isolation && make check && popd (or use installcheck instead of check if an appropriate cluster is already running) Thanks in advance, -Kevin
Excerpts from Kevin Grittner's message of mar abr 19 12:53:07 -0300 2011: > Since I'm unskilled at perl and have never looked at the buildfarm > scripts, could someone with the appropriate skills and knowledge add > this for me? In bash, when I'm in the normal base directory for a > build from source, my usual incantation is: > > pushd src/test/isolation && make check && popd make -C src/test/isolation check -- Álvaro Herrera <alvherre@commandprompt.com> The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On 04/19/2011 11:53 AM, Kevin Grittner wrote: > Andrew Dunstan<andrew@dunslane.net> wrote: > >> I think the best thing might be to add a new step which runs the >> isolation check or installcheck. It would only need a small amount >> of perl code adde3d to the client to accomplish, and nothing on >> the server side. > > Since I'm unskilled at perl and have never looked at the buildfarm > scripts, could someone with the appropriate skills and knowledge add > this for me? In bash, when I'm in the normal base directory for a > build from source, my usual incantation is: > > pushd src/test/isolation&& make check&& popd > > (or use installcheck instead of check if an appropriate cluster is > already running) See <http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2011-04-20%2010%3A17%3A02&stg=isolation-check> I'll include this in the upcoming release of the buildfarm client. cheers andrew
Andrew Dunstan <andrew@dunslane.net> wrote: > http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2011-04-20%2010%3A17%3A02&stg=isolation-check On a cursory scan, that looks like a successful run of the right test suite. Was there anything in particular I should be checking there? > I'll include this in the upcoming release of the buildfarm client. Thanks! This will only be attempted for 9.1 or higher, right? -Kevin
On 04/20/2011 09:54 AM, Kevin Grittner wrote: > Andrew Dunstan<andrew@dunslane.net> wrote: > > http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2011-04-20%2010%3A17%3A02&stg=isolation-check > > On a cursory scan, that looks like a successful run of the right > test suite. Was there anything in particular I should be checking > there? Not that I know of. I just gave you the URL a) to show it's actually being done and b) to show you what the output would be like. > >> I'll include this in the upcoming release of the buildfarm client. > > Thanks! > > This will only be attempted for 9.1 or higher, right? > > Yes, The code checks for the existence of the isolation directory and only runs the step if it exists. cheers andrew