Обсуждение: running make check with only specified tests

I've often wanted to be able to run "make check" and just have it run
the small number of tests I am interested in. Here's a tiny patch along
those lines. It creates a new targe which I have called "check-with" for
want of a better name. And with it I can do:

    $ make check-with TESTS="json jsonb"


and have it do the temp install etc and then run just those two tests.

Thoughts?

cheers

andrew

On Jan26, 2014, at 17:47 , Andrew Dunstan <andrew@dunslane.net> wrote:
> I've often wanted to be able to run "make check" and just have it run the small number of tests I am interested in.
Here'sa tiny patch along those lines. It creates a new targe which I have called "check-with" for want of a better
name.And with it I can do: 
>
>   $ make check-with TESTS="json jsonb"
>
> and have it do the temp install etc and then run just those two tests.

+1 for the feature (+Inf, actually), but will this work if the tests
depend on stuff created by other tests?

best regards,
Florian Pflug

On 01/26/2014 12:01 PM, Florian Pflug wrote:
> On Jan26, 2014, at 17:47 , Andrew Dunstan <andrew@dunslane.net> wrote:
>> I've often wanted to be able to run "make check" and just have it run the small number of tests I am interested in.
Here'sa tiny patch along those lines. It creates a new targe which I have called "check-with" for want of a better
name.And with it I can do:
 
>>
>>    $ make check-with TESTS="json jsonb"
>>
>> and have it do the temp install etc and then run just those two tests.
> +1 for the feature (+Inf, actually), but will this work if the tests
> depend on stuff created by other tests?
>


No, if they do it will be up to you to include those in your test list 
in the right order.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> I've often wanted to be able to run "make check" and just have it run 
> the small number of tests I am interested in. Here's a tiny patch along 
> those lines. It creates a new targe which I have called "check-with" for 
> want of a better name. And with it I can do:
>     $ make check-with TESTS="json jsonb"

The vast majority of the regression tests have interdependencies, which
would make any feature along these lines fairly useless IME.  (And no,
I'm not interested in converting the tests to a no-dependencies style.)

Also, the tests themselves don't take that long, especially in parallel
mode.  If you need to speed up repeated testing, it's more profitable to
avoid the install/initdb overhead of a "make check".  I use a small
script that just reinstalls the postgres executable and does "make
installcheck-parallel" when I'm doing iterative development.
        regards, tom lane

On 01/26/2014 12:08 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> I've often wanted to be able to run "make check" and just have it run
>> the small number of tests I am interested in. Here's a tiny patch along
>> those lines. It creates a new targe which I have called "check-with" for
>> want of a better name. And with it I can do:
>>      $ make check-with TESTS="json jsonb"
> The vast majority of the regression tests have interdependencies, which
> would make any feature along these lines fairly useless IME.  (And no,
> I'm not interested in converting the tests to a no-dependencies style.)
>
> Also, the tests themselves don't take that long, especially in parallel
> mode.  If you need to speed up repeated testing, it's more profitable to
> avoid the install/initdb overhead of a "make check".  I use a small
> script that just reinstalls the postgres executable and does "make
> installcheck-parallel" when I'm doing iterative development.
>
>             


I have something similar, and prodded by your email I've just improved 
it a bit ;-) But it doesn't work so well if you're changing the catalog, 
as you need an initdb anyway. And there are are some cases (the one I 
happen to be working on being one of them) where the tests have no prior 
dependencies.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> On 01/26/2014 12:08 PM, Tom Lane wrote:
>> Also, the tests themselves don't take that long, especially in parallel
>> mode.  If you need to speed up repeated testing, it's more profitable to
>> avoid the install/initdb overhead of a "make check".  I use a small
>> script that just reinstalls the postgres executable and does "make
>> installcheck-parallel" when I'm doing iterative development.

> I have something similar, and prodded by your email I've just improved 
> it a bit ;-) But it doesn't work so well if you're changing the catalog, 
> as you need an initdb anyway.

True.  OTOH, when you're changing the catalogs it seems pretty foolish
to not run the whole test suite.

Anyway, I have no great objection to the proposed patch, I'm just dubious
that it's really worth the trouble.  If you do go through with it, I'd
suggest adding an installcheck-with variant.

In the bikeshedding department, maybe "-tests" instead of "-with"?
        regards, tom lane

Tom Lane wrote:

> Anyway, I have no great objection to the proposed patch, I'm just dubious
> that it's really worth the trouble.  If you do go through with it, I'd
> suggest adding an installcheck-with variant.
> 
> In the bikeshedding department, maybe "-tests" instead of "-with"?

No objection to the proposed idea either, but I wanted to point out that
I had the idea sometime ago that each test would declare which other
test it depended on; so if you specify one to run in isolation, the ones
it depended on got run beforehand.  I never got around to implementing
it because running the whole bunch doesn't take that long anyway, but if
for some reason you want to run a specific test over and over, it is
useful.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services