Обсуждение: BUG #15605: Unstable regression test "tablespace"

The following bug has been logged on the website:

Bug reference:      15605
Logged by:          Alexander Lakhin
Email address:      exclusion@gmail.com
PostgreSQL version: 11.1
Operating system:   Ubuntu 18.04
Description:

When running 'TESTS="$(printf "tablespace %.0s" `seq 1000`)" make
check-tests' I get:
============== running regression test queries        ==============
test tablespace                   ... ok
...
test tablespace                   ... ok
test tablespace                   ... FAILED
test tablespace                   ... FAILED
...
test tablespace                   ... FAILED
test tablespace                   ... ok
test tablespace                   ... FAILED
...
test tablespace                   ... FAILED
test tablespace                   ... ok
...
and so on...

test tablespace                   ... FAILED
============== shutting down postmaster               ==============

===========================
 403 of 1000 tests failed. 
===========================

The differences that caused some tests to fail can be viewed in the
file ".../src/test/regress/regression.diffs".  A copy of the test summary
that you see
above is saved in the file ".../src/test/regress/regression.out".

src/test/regress/regression.diffs contains:

--- .../src/test/regress/expected/tablespace.out    2019-01-23
11:40:25.438743265 +0300
+++ .../src/test/regress/results/tablespace.out     2019-01-23
11:41:00.559072671 +0300
@@ -89,9 +89,9 @@
     where c.reltablespace = t.oid AND c.relname LIKE 'part%_idx';
    relname   |     spcname
 -------------+------------------
- part1_a_idx | regress_tblspace
  part2_a_idx | regress_tblspace
  part_a_idx  | regress_tblspace
+ part1_a_idx | regress_tblspace
 (3 rows)

 -- check that default_tablespace doesn't affect ALTER TABLE index
rebuilds

======================================================================

It seems that the following query in src/test/regress/sql/tablespace.sql:
SELECT relname, spcname FROM pg_catalog.pg_tablespace t, pg_catalog.pg_class
c
    where c.reltablespace = t.oid AND c.relname LIKE 'part%_idx';
needs an "ORDER BY".

=?utf-8?q?PG_Bug_reporting_form?= <noreply@postgresql.org> writes:
> When running 'TESTS="$(printf "tablespace %.0s" `seq 1000`)" make
> check-tests' I get:
> ...
> It seems that the following query in src/test/regress/sql/tablespace.sql:
> SELECT relname, spcname FROM pg_catalog.pg_tablespace t, pg_catalog.pg_class c
>     where c.reltablespace = t.oid AND c.relname LIKE 'part%_idx';
> needs an "ORDER BY".

I'm totally unexcited by this.  Almost every script in our standard
regression suite is somewhat context-dependent, eg many of them depend
on tables or other objects created by previous scripts.  Trying to make
them be robust when run standalone would be a huge and ultimately pretty
pointless exercise.

In the case of the tablespace script, it evidently has some dependency on
being run in a fresh database, but so what?  It always will be, when used
as directed.  (I do not recall the reason why we run it first, but
I remember that there was some good reason, possibly to do with crash
recovery.)

            regards, tom lane

23.01.2019 17:53, Tom Lane пишет:
> =?utf-8?q?PG_Bug_reporting_form?= <noreply@postgresql.org> writes:
>> When running 'TESTS="$(printf "tablespace %.0s" `seq 1000`)" make
>> check-tests' I get:
>> ...
>> It seems that the following query in src/test/regress/sql/tablespace.sql:
>> SELECT relname, spcname FROM pg_catalog.pg_tablespace t, pg_catalog.pg_class c
>>     where c.reltablespace = t.oid AND c.relname LIKE 'part%_idx';
>> needs an "ORDER BY".
> I'm totally unexcited by this.  Almost every script in our standard
> regression suite is somewhat context-dependent, eg many of them depend
> on tables or other objects created by previous scripts.  Trying to make
> them be robust when run standalone would be a huge and ultimately pretty
> pointless exercise.
>
> In the case of the tablespace script, it evidently has some dependency on
> being run in a fresh database, but so what?  It always will be, when used
> as directed.  (I do not recall the reason why we run it first, but
> I remember that there was some good reason, possibly to do with crash
> recovery.)
Yes, I understand that there are some dependencies between test scripts
and most of the scripts are not repeatable, but the tablespace test is.
I can run:
TESTS="$(printf "tablespace %.0s" `seq 10`)" make check-tests
and the test executes fine:
============== running regression test queries        ==============
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
test tablespace                   ... ok
============== shutting down postmaster               ==============
============== removing temporary instance            ==============

======================
 All 10 tests passed.
======================

So the issue is not with repetition itself but with the random change of
the SELECT results.


Best regards,
Alexander