Обсуждение: Regression tests...

Is there a step-by-step guide somewhere that tells me how to add a new
regression test? I've had a bit of a hunt around, and exactly what to do
to add a test isn't clear.

Chris Bitmead <chrisb@nimrod.itg.telstra.com.au> writes:
> Is there a step-by-step guide somewhere that tells me how to add a new
> regression test? I've had a bit of a hunt around, and exactly what to do
> to add a test isn't clear.

There's not that much to it, assuming you don't need any platform-
specific variations in the expected output.  You make a script under
regress/sql/, add its name in an appropriate place in
sql/run_check.tests, run it, and drop the results file into expected/
(hopefully after manual verification ;-))

BTW, am I right in thinking that sql/tests is now dead code?  If so,
we should flush it, or more likely rename run_check.tests to just tests.
Jan?
        regards, tom lane

> Is there a step-by-step guide somewhere that tells me how to add a new
> regression test? I've had a bit of a hunt around, and exactly what to do
> to add a test isn't clear.

Hmm. Not sure. Here is the procedure, and if you want to plop it into
the Developer's Guide sgml sources that would be great:

1) Generate a file containing the test, and place it into
test/regress/sql/. Should be named appropriately, in a style similar
to other files in that directory.

1a) If the test needs new test data, decide if the tables should stay
for the rest of the regression tests or if they should be removed on
completion of the individual test. If they stay, you will need to
update the results of one or two other tests which look at the current
table list.

2) Add the name of the file to sql/run_check.tests, and perhaps for
completeness to sql/tests (the non-parallel version of the test).

2a) If your test gets data from an external file, you will need to put
the templated source file into input/ rather than sql/, and modify the
Makefiles to generate a runable version for sql/

3) Run the regression tests. Your new test will fail (or succeed,
can't remember which) because the "expected" output file does not
exist.

4) Copy results/<your test>.out to expected/<your test>.out

4a) If your test got data from an external file, you will need to put
the templated output file into output/ rather than expected/, and
modify the Makefiles to generate a non-tempated version for expected/

5) Rerun the regression test, making sure that all tests pass, or that
you understand *all* the differences. No fair if you don't analyze the
differences in some detail.

5a) The canonical regression machine is currently a Linux RH5.2 i686
machine. Some platforms produce different results, and will need
platform-specific versions of the regression test results.

6) Send the patches.
                     - Thomas

-- 
Thomas Lockhart                lockhart@alumni.caltech.edu
South Pasadena, California

> BTW, am I right in thinking that sql/tests is now dead code?  If so,
> we should flush it, or more likely rename run_check.tests to just tests.
> Jan?
   Yepp, dead.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#========================================= wieck@debis.com (Jan Wieck) #