Обсуждение: Review: tests for client programs

On 01/31/2014 02:50 AM, Pavel Stehule wrote:
> 
> 5. I didn't test it on windows

I guess that's my cue. I'll be home later today, and will take a look at
it on my Windows test setup.

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

I posted an updated patch in the original thread.  Please see the commit
fest web site for the URL.


On Thu, 2014-01-30 at 19:50 +0100, Pavel Stehule wrote:

> 6. I found only few issues:
> 
> 
> a) Configure doesn't check a required IRC::Run module

Clearly, we will need to figure out something about how to require this
module, and possibly others in the future, as we expand the tests.
Having configure check for it is not necessarily the best solution --
What is configure supposed to do if it can't find it?

We could perhaps use Test::More skip_all to just skip these tests
depending on what modules are available.  And add appropriate
documentation.
> 
> b) Prepared tests fails when PostgreSQL server was up - should be
> checked and should to raise a valuable error message

The original patch used a hard-coded port number, which was a mistake.
I have changed this now to use a nonstandard port number that is
different from the compiled-in one, similar to how pg_regress used to do
it.  It's still not bullet-proof.  Do we need to do more?
> 
> c) I am not sure if infrastructure is enough - for test pg_dump I miss
> a comparation of produced file with some other expected file. This
> objection is not blocker - someone can enhance it (when will write
> tests of pg_dump for example).

Yes, some more infrastructure will need to be added for pg_dump.  But
that's a separate project.  I don't see anything where the current setup
would be in the way of that.

On Sat, Feb 8, 2014 at 10:16 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> Clearly, we will need to figure out something about how to require this
> module, and possibly others in the future, as we expand the tests.
> Having configure check for it is not necessarily the best solution --
> What is configure supposed to do if it can't find it?
>
> We could perhaps use Test::More skip_all to just skip these tests
> depending on what modules are available.  And add appropriate
> documentation.

I would think we would want to keep the number of dependencies
relatively small.  If it gets large, that just means that nobody will
be able to run the tests.  And -1 for the idea of running only the
tests that we can given what's installed; that'll make it very easy to
not run all the tests, which kind of defeats the purpose of having
them IMHO.  We should just "require" whatever we need and let the test
run abort if it's not available.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Pavel Stehule escribió:
> 2014-02-09 4:16 GMT+01:00 Peter Eisentraut <peter_e@gmx.net>:

> > > a) Configure doesn't check a required IRC::Run module
> >
> > Clearly, we will need to figure out something about how to require this
> > module, and possibly others in the future, as we expand the tests.
> > Having configure check for it is not necessarily the best solution --
> > What is configure supposed to do if it can't find it?
> 
> there can be option "--with-client-tests" and this option should to require
> IRC::Run

A configure option seems a workable idea.

In the future we might want to use the Perl test framework for other
things, so perhaps --enable-perl-testing or something like that.  See
for instance
http://www.postgresql.org/message-id/64aaac31739cef275839932f16fda226@biglumber.com

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2/9/14, 1:01 AM, Pavel Stehule wrote:
>     > b) Prepared tests fails when PostgreSQL server was up - should be
>     > checked and should to raise a valuable error message
> 
>     The original patch used a hard-coded port number, which was a mistake.
>     I have changed this now to use a nonstandard port number that is
>     different from the compiled-in one, similar to how pg_regress used to do
>     it.  It's still not bullet-proof.  Do we need to do more?
> 
> 
> you can check before starting test if some Postgres on this port is
> living or not. We have pg_isready.

I'm having trouble reproducing this scenario.  The tests use a
Unix-domain socket in a private directory, so I don't see how that can
conflict.  Can you show me exactly how you invoked the tests and which
tests and which tests failed?  And what platform are you on?

On 2/17/14, 9:19 AM, Alvaro Herrera wrote:
>> there can be option "--with-client-tests" and this option should to require
>> IRC::Run
> 
> A configure option seems a workable idea.
> 
> In the future we might want to use the Perl test framework for other
> things, so perhaps --enable-perl-testing or something like that.

I don't think I want to add another configure option.  That just reduces
flexibility during development and will make it less likely that people
can or will run the tests.

Skipping tests because of missing dependencies is a standard behavior in
TAP test suites.  The alternative is that we make it a hard requirement.I'd be for that, but maybe next release.