Re: mb and ecpg regression tests

Joachim Wieland wrote:
> On Thu, May 17, 2007 at 10:14:45PM +0200, Magnus Hagander wrote:
>> IIRC, Joachim had started working on the ecpg part. (Which is one of the
>> reasons I haven't looked at it myself yet)
> 
>> Joachim, did you ever get anywhere with that?
> 
> I have ecpg tests somehow running under msvc but there are still some issues.
> Here are the problems that I encounter(ed):
> 
> a) ecpg uses a program for each test that runs and generates output. Those
> programs need to be compiled: in MSVC we would either compile them on the
> fly while testing or generate a new project for every single executable.
> Since the second option is overkill, Magnus suggested to try compiling them
> on the fly with the command line compiler. This is ugly but seems to work.

It might also be possible to generate a wildcard msbuild file for them,
perhaps? (If you have Visual Studio 2005, you will have msbuild) Worth
investigating, but this is probably the easiest thing to fix.


> b) ecpg tests use several directories. This causes several problems, one of
> them being the resultmap format to be changed, it currently uses:
> 
> float4/i.86-pc-mingw32=...
> 
> we might want to change the / to some other character here.

Hmm. I guess doable if necessary.


> c) ecpg uses 3 different files per test. stderr, stdout and generated
> source. There is not only just one file to diff against an expected file.
> 
> My approach was to have separate init and test routines for each "module"
> (backend regression tests, ecpg or whatever) and share the rest.
> The init routine sets up stuff and the test routine knows about the specific
> test logic. It generates lists of files to diff. The diffing and reporting
> is kept common of course. A problem here are alternative comparison files.
> We have the resultmap entries and we have "test_n.out".  This has to be
> combined with ecpg having 3 files. We might want to use
> 
> test-platform.stdout instead of test.stdout and
> test_1.stderr instead of test.stderr
> 
> I have not yet sorted out this last item. Proposals welcome. I hope that I
> can fix it on the weekend or at least by the end of next week and then send
> my version to whoever is interested.

Hmm. Probably easier for someone who already knows how the ecpg tests
actually work. I just keep breaking them ;-)

However - do go ahead and send what you have to -patches for comments as
a WIP patch, even if it's not ready. I'd certainly be interested at
looking at what you have so far.

//Magnus