Peter Eisentraut <peter_e@gmx.net> writes:
> Thomas Swan writes:
>> Have you considered something similar to the Mozilla tinderbox approach
>> where you have a daemon checkout the cvs, compile, run regression tests,
>> and report a status or be able to report a status?
> Even if you could achieve near complete coverage of the platforms,
> platform versions, and auxilliary software versions and combinations that
> PostgreSQL runs with, in most cases, something breaks on a new
> version or combination of these things.
Still, whenever we're doing something that interacts at all with the OS,
it seems we get breakages that don't show in the original author's
testing, but only pop up days to months later when some beta tester
tries the code on platform P or using option Q. The current
difficulties with the IPv6 patches are a fine case in point.
If we could get feedback more easily about whether a proposed patch
compiles and passes regression on a variety of platforms, we could
reduce the pain involved by a great deal, simply because the problems
could be fixed while the code is still fresh in mind.
I don't think there is any company involved with Postgres that is
willing to commit the resources to run a Mozilla-style tinderbox setup
singlehanded. But I wonder whether we couldn't set up something that is
community-based: get a few dozen people with different platforms to
volunteer to check the code regularly on their own machines. I'm
imagining a cron job that fires daily in the wee hours, pulls the latest
CVS tip, does "make distclean; configure; make; make check", and mails
the results to someplace that puts 'em up on our website.
It's possible that we could adapt the tinderbox software to work this
way, but even if we had to write our own, it seems like a fairly simple
task. And it'd give *much* better feedback on porting problems than we
have now. Sure, there will always be corner cases you don't catch,
but the first rule of testing is the sooner you find a bug the cheaper
it is to fix.
regards, tom lane