Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD
От | Fabien COELHO |
---|---|
Тема | Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD |
Дата | |
Msg-id | alpine.DEB.2.21.1901180920450.26418@lancre обсуждение исходный текст |
Ответ на | Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD (Tom Lane <tgl@sss.pgh.pa.us>) |
Список | pgsql-hackers |
>> I'd rather keep it by simply adding the "|unknown" alternative. 30 years >> of programming have taught me that testing limit & error cases is useful, >> although you never know when it will be proven so. > > Sorry, I don't buy this line of argument. > Reasonable test design requires making cost/benefit tradeoffs: the cost > to run the test over and over, and the cost to maintain the test itself > (e.g. fix portability issues in it) have to be balanced against the > probability of it finding something useful. I judge that the chance of > this particular test finding something is small, and I've had quite > enough of the maintenance costs. > > Just to point up that we're still not clearly done with the maintenance > costs of supposing that we know how every version of getopt_long will > spell this error message, I note that my Linux box seems to have two > variants of it: > > $ pgbench -z > pgbench: invalid option -- 'z' > Try "pgbench --help" for more information. > $ pgbench --z > pgbench: unrecognized option '--z' > Try "pgbench --help" for more information. > > of which the "invalid" alternative is also not in our list right now. > Who's to say how many more versions of getopt_long are out there, > or what the maintainers thereof might do in the future? ISTM that the getopt implementers imagination should run out in the end:-) invalid, unknown, unrecognized, unexpected, incorrect... Ok English has many words:-) >> I agree that some tests can be useless, but I do not think that it applies >> to this one. This test also checks that under a bad option pgbench stops >> with an appropriate 1 exit status. > > It's possible that it's worth the trouble to check for exit status 1, > but I entirely fail to see the point of checking exactly what is the > spelling of a message that is issued by code not under our control. > > Looking closer at the test case: > > [ > 'bad option', > '-h home -p 5432 -U calvin -d --bad-option', > [ qr{(unrecognized|illegal) option}, qr{--help.*more information} ] > ], > > ISTM that just removing the first qr{} pattern, and checking only that > we get the text that *is* under our control, is a reasonable compromise > here. Possibly. I'd be a little happier if it checks for a non-empty error message, eg qr{...} or qr{option} (the message should say something about the option). -- Fabien.
В списке pgsql-hackers по дате отправления: