On Tue, Dec 12, 2017 at 2:55 AM, Jeremy Finzel <finzelj@gmail.com> wrote: > I understand how to setup a regression suite for a postgres extension, but > what I'm not clear on from the docs is if there is a pattern that exists for > testing not only the latest version of an extension, but also an upgraded > previous version. > > Is there any straightforward way to do this that doesn't involve manually > copying tests?
It is perfectly possible to do tests on a specific version and test upgrade paths using CREATE EXTENSION and ALTER EXTENSION which support versioning in a SQL regression script, say: CREATE EXTENSION foo VERSION "0.1"; -- Do stuff ALTER EXTENSION foo UPDATE TO "0.2"; -- Do other stuff
This works well when you want to test 1.1 binaries with a 1.0 SQL extension definition.
It's not effective if you need to test 1.0 binaries+extension, then an upgrade to 1.1 binaries and extension. You have no way to install and run the 1.0 binaries without going outside pg_regress / TAP and using an external test harness/framework/script. I've been bitten by this multiple times before. For example, if 1.1 always assigns a value to some column that was nullable in 1.0, you're never going to exercise 1.1's handling of nulls in that column.
It also doesn't lend its self to complex multi-path upgrades, where for example you could upgrade 1.0.0 -> 1.0.1 -> 1.1.0 or upgrade directly from 1.0.0 -> 1.1.0. This rapidly becomes an issue when you release 1.0.0, release 1.1.0, then release a maintenance 1.0.1 version. Now you have toadd the 1.0.1 -> 1.1.0 upgrade path, but you still have the 1.0.0 -> 1.1.0 path.
You can handle that with TAP if you're willing to write something to do the various combinations of steps and exercise them. It's not practical with pg_regress.
More thorough testing benefits from an external harness.