Re: RFC: adding pytest as a supported test framework

On Thu, Jun 13, 2024 at 4:06 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> There was a four-step plan sketch at the end of that email, titled "A
> Plan". That was not intended to be "the final detailed plan", because
> I was soliciting feedback on the exact pieces people wanted to try to
> implement first, and I am not the God Emperor of Pytest. But it was
> definitely A Plan.

Well, OK, now I feel a bit dumb. I guess I missed that or forgot about it.

> > - at least as much in-tree support for writing tests as we have today
> > with PostgreSQL::Test::whatever, but not necessarily a 1:1 reinvention
> > of the stuff we have now, and documentation of those facilities that
> > is as good or, ideally, better than what we have today.
>
> I think this is way too much expectation for a v1 patch. If you were
> committing this by yourself, would you agree to develop the entirety
> of PostgreSQL::Test in a single commit, without the benefit of the
> buildfarm checking you as you went, and other people trying to write
> tests with it?

Eh... I'm confused. PostgreSQL::Test::Cluster is more than half of the
code in that directory, and without it you wouldn't be able to write
most of the TAP tests that we have today. You would really want to
call this project done without having an equivalent?

> > The important thing to me here (as it so often is) is to think like a
> > maintainer. Imagine that immediately after the patches for this
> > feature are committed, the developers who did the work all disappear
> > from the community and are never heard from again. How much pain does
> > that end us causing? The answer doesn't need to be zero; that is
> > unrealistic. But it also shouldn't be "well, if that happens we're
> > going to have to rip the feature out"
>
> Can you elaborate on why that's not an okay outcome?

Well, you just argued that it should be an okay outcome, and I do sort
of see your point, but I refer you to my earlier reply about the
difficulty of getting anything reverted in the culture as it stands.

> > or "well, a bunch of committers
> > who didn't want to write tests in Python in the first place are now
> > going to have to do a lot of work in Python to stabilize the work
> > already committed."
>
> Is it that? If the problem is that, we should address that. Because if
> that is truly the fear, I cannot assuage that fear without showing you
> something, and I cannot show you something you do not want to see, if
> you don't want to write tests in Python in the first place.

I have zero desire to write tests in Python. If I could convince
everyone here to spend their time and energy improving the stuff we
have in Perl instead of introducing a whole new test framework, I
would 100% do that. But I'm pretty sure that I can't, and I think the
project needs to pick from among realistic options rather than
theoretical ones. Said differently, it's not all about me.

--
Robert Haas
EDB: http://www.enterprisedb.com