This presumably is due to using mingw's python rather than python.org's python. Seems like a reasonable thing to support for the mingw build.
Melih, you could try to build against the python.org python (i installed in the CI container).
I tried to use the installed python from python.org in the container.
The problem with this is that "LIBDIR" and "LDLIBRARY" configs of python for windows from python.org are empty. Therefore python_libdir or other related variables in configure file are not set correctly.
Seems we're doing something wrong and end up with a 4 byte off_t, whereas python ends up with an 8byte one. We probably need to fix that. But it's not the cause of this problem.
You could take out -s from the make flags and see whether plpython3.dll is being built and installed, and where to.
I couldn't get what's wrong from these logs to be honest. But I see that plpython3.dll exists under src/pl/plpython after build when I run these steps locally.