Re: obsoleting plpython2u and defaulting plpythonu to plpython3u

Pavel Raiskup <praiskup@redhat.com> writes:
> On Wednesday, April 25, 2018 12:17:05 AM CEST Peter Eisentraut wrote:
>> On 4/24/18 07:13, Pavel Raiskup wrote:
>>> What's the expected future migration path from plpython2 to plpython3 in
>>> such cases?  I'm thinking about rewrite of the docs and creating some
>>> scripting which could simplify the migration steps.  Would such patches be
>>> welcome at this point?

>> I'm not sure what you have in mind.  In many cases, you can just change
>> the LANGUAGE clause.  I suppose you could run 2to3 over the function
>> body?  Write a PL/Python function to run 2to3?

> Something along those lines, yes.  Well, I don't have a real plan.  I'd
> like to discuss what's the expected way out, how to give a chance to users
> to migrate from plpython2 to plpython3 _now_ (since both are yet
> available).
> I _guess_ the users don't care much about 2 vs. 3 in plpython - so they
> usually simply live with implicit plpython version - which would OTOH mean
> that the majority will have to face the migration sooner or later.

It's still not very clear to me whether we can or should change anything
about what "plpythonu" means.  However, it seems like we definitely ought
to be encouraging PL/Python users to switch their code to Python 3.

As an exercise, I wrote a little script that runs through all the Python 2
functions in a database and converts them to Python 3, insofar as "2to3"
can do that.  This is by a noticeable margin the largest Python script
I've ever written, so I'm sure it sucks stylistically and perhaps
functionally too, but it will serve to get some discussion started.
Should we provide something like this to users, and if so how exactly
(installed file and/or documentation)?  Would it be sane to squeeze
into v12 at this late date?  I suspect that Fedora might want to have it
in their v12 distro even if we only put it into HEAD.

            regards, tom lane

do $$
import re, subprocess, tempfile

# pattern to extract just the function header from pg_get_functiondef result
aspat = re.compile("^(.*?\nAS )", re.DOTALL)
# pattern for replacing LANGUAGE portion
langpat = re.compile("\n LANGUAGE plpython2?u\n")

# collect info about functions to update
rv = plpy.execute("""
select pg_catalog.pg_get_functiondef(p.oid) as fd, prosrc as body
from pg_catalog.pg_proc p join pg_catalog.pg_language l on p.prolang = l.oid
where lanname in ('plpythonu', 'plpython2u')
""")

# For some benighted reason, lib2to3 has exactly no documented API,
# so we must use the command-line API "2to3" instead.  Adjust path
# and optional arguments for it here.
cmd2to3 = "2to3 --no-diffs"

# Make a temp directory to hold the file for it to work on.
with tempfile.TemporaryDirectory() as tmpdirname:

    # process each function
    for r in rv:
        # extract everything but the body from pg_get_functiondef result
        m = aspat.match(r["fd"])
        if not m:
            raise ValueError('unexpected match failure')
        fheader = m.group(1)

        # replace the language clause
        fheader = langpat.sub("\n LANGUAGE plpython3u\n", fheader, 1)

        # put body in a temp file so we can apply 2to3
        f = open(tmpdirname + "/temp.py", mode = 'w')
        f.write(r["body"])
        f.close()

        # apply 2to3 to body
        subprocess.check_call(cmd2to3 + " -w " + tmpdirname + "/temp.py", shell=True)
        f = open(tmpdirname + "/temp.py", mode = 'r')
        fbody = f.read()
        f.close()

        # construct and execute SQL command to replace the function
        newstmt = fheader + plpy.quote_literal(fbody)
        # uncomment this for debugging purposes:
        # plpy.info(newstmt)
        plpy.execute(newstmt)

        # commit after each successful replacement, in case a later one fails
        plpy.commit()
$$
language plpython3u;