Would it work to use ALTER SYSTEM (postgresql.auto.conf)? Or is that copied by pg_rewind also?
Yes that is copied currently by pg_rewind, as are server logs if they are in the data directory. I think replication slots also might be copied but would have to check.
Even if so, using the ALTER SYSTEM command after pg_rewind might be an easy way of correcting the port back to what you want without doing more complicated processing or re-writing the entire postgresql.conf.
Currently pg_rewind copies all files including postgresql.conf. It would be nice if pg_rewind has an option to not copy postgresql.conf. I sometimes create multiple PostgreSQL database clusters with different port number which postmaster is listening on for a testing purpose. So existing postgresql.conf on the target cluster being overwritten by pg_rewind is annoying. I believe there are some use cases where different port numbers are used among PostgreSQL database clusters in the real world.