So the bug is that the pg_service.conf file is not read from this location.
I can reproduce this after using the EDB installer to install.
I create the directory indicated by `pg_config --sysconfdir`, put the pg_service.conf in it, and psql and pg_dump work without setting either PGSERVICEFILE or PGSYSCONFDIR. But pgbench and PgAdmin4 do not work with the service file this way, but do work if I define PGSERVICEFILE, so there does seem to be something buggy going on.
This was on Windows 2019, Windows_Server-2019-English-Full-Base-2020.07.15 (ami-066a1a3fa81bfbd00). With both 9.6 and 12 (the only ones I tested), and I installed both to the default location and a custom location (on a fresh machine each time), makes no difference.