PANIC: wrong buffer passed to visibilitymap_clear

Buildfarm members spurfowl[1] and thorntail[2] have each shown $SUBJECT
once in the past two days.  The circumstances are not quite the same;
spurfowl's failure is in autovacuum while thorntail's is in a manual
VACUUM command.  Still, it seems clear that there's a recently-introduced
bug here somewhere.  I don't see any obvious candidate for the culprit,
though.  Any ideas?

            regards, tom lane

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spurfowl&dt=2021-04-08%2010%3A22%3A08
[2] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thorntail&dt=2021-04-09%2021%3A28%3A10