I would like to save such records, and only such records, in an auxiliary table.
You will probably need to resort to loading the incoming data into an unlogged (and consider temporary) staging table then executing one query to load the known good data into the main table and a second query to load the known bad data into the auxiliary table. For me this is the most direct solution generally anyway and should be used absent a performance problem necessitating optimization work.
Upsert isn't going to help you load incoming data into an auxiliary table, it can only adjust the data to avoid the conflict relative to the referenced table.