Deleting conflicting rows when creating a foreign key

I am doing an ALTER TABLE to create a foreign key, however with some
rows i get:

insert or update on table "name" violates foreign key constraint "name_fkey"

How can I just drop the conflicting rows while doing that?