Does it fail to clean up the backup folder in all cases where the backup failed, or just in this case?
The cleanup is done in the cases I have seen so far with base pg_basebackup functionality (not including the parallel backup feature) with the message "pg_basebackup: removing contents of data directory"
A similar case was also fixed for parallel backup reported by Rajkumar where the contents of the backup folder were not cleaned up after the error.
What I'm saying is that it's unclear whether there's a bug here or whether it just failed because of the very extreme test scenario you created. Spawning >1000 processes on a small machine can easily make a lot of things fail.