Hi,
Any kind of slowness on your archive directory may cause the archiving process fall behind thus accumulating segment files in your cluster's pg_xlog directory.
I assume that you are on PostgreSQL 9.4. Could you please check your archiver status by typing "select * from pg_catalog.pg_stat_archiver;" in psql ? If the last_archived_wal column's value is not so close to your current xlog location, then it probably means a slow write speed on your archive path compared to your pg_xlog path.
You can check your current xlog file by typing "select pg_xlogfile_name(pg_current_xlog_location());" in psql.
If you are not on PostgreSQL 9.4, you can check your archiver status by typing "ps -ef | grep archiver" in your shell.
I've also assumed that you are not using replication slots.
As far as I know, long running transactions (just sitting in idle) won't affect pg_xlog directory size. Correct me if I'm wrong.
Hope that helps.
Have a nice day.
Koray