Where are you measuring the connections? From your app to PGBouncer, or from PGBouncer to PostgreSQL?
If it is from your app to PGBouncer, that sounds strange, and like the app is not properly releasing connections as it should. If it is from PGBouncer to PostgreSQL, that sounds normal. I haven't used PGBouncer, but i've used other connection poolers in the past. They would start out with a set number of minimum connections, and scale up to the max number specified as needed. The pooler wouldn't release a connection to the DB once it has made one, and just kept it available in the pool.