Postgres memory usage per connection

I am using Postgres 9.3.10 on Linux with the following memory related configuration:

effective_cache_size = 128MB

shared_buffers = 8MB

work_mem = 1MB

maintenance_work_mem = 16MB

Work memory is configured to 1 MB but some of the backend processes (corresponding to client connections) are using 50-90MB of PSS memory.

How do I find out where rest of the memory is consumed?

Is there service side cache like prepared SQL cache etc. used per connection?

There are two types of applications accessing the Postgres using jdbc driver and  Postgres odbc driver respectively.

Thanks,

Raj