Out of memory on SELECT in 8.3.5

We've been having persistent out-of-memory errors occur in our production
8.3 deployment, which is now running 8.3.5. I'm not sure the query here is
the cause of the problem, but this is our most-recent example which
triggered an out-of-memory error for us.

Perhaps our configuration needs tweaking somewhere...

shared_buffers = 4000MB
#temp_buffers = 8MB
work_mem = 128MB
maintenance_work_mem = 256MB
#max_stack_depth = 2MB

I was wondering if somebody could spot anything odd looking in the query
plan or the memory parameters dumped to the log that might be causing
this?

explain SELECT
        s.sale_date,
        s.vin,
        s.dealer_region as sale_region,
        s.dealer_area as sale_area,
        s.dealer_code as sale_dealer,
        d.name as sale_dealer_name,
        s.model as sale_model,
        s.sale_type,
        ml.lead_id,
        l.created,
        lrd.reporting_date as lead_date,
        ml.lead_pos as lead_order,
        ml.lost_sale,
        ml.range,
        lrd.dealer_region as lead_region,
        lrd.dealer_area as lead_area,
        lrd.dealer_code as lead_dealer,
        lrd.model as lead_model,
        l.source as lead_source,
        lrd.oem_type as lead_type,
        lrd.oem_subtype as lead_subtype,
        lrd.unique_all as lead_unique_all,
        lrd.unique_region as lead_unique_region,
        lrd.unique_area as lead_unique_area,
        lrd.unique_dealer as lead_unique_dealer,
        lrd.customer_fname,
        lrd.customer_lname,
        lrdm.item_value as lead_tracking_code,
        address_street.street as owner_street,
        address.city as owner_city,
        address.state as owner_state,
        address.postal_code as owner_postal_code,
        address.country as owner_country,
        email.list_value as owner_email,
        phone.list_value as owner_phone

    FROM mb_sale s
    LEFT OUTER JOIN dealer d on d.dealer_code = s.dealer_code
    LEFT OUTER JOIN mb_lead ml on ml.sale_id = s.id
    LEFT OUTER JOIN lead_reporting_data lrd ON lrd.lead_id = ml.lead_id
    LEFT OUTER JOIN lead_reporting_meta lrdm
        ON lrdm.lead_id = ml.lead_id
        AND lrdm.item_key = '[ALS:prospectid]TrackingCode'
        AND lrdm.pos=1
    LEFT OUTER JOIN lead l ON l.id = ml.lead_id

    LEFT OUTER JOIN lead_reporting_address address ON address.lead_id =
ml.lead_id
    LEFT OUTER JOIN lead_reporting_street address_street
        ON address_street.address_id = address.id AND address_street.pos = 0

    LEFT OUTER JOIN lead_reporting_list_data phone
        ON phone.lead_id = ml.lead_id AND phone.list_type = 'p' AND phone.pos = 0

    LEFT OUTER JOIN lead_reporting_list_data email
        ON email.lead_id = ml.lead_id AND email.list_type = 'e' AND email.pos = 0

    WHERE
        ((s.sale_type = 'd' and ml.lost_sale = FALSE) OR (s.sale_type = 'c' and
ml.lead_pos = 0) OR (s.sale_type = '0'))
        AND s.sale_date BETWEEN date('2009-01-01') AND date('2009-01-31')
    ORDER BY s.sale_date, s.vin
                                                                                                     QUERY
PLAN

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=2297525.76..2297626.60 rows=40336 width=234)
   Sort Key: s.sale_date, s.vin
   ->  Hash Left Join  (cost=1450193.63..2294440.10 rows=40336 width=234)
         Hash Cond: (ml.lead_id = phone.lead_id)
         ->  Hash Left Join  (cost=1341176.18..2140101.73 rows=40336
width=219)
               Hash Cond: (ml.lead_id = email.lead_id)
               ->  Nested Loop Left Join  (cost=1237098.64..2000317.19
rows=40336 width=204)
                     ->  Nested Loop Left Join
(cost=1237098.64..1806747.26 rows=40336 width=141)
                           ->  Nested Loop Left Join
(cost=1237098.64..1633400.88 rows=40336
width=133)
                                 ->  Nested Loop Left Join
(cost=1237098.64..1413899.40 rows=40336
width=118)
                                       ->  Merge Left Join
(cost=1237098.64..1238143.69
rows=40336 width=89)
                                             Merge Cond: (ml.lead_id =
lrdm.lead_id)
                                             ->  Sort
(cost=46384.08..46484.92
rows=40336 width=78)
                                                   Sort Key: ml.lead_id
                                                   ->  Hash Left Join
(cost=9430.33..43298.42
rows=40336 width=78)
                                                         Hash Cond:
((s.dealer_code)::text
=
(d.dealer_code)::text)
                                                         ->  Hash Left
Join
(cost=9094.04..42407.50
rows=40336
width=60)
                                                               Hash Cond:
(s.id =
ml.sale_id)
                                                               Filter:
(((s.sale_type
=
'd'::bpchar)
AND (NOT
ml.lost_sale))
OR
((s.sale_type
=
'c'::bpchar)
AND
(ml.lead_pos
= 0)) OR
(s.sale_type
=
'0'::bpchar))
                                                               ->  Index
Scan using
mb_sale_sale_date_idx
on mb_sale
s
(cost=0.00..14027.94
rows=43804
width=50)
                                                                     Index
Cond:
((sale_date
>=
'2009-01-01'::date)
AND
(sale_date
<=
'2009-01-31'::date))
                                                               ->  Hash
(cost=5577.35..5577.35
rows=281335
width=26)
                                                                     ->
Seq
Scan
on
mb_lead
ml
(cost=0.00..5577.35
rows=281335
width=26)
                                                         ->  Hash
(cost=321.13..321.13
rows=1213
width=23)
                                                               ->  Seq
Scan on
dealer d
(cost=0.00..321.13
rows=1213
width=23)
                                             ->  Sort
(cost=1190714.57..1191036.34
rows=128709 width=19)
                                                   Sort Key: lrdm.lead_id
                                                   ->  Bitmap Heap Scan on
lead_reporting_meta
lrdm
(cost=118675.76..1179791.19
rows=128709 width=19)
                                                         Recheck Cond:
(item_key =
'[ALS:prospectid]TrackingCode'::text)
                                                         Filter: (pos = 1)
                                                         ->  Bitmap Index
Scan on
lead_reporting_meta_item_key_idx

(cost=0.00..118643.58
rows=1476580
width=0)
                                                               Index Cond:
(item_key =
'[ALS:prospectid]TrackingCode'::text)
                                       ->  Index Scan using
lead_reporting_address_lead_id_idx
on lead_reporting_address address
(cost=0.00..4.34 rows=1 width=37)
                                             Index Cond: (address.lead_id
= ml.lead_id)
                                 ->  Index Scan using lead_pkey on lead l
(cost=0.00..5.43 rows=1 width=23)
                                       Index Cond: (l.id = ml.lead_id)
                           ->  Index Scan using
lead_reporting_street_address_id_idx on
lead_reporting_street address_street
(cost=0.00..4.29 rows=1 width=24)
                                 Index Cond: (address_street.address_id =
address.id)
                                 Filter: (address_street.pos = 0)
                     ->  Index Scan using lead_reporting_data_pkey on
lead_reporting_data lrd  (cost=0.00..4.79 rows=1
width=71)
                           Index Cond: (lrd.lead_id = ml.lead_id)
               ->  Hash  (cost=85837.99..85837.99 rows=1459164 width=23)
                     ->  Seq Scan on lead_reporting_list_data email
(cost=0.00..85837.99 rows=1459164 width=23)
                           Filter: ((list_type = 'e'::bpchar) AND (pos = 0))
         ->  Hash  (cost=85837.99..85837.99 rows=1854357 width=23)
               ->  Seq Scan on lead_reporting_list_data phone
(cost=0.00..85837.99 rows=1854357 width=23)
                     Filter: ((list_type = 'p'::bpchar) AND (pos = 0))
(47 rows)

TopMemoryContext: 154528 total in 18 blocks; 5240 free (20 chunks); 149288
used
  TopTransactionContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568
used
  PL/PgSQL function context: 8192 total in 1 blocks; 5512 free (4 chunks);
2680 used
  PL/PgSQL function context: 57344 total in 3 blocks; 42720 free (11
chunks); 14624 used
  PL/PgSQL function context: 8192 total in 1 blocks; 4648 free (7 chunks);
3544 used
  Rendezvous variable hash: 8192 total in 1 blocks; 1680 free (0 chunks);
6512 used
  PLpgSQL function cache: 24520 total in 2 blocks; 3744 free (0 chunks);
20776 used
  Record information cache: 24576 total in 2 blocks; 15984 free (5
chunks); 8592 used
  CFuncHash: 8192 total in 1 blocks; 1680 free (0 chunks); 6512 used
  RI compare cache: 24576 total in 2 blocks; 15984 free (5 chunks); 8592 used
  RI query cache: 24576 total in 2 blocks; 11888 free (5 chunks); 12688 used
  Type information cache: 24576 total in 2 blocks; 11888 free (5 chunks);
12688 used
  Operator class cache: 8192 total in 1 blocks; 1680 free (0 chunks); 6512
used
  Operator lookup cache: 24576 total in 2 blocks; 11888 free (5 chunks);
12688 used
  Prepared Queries: 24576 total in 2 blocks; 15984 free (5 chunks); 8592 used
  MessageContext: 8192 total in 1 blocks; 6976 free (0 chunks); 1216 used
  smgr relation table: 24576 total in 2 blocks; 3584 free (4 chunks);
20992 used
  TransactionAbortContext: 32768 total in 1 blocks; 32736 free (0 chunks);
32 used
  Portal hash: 8192 total in 1 blocks; 1680 free (0 chunks); 6512 used
  PortalMemory: 8192 total in 1 blocks; 7888 free (0 chunks); 304 used
    PortalHeapMemory: 1021952 total in 126 blocks; 11008 free (181
chunks); 1010944 used
      ExecutorState: 4972936 total in 6 blocks; 38456 free (11 chunks);
4934480 used
        HashTableContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
          HashBatchContext: 148889648 total in 28 blocks; 2532544 free (5
chunks); 146357104 used
        HashTableContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
          HashBatchContext: 144695344 total in 27 blocks; 828368 free (12
chunks); 143866976 used
        TIDBitmap: 50356272 total in 16 blocks; 7517392 free (41 chunks);
42838880 used
        TupleSort: 142649352 total in 25 blocks; 92237584 free (1339345
chunks); 50411768 used
        HashTableContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
          HashBatchContext: 131072 total in 4 blocks; 20064 free (3
chunks); 111008 used
        HashTableContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
          HashBatchContext: 25419824 total in 13 blocks; 2650448 free (5
chunks); 22769376 used
        TupleSort: 9953328 total in 11 blocks; 1277840 free (9 chunks);
8675488 used
        TupleSort: 1138736 total in 8 blocks; 768 free (8 chunks); 1137968
used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
        ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
      JoinRelHashTable: 8192 total in 1 blocks; 624 free (0 chunks); 7568
used
  Relcache by OID: 24576 total in 2 blocks; 10752 free (3 chunks); 13824 used
  CacheMemoryContext: 1341680 total in 21 blocks; 174632 free (83 chunks);
1167048 used
    unnamed prepared statement: 516096 total in 6 blocks; 231360 free (2
chunks); 284736 used
    CachedPlan: 3072 total in 2 blocks; 1160 free (0 chunks); 1912 used
    pg_statistic_relid_att_index: 2048 total in 1 blocks; 608 free (0
chunks); 1440 used
    locks_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    lead_reporting_list_data_value_lower_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_list_data_value_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    lead_reporting_list_data_subtype_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    lead_reporting_list_data_type_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_list_data_lead_id_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    lead_reporting_list_data_pkey: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_street_street_lower_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_street_address_id_idx: 2048 total in 1 blocks; 704 free
(0 chunks); 1344 used
    lead_reporting_street_pkey: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_address_postal_code_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_address_state_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_address_city_lower_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_address_lead_id_idx: 2048 total in 1 blocks; 704 free
(0 chunks); 1344 used
    lead_reporting_address_pkey: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    mb_lead_sale_id_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296
used
    mb_lead_lead_id_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296
used
    mb_lead_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    mb_sale_dealer_region_area_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    mb_sale_dealer_code_idx: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    mb_sale_sale_date_idx: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    mb_sale_type_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    mb_sale_model_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    mb_sale_dealer_region_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    mb_sale_dealer_area_idx: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    mb_sale_vin_key: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    mb_sale_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    CachedPlan: 3072 total in 2 blocks; 648 free (0 chunks); 2424 used
    pg_toast_17933_index: 2048 total in 1 blocks; 608 free (0 chunks);
1440 used
    config_xml_name_key: 2048 total in 1 blocks; 704 free (0 chunks); 1344
used
    config_xml_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    CachedPlan: 1024 total in 1 blocks; 312 free (0 chunks); 712 used
    CachedPlanSource: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    SPI Plan: 1024 total in 1 blocks; 832 free (0 chunks); 192 used
    CachedPlan: 1024 total in 1 blocks; 336 free (0 chunks); 688 used
    CachedPlanSource: 1024 total in 1 blocks; 80 free (0 chunks); 944 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlanSource: 1024 total in 1 blocks; 24 free (0 chunks); 1000 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 7168 total in 3 blocks; 3888 free (1 chunks); 3280 used
    CachedPlan: 7168 total in 3 blocks; 3920 free (0 chunks); 3248 used
    CachedPlanSource: 3072 total in 2 blocks; 888 free (0 chunks); 2184 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 7168 total in 3 blocks; 3704 free (1 chunks); 3464 used
    CachedPlanSource: 3072 total in 2 blocks; 896 free (0 chunks); 2176 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 3072 total in 2 blocks; 1456 free (1 chunks); 1616 used
    CachedPlanSource: 3072 total in 2 blocks; 1664 free (0 chunks); 1408 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 1024 total in 1 blocks; 336 free (0 chunks); 688 used
    CachedPlanSource: 1024 total in 1 blocks; 80 free (0 chunks); 944 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 3072 total in 2 blocks; 312 free (0 chunks); 2760 used
    CachedPlanSource: 3072 total in 2 blocks; 1568 free (0 chunks); 1504 used
    SPI Plan: 1024 total in 1 blocks; 800 free (0 chunks); 224 used
    cl_event_first_response_pkey: 2048 total in 1 blocks; 704 free (0
chunks); 1344 used
    CachedPlan: 1024 total in 1 blocks; 336 free (0 chunks); 688 used
    CachedPlanSource: 1024 total in 1 blocks; 80 free (0 chunks); 944 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 1024 total in 1 blocks; 336 free (0 chunks); 688 used
    CachedPlanSource: 1024 total in 1 blocks; 80 free (0 chunks); 944 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 3072 total in 2 blocks; 1728 free (0 chunks); 1344 used
    CachedPlanSource: 3072 total in 2 blocks; 1536 free (0 chunks); 1536 used
    SPI Plan: 1024 total in 1 blocks; 800 free (0 chunks); 224 used
    CachedPlan: 15360 total in 4 blocks; 7104 free (1 chunks); 8256 used
    CachedPlanSource: 7168 total in 3 blocks; 2976 free (2 chunks); 4192 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 3072 total in 2 blocks; 1568 free (0 chunks); 1504 used
    CachedPlanSource: 3072 total in 2 blocks; 1352 free (0 chunks); 1720 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 3072 total in 2 blocks; 368 free (0 chunks); 2704 used
    CachedPlanSource: 3072 total in 2 blocks; 40 free (0 chunks); 3032 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    CachedPlan: 7168 total in 3 blocks; 3704 free (1 chunks); 3464 used
    CachedPlanSource: 3072 total in 2 blocks; 896 free (0 chunks); 2176 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    cl_event_crm_idx: 2048 total in 1 blocks; 704 free (0 chunks); 1344 used
    cl_event_lead_id_idx: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    cl_event_event_name_idx: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    cl_event_crm_lead_id_idx: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    cl_event_pkey: 2048 total in 1 blocks; 704 free (0 chunks); 1344 used
    locality_data_postal_code_norm_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    locality_data_postal_code_key: 2048 total in 1 blocks; 392 free (0
chunks); 1656 used
    locality_data_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296
used
    lead_reporting_meta_item_value_key_idx: 2048 total in 1 blocks; 608
free (0 chunks); 1440 used
    lead_reporting_meta_item_key_idx: 2048 total in 1 blocks; 704 free (0
chunks); 1344 used
    lead_reporting_meta_item_type_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_meta_lead_id_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_meta_pkey: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    CachedPlan: 1024 total in 1 blocks; 312 free (0 chunks); 712 used
    CachedPlanSource: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    SPI Plan: 1024 total in 1 blocks; 832 free (0 chunks); 192 used
    CachedPlan: 1024 total in 1 blocks; 312 free (0 chunks); 712 used
    CachedPlanSource: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    SPI Plan: 1024 total in 1 blocks; 832 free (0 chunks); 192 used
    CachedPlanSource: 3072 total in 2 blocks; 1248 free (1 chunks); 1824 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    pg_toast_18404_index: 2048 total in 1 blocks; 608 free (0 chunks);
1440 used
    pg_attrdef_adrelid_adnum_index: 2048 total in 1 blocks; 608 free (0
chunks); 1440 used
    CachedPlan: 3072 total in 2 blocks; 936 free (0 chunks); 2136 used
    CachedPlanSource: 3072 total in 2 blocks; 1424 free (0 chunks); 1648 used
    SPI Plan: 1024 total in 1 blocks; 832 free (0 chunks); 192 used
    CachedPlanSource: 3072 total in 2 blocks; 360 free (0 chunks); 2712 used
    SPI Plan: 1024 total in 1 blocks; 800 free (0 chunks); 224 used
    CachedPlan: 7168 total in 3 blocks; 3888 free (1 chunks); 3280 used
    CachedPlanSource: 3072 total in 2 blocks; 360 free (0 chunks); 2712 used
    SPI Plan: 1024 total in 1 blocks; 800 free (0 chunks); 224 used
    CachedPlan: 7168 total in 3 blocks; 3704 free (1 chunks); 3464 used
    CachedPlanSource: 3072 total in 2 blocks; 896 free (0 chunks); 2176 used
    SPI Plan: 1024 total in 1 blocks; 808 free (0 chunks); 216 used
    region_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    pg_toast_18005_index: 2048 total in 1 blocks; 608 free (0 chunks);
1440 used
    request_state_idx: 2048 total in 1 blocks; 704 free (0 chunks); 1344 used
    lead_id_idx: 2048 total in 1 blocks; 704 free (0 chunks); 1344 used
    lead_billing_contention_pkey: 2048 total in 1 blocks; 704 free (0
chunks); 1344 used
    lead_reporting_data_delivery_meth_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_data_typeflags_idx: 2048 total in 1 blocks; 600 free (0
chunks); 1448 used
    lead_reporting_data_unique_dealer_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_data_unique_area_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    lead_reporting_data_unique_region_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_data_unique_all_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    lead_reporting_data_oem_subtype_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    lead_reporting_data_oem_type_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_data_dealer_area_idx: 2048 total in 1 blocks; 752 free
(0 chunks); 1296 used
    lead_reporting_data_customer_lname_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_data_customer_fname_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_data_customer_postal_code_idx: 2048 total in 1 blocks;
752 free (0 chunks); 1296 used
    lead_reporting_data_dealer_region_area_idx: 2048 total in 1 blocks;
752 free (0 chunks); 1296 used
    lead_reporting_data_dealer_region_idx: 2048 total in 1 blocks; 752
free (0 chunks); 1296 used
    lead_reporting_data_dealer_code_idx: 2048 total in 1 blocks; 704 free
(0 chunks); 1344 used
    lead_reporting_data_reporting_date_idx: 2048 total in 1 blocks; 704
free (0 chunks); 1344 used
    lead_reporting_data_model_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    lead_reporting_data_pkey: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    lead_reporting: 23552 total in 5 blocks; 7240 free (0 chunks); 16312 used
    lead_processing_state_step_idx: 2048 total in 1 blocks; 656 free (0
chunks); 1392 used
    reporting_modified_idx: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    reporting_last_processed_date_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    reporting_date_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296
used
    lead_created_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    lead_processing_step_idx: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    lead_destination_idx: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    lead_modified_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    lead_source_idx: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    lead_pkey: 2048 total in 1 blocks; 704 free (0 chunks); 1344 used
    metadata_key_value_idx: 2048 total in 1 blocks; 656 free (0 chunks);
1392 used
    metadata_pkey: 2048 total in 1 blocks; 704 free (0 chunks); 1344 used
    dealer_metadata_pkey: 2048 total in 1 blocks; 608 free (0 chunks);
1440 used
    xml_schema_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    ehcache_expiration_idx: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    ehcache_pkey: 2048 total in 1 blocks; 608 free (0 chunks); 1440 used
    external_system_user_name_key: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    external_system_name_key: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    external_system_pkey: 2048 total in 1 blocks; 752 free (0 chunks);
1296 used
    dealer_external_system_id_idx: 2048 total in 1 blocks; 752 free (0
chunks); 1296 used
    dealer_dealer_code_key: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    dealer_pkey: 2048 total in 1 blocks; 752 free (0 chunks); 1296 used
    CachedPlan: 1024 total in 1 blocks; 816 free (0 chunks); 208 used
    CachedPlanSource: 1024 total in 1 blocks; 664 free (1 chunks); 360 used
    xslt_style_sheet_pkey: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    pg_index_indrelid_index: 2048 total in 1 blocks; 704 free (0 chunks);
1344 used
    CachedPlan: 1024 total in 1 blocks; 816 free (0 chunks); 208 used
    CachedPlanSource: 1024 total in 1 blocks; 664 free (1 chunks); 360 used
    pg_ts_dict_oid_index: 3072 total in 2 blocks; 1744 free (3 chunks);
1328 used
    pg_aggregate_fnoid_index: 3072 total in 2 blocks; 1696 free (2
chunks); 1376 used
    pg_language_name_index: 3072 total in 2 blocks; 1744 free (3 chunks);
1328 used
    pg_ts_dict_dictname_index: 3072 total in 2 blocks; 1648 free (2
chunks); 1424 used
    pg_namespace_nspname_index: 3072 total in 2 blocks; 1696 free (2
chunks); 1376 used
    pg_opfamily_oid_index: 3072 total in 2 blocks; 1744 free (3 chunks);
1328 used
    pg_opclass_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_ts_parser_prsname_index: 3072 total in 2 blocks; 1648 free (2
chunks); 1424 used
    pg_amop_fam_strat_index: 3072 total in 2 blocks; 1384 free (2 chunks);
1688 used
    pg_opclass_am_name_nsp_index: 3072 total in 2 blocks; 1576 free (3
chunks); 1496 used
    pg_trigger_tgrelid_tgname_index: 3072 total in 2 blocks; 1600 free (2
chunks); 1472 used
    pg_cast_source_target_index: 3072 total in 2 blocks; 1600 free (2
chunks); 1472 used
    pg_auth_members_role_member_index: 3072 total in 2 blocks; 1648 free
(2 chunks); 1424 used
    pg_attribute_relid_attnum_index: 3072 total in 2 blocks; 1600 free (2
chunks); 1472 used
    pg_ts_config_cfgname_index: 3072 total in 2 blocks; 1648 free (2
chunks); 1424 used
    pg_authid_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_ts_config_oid_index: 3072 total in 2 blocks; 1744 free (3 chunks);
1328 used
    pg_conversion_default_index: 3072 total in 2 blocks; 1432 free (3
chunks); 1640 used
    pg_language_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_enum_oid_index: 3072 total in 2 blocks; 1744 free (3 chunks); 1328
used
    pg_proc_proname_args_nsp_index: 3072 total in 2 blocks; 1576 free (3
chunks); 1496 used
    pg_ts_parser_oid_index: 3072 total in 2 blocks; 1744 free (3 chunks);
1328 used
    pg_database_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_conversion_name_nsp_index: 3072 total in 2 blocks; 1648 free (2
chunks); 1424 used
    pg_class_relname_nsp_index: 3072 total in 2 blocks; 1600 free (2
chunks); 1472 used
    pg_attribute_relid_attnam_index: 3072 total in 2 blocks; 1648 free (2
chunks); 1424 used
    pg_class_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks); 1376
used
    pg_amproc_fam_proc_index: 3072 total in 2 blocks; 1384 free (2
chunks); 1688 used
    pg_operator_oprname_l_r_n_index: 3072 total in 2 blocks; 1384 free (2
chunks); 1688 used
    pg_index_indexrelid_index: 3072 total in 2 blocks; 1696 free (2
chunks); 1376 used
    pg_type_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks); 1376
used
    pg_rewrite_rel_rulename_index: 3072 total in 2 blocks; 1600 free (2
chunks); 1472 used
    pg_authid_rolname_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_auth_members_member_role_index: 3072 total in 2 blocks; 1648 free
(2 chunks); 1424 used
    pg_enum_typid_label_index: 3072 total in 2 blocks; 1648 free (2
chunks); 1424 used
    pg_constraint_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_conversion_oid_index: 3072 total in 2 blocks; 1744 free (3 chunks);
1328 used
    pg_ts_template_tmplname_index: 3072 total in 2 blocks; 1648 free (2
chunks); 1424 used
    pg_ts_config_map_index: 3072 total in 2 blocks; 1624 free (3 chunks);
1448 used
    pg_namespace_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_type_typname_nsp_index: 3072 total in 2 blocks; 1600 free (2
chunks); 1472 used
    pg_operator_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks);
1376 used
    pg_amop_opr_fam_index: 3072 total in 2 blocks; 1600 free (2 chunks);
1472 used
    pg_proc_oid_index: 3072 total in 2 blocks; 1696 free (2 chunks); 1376
used
    pg_opfamily_am_name_nsp_index: 3072 total in 2 blocks; 1624 free (3
chunks); 1448 used
    pg_ts_template_oid_index: 3072 total in 2 blocks; 1744 free (3
chunks); 1328 used
  MdSmgr: 24576 total in 2 blocks; 16192 free (0 chunks); 8384 used
  LOCALLOCK hash: 24576 total in 2 blocks; 11872 free (5 chunks); 12704 used
  Timezones: 53584 total in 2 blocks; 3744 free (0 chunks); 49840 used
  ErrorContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
2009-02-08 20:11:27 CST lms_nna ERROR:  out of memory
2009-02-08 20:11:27 CST lms_nna DETAIL:  Failed on request of size 243.