[GENERAL] slow query on multiple table join
От | tao tony |
---|---|
Тема | [GENERAL] slow query on multiple table join |
Дата | |
Msg-id | MWHPR13MB13896E7A2C6C62E75EC1F13FAAEF0@MWHPR13MB1389.namprd13.prod.outlook.com обсуждение исходный текст |
Список | pgsql-general |
hi guys, I met a query performance issue in postgresql 9.6.2 with multiple tables joined. there were 2 slow queries,and the reasons were the same:the optimizer generate a bad explain which using nest loop. attached is the query and its explain.all tables are small and the indexes were only created on primary keys . in query 1 I noticed the explain forecast the nest loop anti join return 1 row as below,it was the result of (f join p) join pt: -> Nested Loop (cost=1.95..14838.66 rows=1 width=163) Join Filter: ((f.shop)::text = (s.uuid)::text) -> Nested Loop Anti Join (cost=1.95..14743.60 rows=1 width=111) -> Hash Join (cost=1.53..12067.46 rows=4751 width=115) Hash Cond: ((p.shop)::text = (f.shop)::text) -> Seq Scan on shopsku p (cost=0.00..11483.96 rows=106892 width=106) Filter: ((state)::text = 'normal'::text) -> Hash (cost=1.29..1.29 rows=19 width=9) -> Seq Scan on shopfranchise f (cost=0.00..1.29 rows=19 width=9) Filter: (enabled = 1) -> Index Only Scan using platformsku_pkey on platformsku pt (cost=0.42..0.55 rows=1 width=36) Index Cond: (uuid = (p.platformsku)::text) -> Seq Scan on shop s (cost=0.00..75.58 rows=1558 width=61) -> Hash (cost=2823.76..2823.76 rows=43376 width=46) -> Seq Scan on merchantsku m (cost=0.00..2823.76 rows=43376 width=46) while in analyze explain,it actually returns 57458 row.so higher level nest loop would get 57458*1558 rows,this cause this query runs for more than 40 seconds. -> Nested Loop (cost=1.95..14838.66 rows=1 width=163) (actual time=0.817..43150.583 rows=57458 loops=1) Join Filter: ((f.shop)::text = (s.uuid)::text) Rows Removed by Join Filter: 89462106 -> Nested Loop Anti Join (cost=1.95..14743.60 rows=1 width=111) (actual time=0.060..408.092 rows=57458 loops=1) -> Hash Join (cost=1.53..12067.46 rows=4751 width=115) (actual time=0.046..174.523 rows=57485 loops=1) Hash Cond: ((p.shop)::text = (f.shop)::text) -> Seq Scan on shopsku p (cost=0.00..11483.96 rows=106892 width=106) (actual time=0.008..107.416 rows=106580 loops=1) Filter: ((state)::text = 'normal'::text) Rows Removed by Filter: 429 -> Hash (cost=1.29..1.29 rows=19 width=9) (actual time=0.026..0.026 rows=20 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 9kB -> Seq Scan on shopfranchise f (cost=0.00..1.29 rows=19 width=9) (actual time=0.006..0.017 rows=20 loops=1) Filter: (enabled = 1) Rows Removed by Filter: 4 -> Index Only Scan using platformsku_pkey on platformsku pt (cost=0.42..0.55 rows=1 width=36) (actual time=0.003..0.003 rows=0 loops=57485) Index Cond: (uuid = (p.platformsku)::text) Heap Fetches: 0 -> Seq Scan on shop s (cost=0.00..75.58 rows=1558 width=61) (actual time=0.001..0.332 rows=1558 loops=57458) If I disabled nest loop,ti only use 519 ms. in query 2 ,the nest loop join also process more than 200m rows,it runs almost 2 minutes.After disable nest loop,it use hash join,finished in 1.5 sec. purchase join (shopfranchise f_4 join inventory k) join gdname -> Nested Loop Left Join (cost=3972.43..4192.40 rows=1 width=1074) (actual time=268.989..106066.932 rows=45615 loops=1) Join Filter: (((k.shop)::text = (purchase.shop)::text) AND ((k.shopsku)::text = (purchase.shopsku)::text)) Rows Removed by Join Filter: 208410367 (shopfranchise f_4 join inventory k) join gdname -> Hash Join (cost=3972.43..4165.52 rows=1 width=1112) (actual time=247.088..1754.448 rows=45615 loops=1) Hash Cond: (((gdname.shop)::text = (k.shop)::text) AND ((gdname.shopsku)::text = (k.shopsku)::text)) -> CTE Scan on gdname (cost=0.00..140.42 rows=7021 width=1246) (actual time=156.543..1563.121 rows=51153 loops=1) -> Hash (cost=3925.81..3925.81 rows=3108 width=63) (actual time=90.520..90.520 rows=45622 loops=1) Buckets: 65536 (originally 4096) Batches: 1 (originally 1) Memory Usage: 4745kB shopfranchise f_4 join inventory k -> Hash Join (cost=1.53..3925.81 rows=3108 width=63) (actual time=0.046..70.173 rows=45622 loops=1) Hash Cond: ((k.shop)::text = (f_4.shop)::text) -> Seq Scan on inventory k (cost=0.00..3449.47 rows=88747 width=54) (actual time=0.009..22.978 rows=88747 loops=1) -> Hash (cost=1.29..1.29 rows=19 width=9) (actual time=0.025..0.025 rows=19 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 9kB -> Seq Scan on shopfranchise f_4 (cost=0.00..1.29 rows=19 width=9) (actual time=0.006..0.017 rows=19 loops=1) Filter: (enabled = 1) Rows Removed by Filter: 4 -> CTE Scan on purchase (cost=0.00..15.36 rows=768 width=196) (actual time=0.001..1.013 rows=4569 loops=45615) please kindly let me know there's any solution to solve the problem,thanks a lot!
Вложения
В списке pgsql-general по дате отправления:
Предыдущее
От: Adrian KlaverДата:
Сообщение: Re: [GENERAL] Python versus Other Languages using PostgreSQL
Следующее
От: "Armand Pirvu (home)"Дата:
Сообщение: Re: [GENERAL] data transformation and replication