Обсуждение: Unable to execute Query in parallel for partitioned table

On Thu, 2021-02-11 at 05:09 +0530, Brajendra Pratap wrote:
> I am unable to execute the below in parallel plz suggest how can I achieve parallelism here.
>
> select count(*) over () VIEWALLROWCOUNT,abc.*,ROW_NUMBER()  OVER (order by trn_transaction_date desc ) AS RowNumber from (
> select * from transactions where trn_store_date_id=20201202) abc;
>
> Query plan is as mentioned below :-
>
> explain analyze select count(*) over () VIEWALLROWCOUNT,abc.*,ROW_NUMBER()  OVER (order by trn_transaction_date desc ) AS RowNumber from (
> select * from transactions where trn_store_date_id=20201218) abc;
> LOG:  duration: 25820.176 ms  statement: explain analyze select count(*) over () VIEWALLROWCOUNT,abc.*,ROW_NUMBER()  OVER (order by trn_transaction_date desc ) AS RowNumber from (
> select * from transactions where trn_store_date_id=20201218) abc;
>                                                                                               QUERY PLAN
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>  WindowAgg  (cost=4474843.51..4498066.81 rows=774110 width=21297) (actual time=21455.495..25241.738 rows=795190 loops=1)
>    ->  WindowAgg  (cost=4474843.51..4488390.44 rows=774110 width=21289) (actual time=10588.494..15311.865 rows=795190 loops=1)
>          ->  Sort  (cost=4474843.51..4476778.79 rows=774110 width=21281) (actual time=10588.422..11771.300 rows=795190 loops=1)
>                Sort Key: transactions.trn_transaction_date DESC
>                Sort Method: external merge  Disk: 1496856kB
>                ->  Result  (cost=0.00..270640.32 rows=774110 width=21281) (actual time=0.117..4504.159 rows=795190 loops=1)
>                      ->  Append  (cost=0.00..262899.22 rows=774110 width=21281) (actual time=0.094..1449.532 rows=795190 loops=1)
>                            ->  Seq Scan on transactions  (cost=0.00..0.00 rows=1 width=47554) (actual time=0.019..0.019 rows=0 loops=1)
>                                  Filter: (trn_store_date_id = 20201218)
>                            ->  Index Scan using idx_202012_trn_store_date_id on transactions_202012  (cost=0.56..259028.67 rows=774109 width=21281) (actual time=0.074..1357.764 rows=795190 loops=1)
>                                  Index Cond: (trn_store_date_id = 20201218)
>  Planning Time: 116.472 ms
>  Execution Time: 25676.098 ms
>
> Note :- We had tried different options like max_worker_processes, max_parallel_workers, max_parallel_workers_per_gather,max_parallel_maintenance_worker to execute it in parallel but no luck.

I don't think parallelization will help you here.

Your problem is probably the "abc.*" in the SELECT list.

There must be really large data in this table, so it takes a long time to fetch and
sort the rows.  Try selecting only the columns you need.

Alternatively, add a LIMIT clause.  Do you really need all 800000 rows?

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com

Hi Albe,

Thank you so much for information, will check this and get back to you if any help required.

I have a doubt why didn't the parallelism works here ,could u plz guide me?

Thank you so much again.

On Thu, 11 Feb, 2021, 1:23 PM Laurenz Albe, <laurenz.albe@cybertec.at> wrote:

On Thu, 2021-02-11 at 05:09 +0530, Brajendra Pratap wrote:
> I am unable to execute the below in parallel plz suggest how can I achieve parallelism here.
>
> select count(*) over () VIEWALLROWCOUNT,abc.*,ROW_NUMBER()  OVER (order by trn_transaction_date desc ) AS RowNumber from (
> select * from transactions where trn_store_date_id=20201202) abc;
>
> Query plan is as mentioned below :-
>
> explain analyze select count(*) over () VIEWALLROWCOUNT,abc.*,ROW_NUMBER()  OVER (order by trn_transaction_date desc ) AS RowNumber from (
> select * from transactions where trn_store_date_id=20201218) abc;
> LOG:  duration: 25820.176 ms  statement: explain analyze select count(*) over () VIEWALLROWCOUNT,abc.*,ROW_NUMBER()  OVER (order by trn_transaction_date desc ) AS RowNumber from (
> select * from transactions where trn_store_date_id=20201218) abc;
>                                                                                               QUERY PLAN
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>  WindowAgg  (cost=4474843.51..4498066.81 rows=774110 width=21297) (actual time=21455.495..25241.738 rows=795190 loops=1)
>    ->  WindowAgg  (cost=4474843.51..4488390.44 rows=774110 width=21289) (actual time=10588.494..15311.865 rows=795190 loops=1)
>          ->  Sort  (cost=4474843.51..4476778.79 rows=774110 width=21281) (actual time=10588.422..11771.300 rows=795190 loops=1)
>                Sort Key: transactions.trn_transaction_date DESC
>                Sort Method: external merge  Disk: 1496856kB
>                ->  Result  (cost=0.00..270640.32 rows=774110 width=21281) (actual time=0.117..4504.159 rows=795190 loops=1)
>                      ->  Append  (cost=0.00..262899.22 rows=774110 width=21281) (actual time=0.094..1449.532 rows=795190 loops=1)
>                            ->  Seq Scan on transactions  (cost=0.00..0.00 rows=1 width=47554) (actual time=0.019..0.019 rows=0 loops=1)
>                                  Filter: (trn_store_date_id = 20201218)
>                            ->  Index Scan using idx_202012_trn_store_date_id on transactions_202012  (cost=0.56..259028.67 rows=774109 width=21281) (actual time=0.074..1357.764 rows=795190 loops=1)
>                                  Index Cond: (trn_store_date_id = 20201218)
>  Planning Time: 116.472 ms
>  Execution Time: 25676.098 ms
>
> Note :- We had tried different options like max_worker_processes, max_parallel_workers, max_parallel_workers_per_gather,max_parallel_maintenance_worker to execute it in parallel but no luck.

I don't think parallelization will help you here.

Your problem is probably the "abc.*" in the SELECT list.

There must be really large data in this table, so it takes a long time to fetch and
sort the rows.  Try selecting only the columns you need.

Alternatively, add a LIMIT clause.  Do you really need all 800000 rows?

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com