Обсуждение: multiple sessions to create index in parallel support
Hi,
We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create indexes , with only one connection , the create index automatically run in parallel (table size 40G) , then we started two connections and each session creating index on large table, but at this time, only 1 session parallel creating index, the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1 session allowed to parallel index creating at the same time ?
Thanks,
James
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create
indexes, with only one connection , the create index automatically run in parallel (table size 40G) , then we started
twoconnections and each session creating index on large table, but at this time, only 1 session parallel creating
index,the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1
sessionallowed to parallel index creating at the same time ?
Check your settings for max_parallel_workers and
max_parallel_maintenance_workers.
regards, tom lane
Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a large
tableshow parallel 8 workers, the other session creating one index in another separate table always in serial. Even
increasemax_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the other
isdone in serial.
James
-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Tuesday, June 21, 2022 10:30 PM
To: James Pang (chaolpan) <chaolpan@cisco.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: multiple sessions to create index in parallel support
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create
indexes, with only one connection , the create index automatically run in parallel (table size 40G) , then we started
twoconnections and each session creating index on large table, but at this time, only 1 session parallel creating
index,the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1
sessionallowed to parallel index creating at the same time ?
Check your settings for max_parallel_workers and max_parallel_maintenance_workers.
regards, tom lane
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a
largetable show parallel 8 workers, the other session creating one index in another separate table always in serial.
Evenincrease max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the
otheris done in serial.
Hmm, what about max_worker_processes ? That's an independent limit.
regards, tom lane
All max_workers_process got increased to 32 before creating index. The 2 session try to create index on totally
separatelarge tables(40gb) , either one in 8 parallel workers, but not both.
James
-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Tuesday, June 21, 2022 11:04 PM
To: James Pang (chaolpan) <chaolpan@cisco.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: multiple sessions to create index in parallel support
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a
largetable show parallel 8 workers, the other session creating one index in another separate table always in serial.
Evenincrease max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the
otheris done in serial.
Hmm, what about max_worker_processes ? That's an independent limit.
regards, tom lane
Sorry for confusion. This time, I add "max_parallel_workers"=32 , "max_parallel_maintenance_workers"=32 ,
"max_workers_process"=64to postgresql.conf, and restart database. This time, both session runs in parallel now. Last
severaltests only got these parameter changes by set max_parallell_workers... in session level , for each session ,
only1 session parallel creating index.
James
-----Original Message-----
From: James Pang (chaolpan)
Sent: Wednesday, June 22, 2022 9:34 AM
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-admin@lists.postgresql.org
Subject: RE: multiple sessions to create index in parallel support
All max_workers_process got increased to 32 before creating index. The 2 session try to create index on totally
separatelarge tables(40gb) , either one in 8 parallel workers, but not both.
James
-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Tuesday, June 21, 2022 11:04 PM
To: James Pang (chaolpan) <chaolpan@cisco.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: multiple sessions to create index in parallel support
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a
largetable show parallel 8 workers, the other session creating one index in another separate table always in serial.
Evenincrease max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the
otheris done in serial.
Hmm, what about max_worker_processes ? That's an independent limit.
regards, tom lane