On Tuesday, May 31, 2022, Praneel Devisetty <
devisettypraneel@gmail.com> wrote:
Initially it was processing 1000 tables per minute. Performance is gradually dropping and now after 24 hr it was processing 90 tables per minute.
That seems like a fairly problematic metric given the general vast disparities in size tables have.
Building indexes is so IO heavy that the non-IO bottlenecks that exists likely have minimal impact on the overall times this rebuild everything will take. That said, I’ve never done anything at this scale before. I wouldn’t be too surprised if per-session cache effects are coming into play given the number of objects involved and the assumption that each session used for parallelism is persistent. I’m not sure how the parallelism works for managing the work queue though as it isn’t documented and I haven’t inspected the source code.