Обсуждение: Question on pg_cron
Hello All,
We have around 10 different partition tables for which the partition maintenance is done using pg_partman extension. These tables have foreign key dependency between them. We just called partman.run_maintanance_proc() through pg_cron without any parameters and it was working fine. So we can see only one entry in the cron.job table. And it runs daily once.
It was all working fine and we were seeing the historical partition being dropped and new partitions being created without any issue. But suddenly we started seeing, its getting failed with error "ERROR: can not drop schema1.tab1_part_p2023_12_01 because other objects depend on it"
Then we realized , it may be the case that it's trying to run the partition maintenance for the Parent partition table first before the child partition table. So not sure how pg_partman handles the ordering of tables while doing the partition maintenance as we don't see any parameter to drive the ordering of the partition maintenance in part_config and we were under the impression pg_partman will take care of the sequence of partition maintenance automatically.
So want to understand if anybody encountered such issues?
And to handle the above issue , we are planning to call the partition maintenance of each of the TABLE by passing the table name explicitly to the run_maintanance_proc(), something as below. Is this advisable?
SELECT cron.unschedule('run_maintenance_proc');
SELECT cron.schedule(
'daily_partition_maintenance',
'0 2 * * *',
$$
DO $$
BEGIN
-- Run maintenance for child tables first
PERFORM partman.run_maintenance_proc('schema1.child_table1');
PERFORM partman.run_maintenance_proc('schema1.child_table2');
-- Add more child tables as needed
-- Run maintenance for parent tables next
PERFORM partman.run_maintenance_proc('schema1.parent_table1');
PERFORM partman.run_maintenance_proc('schema1.parent_table2');
-- Add more parent tables as needed
END;
$$;
$$
);
Or else
create a function like below and then call/schedule it through pg_cron
CREATE OR REPLACE FUNCTION run_partition_maintenance()
RETURNS void AS $$
BEGIN
-- Run maintenance for child tables first
PERFORM partman.run_maintenance_proc('schema1.child_table1');
PERFORM partman.run_maintenance_proc('schema1.child_table2');
-- Add more child tables as needed
-- Run maintenance for parent tables next
PERFORM partman.run_maintenance_proc('schema1.parent_table1');
PERFORM partman.run_maintenance_proc('schema1.parent_table2');
-- Add more parent tables as needed
END;
$$ LANGUAGE plpgsql;
SELECT cron.unschedule('run_maintenance_proc');
SELECT cron.schedule(
'daily_partition_maintenance',
'0 2 * * *',
'CALL run_partition_maintenance()'
);
We have around 10 different partition tables for which the partition maintenance is done using pg_partman extension. These tables have foreign key dependency between them. We just called partman.run_maintanance_proc() through pg_cron without any parameters and it was working fine. So we can see only one entry in the cron.job table. And it runs daily once.
It was all working fine and we were seeing the historical partition being dropped and new partitions being created without any issue. But suddenly we started seeing, its getting failed with error "ERROR: can not drop schema1.tab1_part_p2023_12_01 because other objects depend on it"
Then we realized , it may be the case that it's trying to run the partition maintenance for the Parent partition table first before the child partition table. So not sure how pg_partman handles the ordering of tables while doing the partition maintenance as we don't see any parameter to drive the ordering of the partition maintenance in part_config and we were under the impression pg_partman will take care of the sequence of partition maintenance automatically.
So want to understand if anybody encountered such issues?
And to handle the above issue , we are planning to call the partition maintenance of each of the TABLE by passing the table name explicitly to the run_maintanance_proc(), something as below. Is this advisable?
SELECT cron.unschedule('run_maintenance_proc');
SELECT cron.schedule(
'daily_partition_maintenance',
'0 2 * * *',
$$
DO $$
BEGIN
-- Run maintenance for child tables first
PERFORM partman.run_maintenance_proc('schema1.child_table1');
PERFORM partman.run_maintenance_proc('schema1.child_table2');
-- Add more child tables as needed
-- Run maintenance for parent tables next
PERFORM partman.run_maintenance_proc('schema1.parent_table1');
PERFORM partman.run_maintenance_proc('schema1.parent_table2');
-- Add more parent tables as needed
END;
$$;
$$
);
Or else
create a function like below and then call/schedule it through pg_cron
CREATE OR REPLACE FUNCTION run_partition_maintenance()
RETURNS void AS $$
BEGIN
-- Run maintenance for child tables first
PERFORM partman.run_maintenance_proc('schema1.child_table1');
PERFORM partman.run_maintenance_proc('schema1.child_table2');
-- Add more child tables as needed
-- Run maintenance for parent tables next
PERFORM partman.run_maintenance_proc('schema1.parent_table1');
PERFORM partman.run_maintenance_proc('schema1.parent_table2');
-- Add more parent tables as needed
END;
$$ LANGUAGE plpgsql;
SELECT cron.unschedule('run_maintenance_proc');
SELECT cron.schedule(
'daily_partition_maintenance',
'0 2 * * *',
'CALL run_partition_maintenance()'
);
On Sat, Jun 8, 2024 at 5:31 AM yudhi s <learnerdatabase99@gmail.com> wrote:
Hello All,
We have around 10 different partition tables for which the partition maintenance is done using pg_partman extension. These tables have foreign key dependency between them. We just called partman.run_maintanance_proc() through pg_cron without any parameters and it was working fine. So we can see only one entry in the cron.job table. And it runs daily once.
It was all working fine and we were seeing the historical partition being dropped and new partitions being created without any issue. But suddenly we started seeing, its getting failed with error "ERROR: can not drop schema1.tab1_part_p2023_12_01 because other objects depend on it"
Have you changed version lately of PG, pg_cron or pg_partman? Or maybe what pg_cron or pg_partman depends on?
On Sat, 8 Jun, 2024, 9:53 pm Ron Johnson, <ronljohnsonjr@gmail.com> wrote:
On Sat, Jun 8, 2024 at 5:31 AM yudhi s <learnerdatabase99@gmail.com> wrote:Hello All,
We have around 10 different partition tables for which the partition maintenance is done using pg_partman extension. These tables have foreign key dependency between them. We just called partman.run_maintanance_proc() through pg_cron without any parameters and it was working fine. So we can see only one entry in the cron.job table. And it runs daily once.
It was all working fine and we were seeing the historical partition being dropped and new partitions being created without any issue. But suddenly we started seeing, its getting failed with error "ERROR: can not drop schema1.tab1_part_p2023_12_01 because other objects depend on it"Have you changed version lately of PG, pg_cron or pg_partman? Or maybe what pg_cron or pg_partman depends on?
No version change, but we updated the part_config to set premake from 30 to 60 for all the tables. But not sure how that impacted this behavior.
However, do you think, we should better control the order of execution rather letting postgres to decide it's own, considering there is no such parameters for this ordering in part_config? And in that case which approach should we use out of the two i mentioned. Or any other strategies, should we follow, please advise?
On Sat, Jun 8, 2024 at 10:05 PM yudhi s <learnerdatabase99@gmail.com> wrote:
On Sat, 8 Jun, 2024, 9:53 pm Ron Johnson, <ronljohnsonjr@gmail.com> wrote:On Sat, Jun 8, 2024 at 5:31 AM yudhi s <learnerdatabase99@gmail.com> wrote:Hello All,
We have around 10 different partition tables for which the partition maintenance is done using pg_partman extension. These tables have foreign key dependency between them. We just called partman.run_maintanance_proc() through pg_cron without any parameters and it was working fine. So we can see only one entry in the cron.job table. And it runs daily once.
It was all working fine and we were seeing the historical partition being dropped and new partitions being created without any issue. But suddenly we started seeing, its getting failed with error "ERROR: can not drop schema1.tab1_part_p2023_12_01 because other objects depend on it"Have you changed version lately of PG, pg_cron or pg_partman? Or maybe what pg_cron or pg_partman depends on?No version change, but we updated the part_config to set premake from 30 to 60 for all the tables. But not sure how that impacted this behavior.However, do you think, we should better control the order of execution rather letting postgres to decide it's own, considering there is no such parameters for this ordering in part_config? And in that case which approach should we use out of the two i mentioned. Or any other strategies, should we follow, please advise?
I believe, You should log this as an issue in the pg_partman open source project.