Re: Add an option to skip loading missing publication to avoid logical replication failure

On Mon, 3 Mar 2025 at 16:41, Amit Kapila  wrote:
>
> On Mon, Mar 3, 2025 at 2:30 PM vignesh C  wrote:
> >
> > On Tue, 25 Feb 2025 at 15:32, vignesh C  wrote:
> > >
> > > The attached script has the script that was used for testing. Here the
> > > NUM_RECORDS count should be changed accordingly for each of the tests
> > > and while running the test with the patch change uncomment the drop
> > > publication command.
> >
> > I have done further analysis on the test and changed the test to
> > compare it better with HEAD. The execution time is in milliseconds.
> > Brach/records  |  100     |  1000   |  10000    |  100000  |  1000000
> > Head               |   10.43  |  15.86  |   64.44    |  550.56  |   8991.04
> > Patch              |   11.35  |  17.26   |   73.50    |  640.21  |  10104.72
> > % diff              |   -8.82  |  -8.85    |   -14.08   |   -16.28  |  -12.38
> >
> > There is a  performance degradation in the range of 8.8 to 16.2 percent.
> >
>
> - /* Validate the entry */
> - if (!entry->replicate_valid)
> + /*
> + * If the publication is invalid, check for updates.
> + * This optimization ensures that the next block, which queries the system
> + * tables and builds the relation entry, runs only if a new publication was
> + * created.
> + */
> + if (!publications_valid && data->publications)
> + {
> + bool skipped_pub = false;
> + List    *publications;
> +
> + publications = LoadPublications(data->publication_names, &skipped_pub);
>
> The publications_valid flag indicates whether the publications cache
> is valid or not; the flag is set to false for any invalidation in the
> pg_publication catalog. I wonder that instead of using the same flag
> what if we use a separate publications_skipped flag? If that works,
> you don't even need to change the current location where we
> LoadPublications.

There is almost negligible dip with the above suggested way, the test
results for the same is given below(execution time is in milli
seconds):
Brach/records  |  100     |  1000   |  10000    |  100000 |  1000000
Head               |   10.25  |  15.85  |   65.53    |  569.15  |  9194.19
Patch              |   10.25  |  15.84  |   65.91    |  571.75  |  9208.66
% diff              |      0.00 |    0.06  |    -0.58    |     -0.46  |    -0.16

There is a performance dip in the range of 0 to 0.58 percent.
The attached patch has the changes for the same. The test script used
is also attached.

Regards,
Vignesh