On Fri, Nov 12, 2021 at 12:09 PM Robert Haas <robertmhaas@gmail.com> wrote:
> On Thu, Nov 11, 2021 at 5:04 PM Isaac Morland <isaac.morland@gmail.com> wrote:
> > Wouldn't an existing index only have characters that were already part of the collation? Attempting to use one not
coveredby the collation I would have expected to cause an error at insert time. But definitely I agree I wouldn't feel
confidentabout the safety of any change.
>
> I mean it's not like we are updating the definition of
> pg_utf8_verifychar() every time they define a new code point.
Right, and there may be other systems that do this. That is, reject
invalid code points, because they have no sort order. You can see
some sign of this in the major and minor collation version numbers
reported by Windows (though I'm not sure if this was lost with the
recent move to ICU): if only the minor version changes, the
documentation says it means "we only added new code points, no
existing code points changed", so a sufficiently clever program
doesn't need to rebuild its persistent ordered data structures, if it
never allowed any unknown code points into the structure before. Not
only does PostgreSQL not have the logic for that, it also doesn't have
the data: the set of existing code points has to be the one used by
the collation provider, and the collection providers we have don't
reject unknown code points on comparison.