Here are patches to add support for Unicode normalization into SQL, per
SQL standard:
normalize($string [, form])
$string is [form] normalized
(comment about silly SQL syntax here)
We already have all the infrastructure for Unicode normalization for the
SASLprep functionality. The first patch extends the internal APIs to
support all four normal forms instead of only NFKC used by SASLprep.
The second patch adds the SQL layer on top of it.
This could be used to preprocess or check strings before using them with
deterministic collations or locale implementations that don't deal with
non-NFC data correctly, perhaps using triggers, generated columns, or
domains. The NFKC and NFKD normalizations could also be used for
general data cleaning, similar to what SASLprep does.
As a future idea, I think we could also hook Unicode normalization into
the protocol-level encoding conversion.
Also, there is a way to optimize the "is normalized" test for common
cases, described in UTR #15. For that we'll need an additional data
file from Unicode. In order to simplify that, I would like my patch
"Add support for automatically updating Unicode derived files"
integrated first.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services