Re: [PATCH] Completed unaccent dictionary with many missing characters

Peter Eisentraut wrote on 5/4/2022 5:17 PM:

On 28.04.22 18:50, Przemysław Sztoch wrote:

Current unnaccent dictionary does not include many popular numeric symbols,
in example: "m²" -> "m2"

Seems reasonable.

Can you explain what your patch does to achieve this?

I used an existing python implementation of the generator.
It is based on ready-made unicode dictionary: src/common/unicode/UnicodeData.txt.
The current generator was filtering UnicodeData.txt too much.
I relaxed these conditions, because the previous implementation focused only on selected character types.

Browsing the unaccent.rules file is the easiest way to see how many and what missing characters have been completed.

For FTS, the addition of these characters is very much needed.