Hi eveyone,
I've been trying to understand the text search parser's behaviour.
Looking at the source code [1] it seems as if there was a
sophisticated FSM mapping the input string to a list of tuples of
category (as defined in [1], lines 32-56, or [2]) and a substring from
the original one order by the appearance of the latter one in the
original.
* Is there any documentation to be found on this parser?
As the parser is not aware of the underlying language, I would like to
create my own one.
* Is adding one to pg_ts_parsers the right way or should this rather
be done outside of the PG internals?
* For the first case, is there any manual or documentation how to do so?
If you want to comprehend my aims, try these commands:
select (ts_parse(3722,s)).*, (ts_debug(s)).*, (ts_debug('french',s)).*
from (select 'aujourd''hui ils m''ont dit qu''il y aura peut-être plus
de 10 000 personnes'::text s) x;
select (ts_parse(3722,s)).*, (ts_debug(s)).* from (select 'heu
d''anar-hi'::text s) x;
Best
Johannes
[1] http://doxygen.postgresql.org/wparser__def_8c_source.html
[2] http://www.postgresql.org/docs/9.3/static/textsearch-parsers.html