Re: tsearch stop words

On Mon, 2 Sep 2002, Christopher Kings-Lynne wrote:

> Ummm...I totally don't understand the format of that array at all!

stop words in suffix tree :) I've told it's default list of stop
words and it's certainly far from perfect. Please find attached file
with stop words we used (I just found on my notebook). But again,
it's better to build your own dictionary specific for you domain,
using makedict script.

OpenFTS is much more flexible in this respect and we hope we'd be able
to implement most features of OpenFTS in tsearch, so OpenFTS would be just
a perl wrapper.
Oleg


>
> static ESWNODE engstoptree[] = {
>         {'m',L,9,126},
>         {'d',L,4,71},
>         {'b',L,2,40},
>         {'a',F,0,14},
>         {'c',0,0,62},
>         {'f',L,2,79},
>         {'e',0,0,75},
>         {'h',0,1,90},
>         {'i',F,0,108},
>         {'t',L,4,177},
>         {'o',L,2,135},
>         {'n',0,0,131},
>         {'s',0,0,156},
>
> How do I figure out the actual word it's matching?
>
> Chris
>
>
> > -----Original Message-----
> > From: Oleg Bartunov [mailto:oleg@sai.msu.su]
> > Sent: Monday, 2 September 2002 5:33 PM
> > To: Christopher Kings-Lynne
> > Cc: Hackers
> > Subject: Re: [HACKERS] tsearch stop words
> >
> >
> > Christopher,
> >
> > current implementation is ugly, we still didn't move functionality
> > from OpenFTS to tsearch. Look at makedict subdirectory to create your
> > custom dictionary. Default list is in engstoptree[] defined
> > in dic/porter_english.dct
> >
> > On Mon, 2 Sep 2002, Christopher Kings-Lynne wrote:
> >
> > > How do I get a list of what tsearch considers a stop word?
> > >
> > > eg. 'and', 'or', 'the', 'up', 'down', etc. There seem to be heaps of
> > > them...!
> > >
> > > Chris
> > >
> > >
> > > ---------------------------(end of broadcast)---------------------------
> > > TIP 2: you can get off all lists at once with the unregister command
> > >     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> > >
> >
> >     Regards,
> >         Oleg
> > _____________________________________________________________
> > Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
> > Sternberg Astronomical Institute, Moscow University (Russia)
> > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> > phone: +007(095)939-16-83, +007(095)939-23-83
> >
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
>
Regards,    Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83