The Hermit Hacker <scrappy@hub.org> writes:
> What do you mean by "fulltext searching"?
He's talking about inverted text indices, where text is indexed such
that a word is the key, and the index returns pointers to all the
places where that word occurs. Knowledge of word structure is usually
built in, so that "hacks", "hacker", "hackers", "hacking" and so on
are known to be derivatives of "hack", and can match it if requested.
Noise words such as "a", "the" and so forth are usually not indexed.
Inverted indexed text storage tends to take up much space, but there
are ways to reduce this, and the best implementations do it remarkably
well. A simple example: it is not really necessary to actually store
the original text; it can instead be a sequence of links to the store
of all individual words in the text database.
See http://glimpse.cs.arizona.edu/ for a powerful inverted indexing
engine and various related software.
-tih
--
Popularity is the hallmark of mediocrity. --Niles Crane, "Frasier"