"Mitch Vincent" <mitch@venux.net> writes:
> Also feel free to suggest some kind of indexing system for the flat files,
Glimpse, http://glimpse.cs.arizona.edu/
If you are trying for a full-text index over a large amount of text,
Postgres is really not the right tool --- we don't have the right kind
of index support for searching for individual words/phrases within text
fields. You can fake it with LIKE or regexp matches but performance
will be awful, because the searches will have to scan the whole database.
(There is code that partially addresses this in contrib/fulltextindex,
but performance will still be nothing to write home about.)
Much better to use something that's actually designed for the purpose.
Glimpse is a really nice tool: speedy, and its index files are quite
small. I've used it for years to keep full-text indexes of all my
files.
regards, tom lane