I have a problem that I'm hoping someone can help me with. I've built a
web-pased application that is back-ended by PGsql.. It works great, I
couldn't ask for better preformance.
However, I just recently coded another module for it and this module needed
to accept text files (or just large text blocks lets say). I ran into the 8k
query limit barrier. I need these blocks of text to be searchable and am
having to do this with a flat-file based solution right now, it's slow and
cumbersome. I would much rather have the text stored in a PG field so I
don't have to stray from the design of the search engine (which is designed
to search only the pg database, but has been hacked to search flat files
too).. I could break the texts up but they really all need to be in a single
field, not broken up. I can't think up a way to do it with the 8k query
limit.
Another problem I see is that well, there are a lot of text files (4000 now,
with about 500 added a week), many over 8k in size. Would this be too much
for PG to handle saying that I could get all the text inserted?
Also feel free to suggest some kind of indexing system for the flat files,
the only problem with that will be integration into my existing application
but I've left some hooks open so I can deal with it..
Thanks guys!
-Mitch