Re: RAID arrays and performance

Matthew Wakeling <matthew@flymine.org> writes:

In order to improve the performance, I made the system look ahead in the 
source, in groups of a thousand entries, so instead of running:
SELECT * FROM table WHERE field = 'something';
a thousand times, we now run:
SELECT * FROM table WHERE field IN ('something', 'something else'...);
with a thousand things in the IN. Very simple query. It does run faster 
than the individual queries, but it still takes quite a while. Here is an 
example query:   

Have you considered temporary tables? Use COPY to put everything you want to query into a temporary table, then SELECT to join the results, and pull all of the results, doing additional processing (UPDATE) as you pull?

Cheers,
mark

-- 
Mark Mielke <mark@mielke.cc>