I'm writing a search engine using python and postgresql which requires to
store a temporary list of results in an sql table for each request. This
list will contain at least 50 records and could grow to about 300. My
options are either to pickle the list and store a single entry or use the
postgresql COPY command (as opposed to INSERT which would be too slow) to
store each of the temporary records.
Question is, how can I make an educated decision on which option to
select? What kind of questions should I be asking myself? Should I
actually go through the trouble of implementing both alternatives and
profiling each seperately? If so, how can I predict what will happen under
a heavy load which is hard to simulate when benchmarking each option?
Thanks