I'm writing a python script that (among other things) caches websites.
Ultimately, the data is all stored in a string (pickled, possibly
zipped). (lots of related stuff in stored in postgresql tables).
I am wondering whether it would be better to store each website in a
record in a table, or instead have a table which links URLs to filenames
(the file would contain the pickled website). The sites will of course
vary greatly in size, but typically be between 1k and 200k (I probably
won't store anything bigger than that).
This seems like a simple question, and I suspect there's an obvious
answer for which data storage method makes more sense, I just don't know
how to go about researching that. What would be the considerations for
using one method of data storage vs. the other?
Any suggestions for me?
--Adam