Sorry, I lost original post so replying several quote level deep.
On 18 Jun 2003 at 13:22, Matthew Nuzum wrote:
> > Where would the 400-4000GB of data go in this setup? On all the distinct
> > postgresql-servers? On a single SAN/NAS? (Postgresql doesn't really work
> > with that, does it? At least not in a loadbalancing setup).
> > And why can't a scriptinglanguage not be used with loadbalancing? I know
> > it is hard or impossible to get a connectionpool in such setups, but
> > that doesn't mean they can't be used with loadbalancing...
May be this is another approach, may be dumb but consider.
The data is read-only archive data. Not likely to be changed once loaded.
Besides this is not OLTP style stuff where heavy concurrent transactions are
involved.
I suggest that keep BLOBs in file system rather than in database. I know it is
already suggested but just ealborating on that.
There is no need to replicate huge amount of data. The app can store file name
and checksum in database. Once loading is complete and correct, an archive copy
of data can be kept offline for recovery purpose.
The app. can retrieve data and file separately and checksum it. May be it would
need a small kind of caching layer but should be fairly trivial.
Once that is done, the rest of the actual database would be pretty small and
could be replicated if required. The database can be hosted on a small-medium
server and BLOBS can be sent to SAN/NAS.
Data loading need to be verified but that is a small cost to pay for advantages
this method offers.
Just a thought..
Bye
Shridhar
--
No one wants war. -- Kirk, "Errand of Mercy", stardate 3201.7