Simon Riggs wrote:
>>Joe Conway writes
>>I'm seeing the following errors after a few hours of fairly aggressive
>>bulk load of a database running on Postgres 7.4.2:
>
>>When I say aggressive, I mean up to 6 simultaneous COPY processes. It is
>>different from the issue Tom solved the other day in that we don't get
>>SIGABORT, just corrupt index pages.
>
> OK, problem accepted, but why would you run 6 simultaneous COPYs? Presumably
> on > 1 CPU? Sounds like you're hitting the right edge of the index really
> hard (as well as finding a hole in the logic).
This is fairly high end hardware -- 4 hyperthreaded CPUs (hence 8 CPUs
from the OS perspective), 8GB RAM.
But in any case, since last report we've reproduced the problem with a
single COPY at a time.
> Can I ask, are you also running simultaneous INSERTs or just COPYs? And
> presumably you're mixing that with SELECTs that are doing index scans? Does
> the table have random deletes on it, or just occaisional regular bulk
> deletes? How often is it VACUUMed?
This is an initial data load, coming from an export from a large
cOmmeRciAl database. No other activity other than loading, no inserts.
The total rows to be imported into Postgres is ~900 million -- that's
why we would like to do as much in parallel as possible.
Joe