On Thu, 5 Nov 1998, David Ben-Yaacov wrote:
> Even under normal operations when the database is fast, we still have
> problems inserting data. We have examined the insert query identified
> in our error log to see if there is a problem with our code.
> Apparently not, as the insert query that failed while running the
> ingest process has no problems when entered manually. Anyone have any
> thoughts??
Shared memory corruption?
> You may be wondering what kind of data we store. The system is a
> development system utilizing a real-time alphanumeric weather feed.
> So if we drop the tables and thus the weather, we can always wait a
> couple of hours for it to reload. On our production/deployed systems,
> this is not an option.
Wow...how many records per second are being ingested?
> As to the wise suggestions of Mr. Fournier and others, we have
> adequate RAM, 256 MBytes, adequate CPU, 2 R10000s, adequate swap, 260
> Mbytes, and the postgres user located on a separate disk from the swap
> file. (SGI IRIX 6.5 Dual Processor Octane, Postgres 6.3 built using
> SGI's C compiler, Pg) (We tried building 6.3.2 using GNU's C compiler
> and SGI's C compiler but the problem appeared much worse; obviously we
> have not yet tried 6.4).
How hard is it for you to upgrade to 6.4? I'd be curious if the
problem does become worse, and, if so, if you have a little time, would
love to see if we can debug the reasons why...sounds like you guys would
make for a great test case,...
> As to running with the fsync disabled, we have tried both ways but
> usually disable fsync. The reason is perhaps humourous; I sit by the
> machine and it makes a lot of drive noise if we do not disable the
> fsync. Could that be a potential cause of our problem?
Shouldn't be...but I won't say impossible...
> Finally, if we lose the database connection when running our Perl
> ingest routines, we automatically try to reconnect to the database (as
> per Mr. Fournier's advice below).
> My question is this: Has anyone got some advice for Mr. Mackintosh or
> myself? Has anyone experienced this and found a solution?
First advice, move up to 6.4 ... if you can get it to run there,
even with the same problems, at least you are working with a much newer
set of features and fixes, and we should be able to direct you better in
how to debug it...the same applies to Terry, of course. Having two with
similar problem reports should be very helpful in debugging this...
Terry's might be easier to work with because he's running *cough*
*gack* Linux *choke* ... but "the more, the merrier" :)
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org