> "Chris Ochs" <chris@paymentonline.com> writes:
> > Is this a crazy way to handle this?
>
> Depends. Do you care if you lose that data (if the system crashes
> before your daemon can insert it into the database)? I think the
> majority of the win you are seeing comes from the fact that the data
> doesn't actually have to get to disk --- your "write to file" never
> gets further than kernel disk buffers in RAM.
>
> I would think that you could get essentially the same win by aggregating
> your database transactions into bigger ones. From a reliability point
> of view you're doing that anyway --- whatever work the daemon processes
> at a time is the real transaction size.
>
> regards, tom lane
>
The transactions are as big as they can be, all the data is committed at
once. I'm guessing that for any database to be as fast as I want it, it
just needs bigger/better hardware, which isnt' an option at the moment.
I was also thinking about data loss with the disk queue. Right now it's
such a small risk, but as we do more transactions it gets bigger. So right
now yes it's an acceptable risk given the chance of it happening and what a
worst case scenario would look like. but at a point it wouldnt' be.
Chris