On 4/2/2014 7:30 PM, Rob Sargent wrote:
>>
>>>
>>
> Well things slow down over time, and lots of "too frequent"s:
>
> Have done 500 batches in 24219 ms
> Have done 1000 batches in 52362 ms
> Have done 1500 batches in 82256 ms
> Have done 2000 batches in 113754 ms
> Have done 2500 batches in 149637 ms
> Have done 3000 batches in 211314 ms
> Have done 3500 batches in 301989 ms
> Have done 4000 batches in 430817 ms
> Have done 4500 batches in 596043 ms
> Have done 5000 batches in 804250 ms
>
> where a batch is 500,000 lines. This on the java side of course.
>
>
>
>
Have you watched cpu usage of the java process vs the PG process in top?
I had a perl program importing data that was the bottleneck because it
was calling rand(). I'll bet generating uuid's is cpu intensive too.
Is the java app cpu bound?
Also watch "vmstat 3" for a minute or two. The last two numbers (wa &
id) (some vmstat's have a steal, ignore that) will tell you if you are
io bound.
-Andy