On Tue, 17 Jun 2008, Scott Marlowe wrote:
> We had a reporting server with about 80G of data on a machine with 4G
> ram last place I worked, and it could take it a few extra seconds to
> hit the old data, but the SW RAID-10 on it made it much faster at
> reporting than it would have been with a single disk.
I agree with your statement above, that query time could likely be dropped
a few seconds with a better disk setup. I just question whether that's
necessary given the performance target here.
Right now the app is running on an underpowered Windows box and is
returning results in around 10s, on a sample data set that sounds like 1/8
of a year worth of data (1/40 of the total). It is seemingly CPU bound
with not enough processor to handle concurrent queries being the source of
the worst-case behavior. The target is keeping that <30s on more powerful
hardware, with at least 6X as much processor power and a more efficient
OS, while using yearly partitions to keep the amount of data to juggle at
once under control. That seems reasonable to me, and while better disks
would be nice I don't see any evidence they're really needed here. This
application sounds a batch processing/reporting one where plus or minus a
few seconds doesn't have a lot of business value.
--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD