[Chris - Fri at 02:32:05PM +1100]
> Not really. A bad query is a bad query (eg missing a join element). It
> won't show up for 3000 rows, but will very quickly if you increase that
> by a reasonable amount. Even as simple as a missing index on a join
> column won't show up for a small dataset but will for a larger one.
Ok, you're talking about O(n^2) and such stuff :-)
> It's a pretty common mistake to assume that a small dataset will behave
> exactly the same as a larger one - not always the case.
No. :-) We had the worst experience when launching our product - it had
been stress tested, but only by increasing the number of customers, not
by increasing the overall size of the data set available for browsing.
When opening the web pages for the public, this data set was already
some ten-hundred times bigger than in the version enduring the stress
tests - and the servers had no chances processing all the traffic.
The worst bottle neck was not the database this time, but some horror
algorithm one of the programmers had sneaked in ... poorly documented,
cryptically programmed, slow ... and since I never understood that
algorithm, I can only guess it must have been of O(n^2).