On Aug 17, 2008, at 10:21 AM, Madison Kelly wrote:
> Truth be told, I sort of expected this would be what I had to do. I
> think I asked this more in hoping that there might be some "magic"
> I didn't know about, but I see now that's not the case. :)
>
> As my data points grow to 500,000+, the time it took to return
> these results grew to well over 10 minutes on a decent server and
> the DB size was growing rapidly, as you spoke of.
>
> So I did just as you suggested and took the variable names I knew
> about specifically and created a table for them. These are the ones
> that are being most often updated (hourly per customer) and made
> each column an 'int' or 'real' where possible and ditched the
> tracking of the adding/modifying user and time stamp. I added those
> out of habit, more than anything. This data will always come from a
> system app though, so...
>
> Given that my DB is in development and how very long and intensive
> it would have been to pull out the existing data, I have started
> over and am now gathering new data. In a week or so I should have
> the same amount of data as I had before and I will be able to do a
> closer comparison test.
>
> However, I already suspect the growth of the database will be
> substantially slower and the queries will return substantially faster.
I strongly recommend you also re-think using EAV at all for this. It
plain and simple does not scale well. I won't go so far as to say it
can never be used (we're actually working on one right now, but it
will only be used to occasionally pull up single entities), but you
have to be really careful with it. I don't see it working very well
for what it sounds like you're trying to do.
--
Decibel!, aka Jim C. Nasby, Database Architect decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828