Re: [GENERAL] Re: [PHP3] Re: PostgreSQL vs Mysql comparison

I think that everyone is dancing all around the crux of the problem.  Benchmarks define a
set
of circumstances by which products are compared.  To define a problem space that does
not favor one system or the other is a non  trivial problem in and of itself. Real world
tasks
are complex and have many facets. So,  how do we define a single real world task that
represents a fair distribution and utilization of the features that are important in
multiple
problem spaces? I don't know, and haven't seen one yet.
We should  accept that one solution is better in a situation and another is better in
another but
understand the criteria.

Does this sound too much like ENGINEERING?
An understanding of what a particular benchmark is designed to illustrate is far more
important than
the results of a particular test.  Know YOUR problem space and pick a product that fits.

Tom

"Aaron J. Seigo" wrote:

> hi all...
>
> > Note that it would be much better to write a benchmark that can be run
> > against any SQL server.  As the MySQL benchmarks are designed to do
> > exactly this, I would suggest that you use the MySQL benchmark suit as
> > a base for this!
>
> i disagree heartily. standardized benchmarks such as these are rediculous wastes
> of time, especially when done with database engines. why?
>
> because, as has been shown time and time again on both sides of any fence you
> look at in our wonderful world of software, benchmarks usually favour one side
> or the other due to the fact that they are standard. well, software isn't
> standard. MySQL and PostgreSQL use different methods for handling queries,
> indecies, data, etc.. should we expect the exact same queries to perform equally
> well on both? no! it will most probably favour one or the other, depending how
> it is written. i'm not saying this is always intentional, its merely a fact
> about software and standardized benchmarking as i see it.
>
> what is better? well, what is software used for? to run arbitrary
> queries/functions? NO! to solve a problem, complete a task. SO. give me a task.
> allow me to use the software to complete that task. because in the real world,
> i don't write the same queries in the same way for two different database
> engines. i don't use X the same way i use Windows. i don't use GIMP the same
> way i use photoshop.
>
> benchmarks are a waste of time. problem solutions designec for the products at
> hand show the REAL potential of the systems and can't be screwed with. after
> all, all parties are going to use everything they can to make it fast. same
> results. different paths. different software. apples and apples.
>
> its not the method. its the answer to the question.
>
> > I don't think the important things is just to optimize some specific
> > queries; It's much more important to test a lot of different types of
> > queries.
>
> on this i agree. so, lets optimize all the queries (or not =), but leave it to
> the hackers at their discretion.
>
> > I think it's important to using queries that are common to many
> > applications.
> >
> > For normal queries there isn't that much to optimize.  There is of
> > course also the option to add some test to 'solve some problem'.  In
> > this case one can use different methods to solve the query for
> > different databases;  Normally these kind of tests are more
> > interesting than useful as this isn't normally portable between
> > database servers.
>
> portability is a non issue, in my mind. when i use a database engine, i use
> every nook and cranny of it when querying, etc to get performance. example:
> would you forgo (sp) using PL/SQL in Oracle because it isn't supported the
> exact same way everywhere (anywhere?) else? of course not. portability is a non
> issue. problem solution power when given to a skilled user is. or an unskilled
> user.
>
> i think such tests are more telling than benchmarks. benchmarks are
> interesting, but completely uninterpretable in a meaningful way (see above).
> problem solution capability shows the true power of a system, esp. if that
> system has unique ways of doing things that REALLY make a difference.
>
> > Aaron> this is open source, remember? its about choice. its about the right tool for
> > Aaron> the right job. its about a FUDless environment where we get the tools to do
> > Aaron> what we NEED to do.
> >
> > This is exactly the aim of the MySQL benchmarks.  (They are GPL)
>
> this (GPLing) is indeed applaudable!
>
> > Aaron> as coordinator, i'd be willing to collect the final parameters, communicate
> > Aaron> with representatives from each side (probably the developers?) and post the
> > Aaron> results (after running them, of course :-)
> >
> > In this case;  Can you take a look at the MySQL benchmark suite and
> > comment on this?
>
> i believe my comments above stand. i do not support benchmarking. it only works
> when you have two identical systems. but the point of such exercises is to
> differences between DIFFERENT systems. =)
>
> that said (again), i'll go grab the crash_me test suite and look it over, toss
> in my 2 cents, if i have that much at the end of the exercise =)
>
> --
> Aaron J. Seigo
> Sys Admin
>
> ************