Re: internet week article

Eric Kolve wrote:
>   http://www.internetweek.com/reviews01/rev090301.htm
>
> I would be most interested to here this lists response to the following
> paragraph.  Though much of the specifics of the benchmark are left out,
> so tuning could very much of been an issue.
>
>   Database performance is a key issue for most businesses.
>   If the database is slow, the business is too. Our benchmarks
>   showed that PostgreSQL is slower than Oracle 8.1.7 in many
>   operations. The tests were performed on a 1-GHz Pentium III
>   system with 1 GB of RAM running Red Hat 7.1.
>
>   Oracle performed the count 33 percent faster and the distinct
>   count 27 percent faster. The simple query was approximately
>   18 percent faster in Oracle, but the most concerning was the
>   slowdown of PostgreSQL in the two- and three-table join test.

    We  are  aware  of the fact, that MVCC and not beeing able to
    shortcut aggregates have an impact on  things  like  count().
    Would  of  course  be  interesting  to get more details (full
    schema, queries and if DB was vacuumed properly).

    OTOH  this  kinda  stuff  is  IMHO  totally  braindead.   All
    benchmarks  from  the  Transaction  Processing  Council (TPC)
    define a system under test (SUT) and the values you get  from
    them  are  maximum troughput and cost per transaction for the
    given  implementation/setup.  And  there  are  "really  good"
    reasons behind it.

    1.  Why defining a SUT?

        The  raw  database speed itself on two or three different
        queries is meaningless.  Nobody  uses  SQL  as  the  user
        interface in the real world. The entire business logic is
        allways implemented in an application  AND  the  database
        together,  with  amounts varying (some TPC specs allow to
        implement a given  feature  of  the  SUT  either  in  the
        application  or on the DB side as stored procedure etc.).

        You're interested in how fast the user gets the  response
        for  his  form.  The  functionality  it  provides  can be
        implemented in a couple of different ways,  depending  on
        the  features  provided  by  the  middleware  and/or  the
        database, why  a  simple  comparision  of  "how  fast  is
        count(*)" is nonsense.

    2.  When to look at maximum throughput

        If  you are looking for speed, only speed and nothing but
        speed, you want to know  how  high  you  can  scale  your
        system.  The cost in that case is not really relevant, so
        you're happy if you find a vendor  selling  you  hardware
        for  a  couple  million bucks and a DB vendor you can pay
        another million or two  every  year,  as  long  as  these
        products  get  the  job  done  in  time  (budget  doesn't
        matter).

        You  want  to  know  "does  this   combo   handle   5,000
        simultaneous  users  with  7.5  terabytes of data". If it
        doesn't, it is  irrelevant  if  you  can  afford  it,  it
        doesn't fit your needs anyway.

    3.  When to look at cost per transaction

        If  you want to get your job done for a reasonable price,
        you're looking for a good cost per  transaction.  In  the
        TPC  specs, the full disclosure report has to include OS,
        DB and Middleware license fees as well  as  the  hardware
        and   maintainence  costs,  plus  average  administration
        expenses and support contracts. So it's an estimation  of
        the total cost to run THIS setup in production.

        You  want to know if you can afford the installation. The
        question is  "does  it  get  my  job  done  in  time  and
        budget?".

    4.  Why the truth is in between

        If  you  visit  www.tpc.org, you can find a collection of
        presentation slides which include a two dimensional chart
        with  dot's  of  full  disclosure  reports  for the TPC/C
        benchmark. These dot's mostly show that MS/SQL server  is
        usually  cheaper  than Oracle on various platforms, while
        Oracle dominates the high-end troughput area (no dot's at
        all  for  MS there - so that OS/DB simply sucks if you're
        interested in speed only).

        You want to know "does it get my job  done  in  time  and
        budget  NOW?". But you also want to know if it'll get the
        job done in the future, when your business  increased  as
        well as your budget.

    At  some point in the future, your business might outgrow the
    capabilities of your today choice. At that time, you'll  have
    to  migrate  (back) to the high-end solution and that's why I
    (personally) accept Oracle  compatibility  ugly-kludges  over
    SQL92 evangelism sometimes.

    The bottom line is, that if you just throw half of the Oracle
    license  fees  into  a  hardware  upgrade,   I'm   sure   the
    "benchmark"  you're talking about will show totally different
    numbers, right? So what does that  "benchmark"  really  tell?
    For  me  it  tells  "if  you  spend more money you get better
    performance". Well, that is a general rule of  thumb  we  all
    knew already, tell me something new.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com