Re: Great Bridge benchmarks?

Andrew,

I found this in my email archive.  HTH...

Regards,
Ned



> Open source database routs competition in new benchmark tests
>
> PostgreSQL meets or exceeds speed and scalability of proprietary database leaders,
>
> and significantly surpasses open source competitors
>
>
> NORFOLK, Va, August 14, 2000 -PostgreSQL, the world's most advanced open source
> database, routed the competition in recent benchmark testing, topping the
> proprietary database leaders in industry-standard transaction-processing tests.
> PostgreSQL, also known as "Postgres," is an object-relational database management
> system (DBMS) that newly formed Great Bridge LLC will professionally market,
> service and support. Postgres also consistently outperformed open source
> competitors, including MySQL and Interbase, in the benchmark tests. Great Bridge
> will market Postgres-based open source solutions as a highly reliable and lower
> cost option for businesses seeking an alternative to proprietary databases.
>
> On the ANSI SQL Standard Scalable And Portable (AS3AP) benchmark, a rudimentary
> information retrieval test that measures raw speed and scalability, Postgres
> performed an average of four to five times faster than every other database
> tested, including two major proprietary DBMS packages, the MySQL open source
> database, and Interbase, a formerly proprietary product which was recently made
> open source by Inprise/Borland. (See Exhibit 1)
>
> In the Transaction Processing Council's TPC-C test, which simulates a real-world
> online transaction processing (OLTP) environment, Postgres consistently matched
> the performance of the two leading proprietary database applications. (See Exhibit
> 2)  The two industry leaders cannot be mentioned by name because their restrictive
> licensing agreements prohibit anyone who buys their closed-source products from
> publishing their company names in benchmark testing results without the companies'
> prior approval.
>
> "The test results show that Postgres is a robust, well-built product that must be
> considered in the same category as enterprise-level competition," said Robert
> Gilbert, Great Bridge President and CEO. "Look at the trendlines in the AS3AP
> test:  Postgres, like the proprietary leaders, kept a relatively consistent output
> level all the way up to 100 concurrent users - and that output was four to five
> times faster than the proprietary products.  Interbase and MySQL fell apart under
> heavy usage.  That's a strong affirmation that Postgres today is a viable
> alternative to the market-leading proprietary databases in terms of performance
> and scalability-and the clear leader among open source databases."
>
> The tests were conducted by Xperts Inc. of Richmond, Virginia, an independent
> technology solutions company, using Quest Software's Benchmark Factory
> application.  Both the AS3AP and the TPC-C benchmarks simulated transactions by
> one to 100 simultaneous users in a client-server environment. One hundred
> concurrent users approximates the middle range of a traditional database user
> pool; many applications never see more than a few users on the system at any given
> time, while other more sophisticated enterprise platforms number concurrent users
> in the thousands.  In a Web-based application, where the connection to the
> database is measured in milliseconds, 100 simultaneous users would represent a
> substantial load-the equivalent of 100 customers hitting the "submit" button on an
> order form at exactly the same time.
>
> The AS3AP test measures raw database data retrieval power, showing an
> application's scalability, portability and ease of use and interpretation through
> the use of simple ANSI standard SQL queries.  The TPC-C test simulates a warehouse
> distribution system, including order creation, customer payments, order status
> checking, delivery, and inventory management.
>
> "What stood out for us was the consistent performance of Postgres, which stayed
> the same or tested better than those of the leading proprietary applications.
> Postgres performed consistently whether it was being used by one or 100 people,"
> said Richard Brosnahan, senior software developer at Xperts.
>
> Postgres is a standards-based object-relational SQL database designed for
> e-business and enterprise applications. The software is open source and freely
> owned, continuously augmented by a global collaborative community of elite
> programmers who volunteer their time and expertise to improve the product. In the
> last two years, with the introduction of versions 6.5 and 7.0 of the software,
> Postgres  has seen rapid enhancement through a series of high-level refinements.
>
> "Postgres' performance is a powerful affirmation of the open source method of
> development," said Gilbert of Great Bridge. "Hundreds, even thousands, of open
> source developers work on this software, demonstrating a rate of innovation and
> improvement that the proprietary competition simply can't match.  And it's only
> going to get better."
>
> A closer look
>
> Xperts ran the benchmark tests on Compaq Proliant ML350 servers with 512 mb of RAM
> and two 18.2 Gb hard disks, equipped with Intel Pentium III processors and Red Hat
> Linux 6.1 and Windows NT operating systems.  The company ensured the tests'
> consistency by using the same computers for each test, with each product
> connecting to the tests through its own preferred ODBC driver.  While Benchmark
> Factory does provide native drivers for some commercial databases, using each
> product's own ODBC ensured the most valid "apples to apples" comparison.
>
> In the AS3AP tests, PostgreSQL 7.0 significantly outperformed both the leading
> commercial and open source applications in speed and scalability.  In the tested
> configuration, Postgres peaked at 1127.8 transactions per second with five users,
> and still processed at a steady rate of 1070.4 with 100 users. The proprietary
> leader also performed consistently, with a high of 314.15 transactions per second
> with eight users, which fell slightly to 288.37 transactions per second with 100
> users. The other leading proprietary database also demonstrated consistency,
> running at 200.21 transactions per second with six users and 197.4 with 100.
>
> The other databases tested against the AS3AP benchmarks, open source competitors
> MySQL 3.22 and Interbase 6.0, demonstrated some speed with a low number of users
> but a distinct lack of scalability. MySQL reached a peak of 803.48 with two users,
> but its performance fell precipitously under the stress of additional users to a
> rate of 117.87 transactions per second with 100 users. Similarly, Interbase
> reached 424 transactions per second with four users, but its performance declined
> steadily with additional users, dropping off to 146.86 transactions per second
> with 100 users.
>
> "It's just astounding, and unexpected," said Xperts' Brosnahan of Postgres'
> performance. "I ran the test twice to make sure it was running right. Postgres is
> just a really powerful database."
>
> In the TPC-C tests, Postgres performed neck and neck with the two leading
> proprietary databases.  The test simultaneously runs five different types of
> simulated transactions; the attached graph of test results (Exhibit 2) shows
>
> steadily ascending intertwined lines representing all three databases, suggesting
> the applications scaled at comparable rates. With all five transactions running
> with 100 users, the three databases performed at a rate of slightly above five
> transactions per second.
>
> "The TPC-C is a challenging test with five transactions running at once while
> querying against the database and the stress of a growing number of users. It
> showed that all the databases we tested handle higher loads very well, the way
> they should," Brosnahan explained.
>
> Neither Interbase nor MySQL could be tested for TPC-C benchmarks. MySQL could not
> run the test because the application is not adequately compliant with minimal ANSI
> SQL standards set in 1992. Interbase 6.0, recently released as open source, does
> not have a stable ODBC driver yet; while Xperts was able to adapt the version 5
> ODBC driver for the AS3AP tests, the TPC-C test would not run.  "With MySQL it's
> an inherent design issue. Interbase 6 should run the TPC-C test, and perhaps would
> with tweaking of the test's code," said Brosnahan.
>
> Great Bridge's Gilbert attributes Postgres' high performance to a quality
> differential that comes from the open source development process; the source code
> for Postgres has been subjected to years of rigorous peer review by some of the
> best programmers in the world, many of whom use the product in their work
> environments.  "Great Bridge believes that Postgres is by far the most robust open
> source database available.  These tests provide strong affirmation of that
> belief," he said.  The company intends to work with hardware vendors and other
> interested parties to continue larger-scale testing of Postgres and other leading
> open source technologies.
>
> About Great Bridge
>
> Great Bridge LLC provides open source solutions powered by PostgreSQL, the world's
> most advanced open source database.  Great Bridge delivers value-added open source
> software and support services based on PostgreSQL, empowering e-business builders
> with an enterprise-class database and tools at a fraction of the cost of closed,
> proprietary alternatives.
>
> Headquartered in Norfolk, Virginia, Great Bridge is a privately held company
> funded by Landmark Communications, Inc., the media company that also owns The
> Weather Channel, weather.com, and national and international interests in
> newspapers, broadcasting, electronic publishing, and interactive media.
>
> # # #
>
>
>


--------------------------------------------------------------------------------






--------------------------------------------------------------------------------