Обсуждение: [9.3 CF 1] 2 Weeks In & The problem with Performance Patches
Folks, We are starting the 3rd week of the commitfest tommorrow, and here's the update on status. In the last week, we are at the following status levels: - 40 patches still Need Review, including 11 which have no reviewer assigned. - 23 patches are Waiting on Author, most of thembecause they are still under discussion on this list. - 17 patches are marked Ready for Committer,including 10 of the 11 regression tests* - 17 patches have been Committed - 9 patches have been Returned with Feedback - 1 patch has been Rejected (* the tests actually need a test of the cumulative total time to run, which I'll be working on today) At current commit/return rates, this commifest will finish around July 28th. What seems prepared to drag out this commitfest possibly beyond the end of July are the several performance optimization patches, which include: - Remove PD_ALL_VISIBLE** - Improvement of checkpoint IO scheduler for stable transaction responses - Use posix_fallocate to build (new) WAL files** - Performance Improvement by reducing WAL for Update Operation** - GIN improvements (4 patches)** - XLogInsert scaling Items marked ** are waiting on review. The problem with these patches is that (a) people feel reluctant to review them, since not too many hackers feel qualified to review performance, and (b) the review will need to include performance testing, which will make it take longer and likely bottleneck on qualified testers. So, two things: (1) if you are qualified to review any of the above, please get started *now*, and leave the non-performance patches for later; (2) ideas on how we can speed up/parallelize performance testing efforts are extremely welcome. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
On 2013-06-28 12:16:09 -0700, Josh Berkus wrote: > - Remove PD_ALL_VISIBLE** I think this is primarily delayed because people don't agree it's a good idea. > - Use posix_fallocate to build (new) WAL files** I don't think this needs much further testing. We should just remove the enable/disable knob and go ahead. > - XLogInsert scaling I don't see to many problems here so far. For a complex feature it's imo already in a pretty good shape. Some more eyes on the code would be good, given it touching lots of code that's central for crash safety, but if not, I don't think it will block things. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On Fri, Jun 28, 2013 at 4:16 PM, Josh Berkus <josh@agliodbs.com> wrote: > (2) ideas on how we can speed up/parallelize performance testing efforts > are extremely welcome. An official perf-test script in GIT, even if it only tests general pg-bench-like performance, that can take two builds and compare them, like unladen-swallow's perf.py[0][1], would enable regular folk perform the testing on various hardware configurations and contribute their results more easily. Results would be in standardized format, which would help on the interpretation of those numbers, and patches would also be able to add patch-specific tests to the script's battery. I had a bash script that ran a few tests I used when evaluating the readahead patch. It's very very green, and very very obvious, so I doubt it'd be useful, but if noone else has one, I could dig through my backups. The test handled the odd stuff about clearing the OS cache between test runs, and stuff like that, which is required for meaningful results. I think this is where an official perf test script can do wonders: accumulate and share knowledge on testing methodology. [0] http://code.google.com/p/unladen-swallow/wiki/Benchmarks [1] http://code.google.com/p/unladen-swallow/source/browse/tests/perf.py