Обсуждение: SSI update
Circumstances have conspired to leave me with very little time to work on the SSI patch during the last few weeks. I'm still convinced that the work mentioned in this post is necessary to have a commit-quality patch: http://archives.postgresql.org/pgsql-hackers/2010-10/msg01754.php I also think it's going to be very desirable to convert the conflict pointers used in the papers (and in the patch so far) to lists, to eliminate one source of false positives and allow more aggressive clean-up of transactions. I we implement the logic from the above post and *then* convert the pointers to lists, it seems like more work than implementing the lists first. Therefore, I'm planning on doing the lists first. Anyone who thinks that's a bad idea, please speak up soon, as I'm starting coding on that today. There's no way there will be a patch implementing this in time for the 2010-11 CF. A couple months ago there were a couple people who said they'd be willing to look at this between CFs, so I'm hoping that their schedules still permit them to do that after this unfortunate delay. -Kevin
That it's not ready for commit this minute does not mean that it shouldn't be in the CF this month. Delaying the first review of the patch until the next CF pretty much ensures that we'll miss 9.1 with it, so please add to the current CF :) Cheers, David. On Sat, Nov 13, 2010 at 02:15:40PM -0600, Kevin Grittner wrote: > Circumstances have conspired to leave me with very little time to > work on the SSI patch during the last few weeks. I'm still convinced > that the work mentioned in this post is necessary to have a > commit-quality patch: > > http://archives.postgresql.org/pgsql-hackers/2010-10/msg01754.php > > I also think it's going to be very desirable to convert the conflict > pointers used in the papers (and in the patch so far) to lists, to > eliminate one source of false positives and allow more aggressive > clean-up of transactions. I we implement the logic from the above > post and *then* convert the pointers to lists, it seems like more > work than implementing the lists first. Therefore, I'm planning on > doing the lists first. > > Anyone who thinks that's a bad idea, please speak up soon, as I'm > starting coding on that today. > > There's no way there will be a patch implementing this in time for > the 2010-11 CF. A couple months ago there were a couple people who > said they'd be willing to look at this between CFs, so I'm hoping > that their schedules still permit them to do that after this > unfortunate delay. > > -Kevin > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
David Fetter <david@fetter.org> wrote: > That it's not ready for commit this minute does not mean that it > shouldn't be in the CF this month. Delaying the first review of > the patch until the next CF pretty much ensures that we'll miss > 9.1 with it, so please add to the current CF :) Well, I had a version which compiled without warning and passed the regular regression tests before the deadline, but it would have been rather disingenuous to post it. I'm in the middle of reworking the guts of it and there's a lot of wet paint at the moment. Most of the dcheck tests are failing, which isn't a surprise because I haven't finished re-implementing everything which was working before this weekend. I'm not sure what a reviewer could say right now other than to point out all the loose ends fluttering in the breeze. To be clear -- before this weekend I had something which worked correctly in all respects except, as pointed out by Heikki and Jeff, it was vulnerable to filling its in-memory structures if there was a long-running transaction concurrent with a lot of short-lived transactions. At that point the only options, as the patch stood, were to refuse to start new serializable transactions or to start killing off the oldest active serializable transactions. This issue has been on the Wiki page as an R&D "What should we do about this?" item since the 25th of January, but it took the discussion around the review in the last CF to lead to a design for a solution. Then my father landed in the hospital after a cardiac arrest, an in-law died, and I got a nasty toothache which ultimately needed a root canal last week. On top of that, I've been assigned some high priority tasks at work which relegate work on this patch to nights and weekends for the time being. Dan has also been unavailable, although for happier reasons -- he got married and took a vacation. Overall, though, a tough few weeks for making progress on implementing the designed solution. It's likely to be two to four weeks before I have something in a condition which it would make sense for anyone to spend time on. Anyone who wants to watch the progress is welcome to track the git repo. I pushed this weekend's work in this commit, wet paint and all: http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git;a=commitdiff;h=208ade2acb6c34177c4bfd49e1f240fb751b8be1 That went in an hour and a half before the CF deadline, but I just didn't feel right putting it into the CF in that shape. I'm most of the way through replacing the conflict pointers with conflict lists, which I decided was a prerequisite for solving the memory management issues in an effective way. I have yet to start work on the memory management issues themselves. -Kevin
Kevin Grittner wrote: > That went in an hour and a half before the CF deadline, but I just > didn't feel right putting it into the CF in that shape. Then keep on working on it and we can revisit its state when you're happy with it. The purpose of the CommitFest cut-off is not to block work on long-term development just because a deadline passed; it's to make sure patches which might otherwise never get a review are looked at eventually. Finding a reviewer for these larger and complicated patches is a very different sort of job than finding one for an average patch anyway. I could use a brief reminder of how this bit fits into the "serializable lock consistency" patch that's already sitting into the CF queue as "Ready for Committer" though. -- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services and Support www.2ndQuadrant.us
Excerpts from Greg Smith's message of lun nov 15 12:32:25 -0300 2010: > I could use a brief reminder of how this bit fits into the "serializable > lock consistency" patch that's already sitting into the CF queue as > "Ready for Committer" though. I remind you, though, that the most painful point that caused the commitfest process to come into existance was the HOT patch, which was very large and intrusive and didn't get any review until very late in the cycle. So getting some review earlier than at the last minute is important. -- Álvaro Herrera <alvherre@commandprompt.com> The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Greg Smith <greg@2ndquadrant.com> wrote: > I could use a brief reminder of how this bit fits into the > "serializable lock consistency" patch that's already sitting into > the CF queue as "Ready for Committer" though. Florian's patch prevents an "integrity inversion" that PostgreSQL has had for ages, where attempting to prevent serialization anomalies under snapshot isolation through SELECT FOR UPDATE/SHARED gave you less protection than in the less strict levels. This has been a particular nuisance to those trying to convert from Oracle, where using the more strict isolation level always gives *more* integrity protection, not less. That patch appears to strengthen PostgreSQL integrity guarantees around SELECT FOR UPDATE/SHARED to match Oracle. One difference between that and the SSI patch is that with Florian's patch you still need to recognize potential interactions which could generate anomalies and explicitly code to prevent them, while in SSI this is sorted out by the database engine automatically at run time. Another difference is that Florian's patch introduces blocking to protect integrity, while SSI does not. There are definitely use cases for both. In general, SSI is a "big shop" oriented solution, while the explicit coding may be more appropriate in many environments where you have just a few programmers working with dozens or hundreds of database transaction types rather than thousands. The latter is also likely to be a popular choice for those converting from other databases, even for big shops, where they already have a working scheme using explicit locking. -Kevin
On Mon, Nov 15, 2010 at 10:56 AM, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote: > Greg Smith <greg@2ndquadrant.com> wrote: > >> I could use a brief reminder of how this bit fits into the >> "serializable lock consistency" patch that's already sitting into >> the CF queue as "Ready for Committer" though. > > [explanation] The short version is that they are independent. The serializable lock consistency patch, I think, a very important one, although I am not entirely sure that I have the skill to do it justice. (I'm kinda hoping Tom will commit it, but If he doesn't have time I may have to take a crack at it.) -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Alvaro Herrera <alvherre@commandprompt.com> wrote: > I remind you, though, that the most painful point that caused the > commitfest process to come into existance was the HOT patch, which > was very large and intrusive and didn't get any review until very > late in the cycle. So getting some review earlier than at the > last minute is important. I've tried to avoid that by keeping a Wiki up-to-date with design, status, and issues publicly available. I've been pushing the code to a public git repo as it's been developed, since January. I avoided discussing issues like the one causing the current refactoring on-list for several months because Tom complained that such discussions were distracting from the effort to get 9.0 out the door, but I submitted a WIP patch to the first 9.1 CF and a patch I had hopes of seeing committed to the second 9.1 CF. I've missed submitting to this CF, but it would be entirely wrong to say that the patch has had no review. Joe Conway did a round of review, although much of the discussion was off-list. This resulted in a commit of a portion of the patch which could be usefully split out. Heikki Linnakangas didn't officially sign up as a reviewer, but he posted a lot of questions and suggestions showing that he had looked at the code. Jeff Davis gave it an official review, ultimately resulting in a disposition of "Returned with Feedback". There is the one big issue of degrading gracefully in the face of a long-running transaction concurrent with many shorter ones, which I am attempting to address now. All of the above reviews have resulted in changes to the code which I feel are improvements on the initial effort. I've been doing everything I can thing of to avoid having this be a last minute submission while still "playing nice" with the community. Any suggestions on what else I can do to improve this are welcome. -Kevin