Обсуждение: best way to test new index?
Hello pgsql-hackers, what is the best way to test a new developed index structure? Greets, Yves
Yves Weißig<weissig@rbg.informatik.tu-darmstadt.de> wrote: > what is the best way to test a new developed index structure? Are you talking about a new AM (like btree, hash, GiST, or GIN)? -Kevin
Yes I do! Am 21.04.2011 20:56, schrieb Kevin Grittner: > Yves Weißig<weissig@rbg.informatik.tu-darmstadt.de> wrote: > >> what is the best way to test a new developed index structure? > > Are you talking about a new AM (like btree, hash, GiST, or GIN)? > > -Kevin >
Yves Weißig<weissig@rbg.informatik.tu-darmstadt.de> wrote: > Am 21.04.2011 20:56, schrieb Kevin Grittner: >> Yves Weißig<weissig@rbg.informatik.tu-darmstadt.de> wrote: >> >>> what is the best way to test a new developed index structure? >> >> Are you talking about a new AM (like btree, hash, GiST, or GIN)? > > Yes I do! The tests are going to depend somewhat on what the index is intended to do. That said, I would start by making sure that basic things like CREATE INDEX and DROP INDEX work. Then I would test that INSERT, UPDATE, and DELETE do the right things with the index. Then I would make sure that vacuum did the right thing. Then I would make sure that the optimizer was doing a reasonable job of knowing when it was usable and what it cost, so that it would be chosen when appropriate. Once everything seemed to be behaving I would benchmark it against the reasonable alternatives. -Kevin
On 4/21/11 1:28 PM, Kevin Grittner wrote: > That said, I would start by making sure that basic things like > CREATE INDEX and DROP INDEX work. Then I would test that INSERT, > UPDATE, and DELETE do the right things with the index. Then I would > make sure that vacuum did the right thing. Then I would make sure > that the optimizer was doing a reasonable job of knowing when it was > usable and what it cost, so that it would be chosen when > appropriate. Once everything seemed to be behaving I would > benchmark it against the reasonable alternatives. ... And then finally, kill -9 the database server while updating the index to test crash-safeness. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
Ok, but I thought more like an automated test, or a test which checks if the interface is correctly implemented. By the way, tho broaden the topic, what would be the best way to debug in pg? elog? asserts? Am 22.04.2011 01:26, schrieb Josh Berkus: > On 4/21/11 1:28 PM, Kevin Grittner wrote: >> That said, I would start by making sure that basic things like >> CREATE INDEX and DROP INDEX work. Then I would test that INSERT, >> UPDATE, and DELETE do the right things with the index. Then I would >> make sure that vacuum did the right thing. Then I would make sure >> that the optimizer was doing a reasonable job of knowing when it was >> usable and what it cost, so that it would be chosen when >> appropriate. Once everything seemed to be behaving I would >> benchmark it against the reasonable alternatives. > > ... And then finally, kill -9 the database server while updating the > index to test crash-safeness. >
Yves Weißig<weissig@rbg.informatik.tu-darmstadt.de> wrote: > Ok, but I thought more like an automated test, or a test which > checks if the interface is correctly implemented. I'm not aware of any automated tests which would test whether a new index type is correctly implemented. For starters, how would the test know when the AM should be used, or what correct results would be? Perhaps if this was a drop-in replacement for btree you could cobble something together so that the standard regression tests (`make check` and such) would use your index type instead of btree, yielding similar results; but you haven't given us the slightest clue whether that's the sort of index you're developing. > what would be the best way to debug in pg? elog? asserts? If there are clear guidelines on this anywhere, I've missed it or forgotten it. Here's what I tend to go by: (1) If it's a message which is expected enough to warrant translation to all the supported languages, use ereport. (2) If it's something which should be rare and technical enough not to warrant burdening the translators, use elog. (3) If it's something which "should never happen" unless there is a programming error within the PostgreSQL code, it should not normally be checked in production (just development and through beta testing), and it is sane to panic (effectively restarting PostgreSQL) when the condition is encountered, use Assert. That's all independent of what logging level you use. You may want to sprinkle your code with elog calls at the DEBUG level during development, and consider which might be worth keeping at which debug levels later on. Sometimes debug calls are conditioned on #ifdef conditions so that they're not in the executable without a special build option. -Kevin
Thanks for the answer Kevin! I am developing an encoded bitmap index, as proposed in http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.44.7570&rank=1 Automated, in my words, would have meant... some SQL-Statements which create tables, do inserts, deletes, selects and so on and the results can be compared to a master file or whatsoever. On low level I tend to go your way and debug with elog, I just thought somebody might have a different approach on debugging such code. Yves Am 22.04.2011 18:30, schrieb Kevin Grittner: > Yves Weißig<weissig@rbg.informatik.tu-darmstadt.de> wrote: > >> Ok, but I thought more like an automated test, or a test which >> checks if the interface is correctly implemented. > > I'm not aware of any automated tests which would test whether a new > index type is correctly implemented. For starters, how would the > test know when the AM should be used, or what correct results would > be? Perhaps if this was a drop-in replacement for btree you could > cobble something together so that the standard regression tests > (`make check` and such) would use your index type instead of btree, > yielding similar results; but you haven't given us the slightest > clue whether that's the sort of index you're developing. > >> what would be the best way to debug in pg? elog? asserts? > > If there are clear guidelines on this anywhere, I've missed it or > forgotten it. Here's what I tend to go by: > > (1) If it's a message which is expected enough to warrant > translation to all the supported languages, use ereport. > > (2) If it's something which should be rare and technical enough not > to warrant burdening the translators, use elog. > > (3) If it's something which "should never happen" unless there is a > programming error within the PostgreSQL code, it should not normally > be checked in production (just development and through beta > testing), and it is sane to panic (effectively restarting > PostgreSQL) when the condition is encountered, use Assert. > > That's all independent of what logging level you use. You may want > to sprinkle your code with elog calls at the DEBUG level during > development, and consider which might be worth keeping at which > debug levels later on. Sometimes debug calls are conditioned on > #ifdef conditions so that they're not in the executable without a > special build option. > > -Kevin >