Re: Regression test fails when BLCKSZ is 1kB

On Tue, Apr 22, 2008 at 4:25 PM, Martijn van Oosterhout <<a
href="mailto:kleptog@svana.org">kleptog@svana.org</a>>wrote:<br /><div class="gmail_quote"><blockquote
class="gmail_quote"style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:
1ex;"><divclass="Ih2E3d">On Tue, Apr 22, 2008 at 10:31:53AM +0200, Zdenek Kotala wrote:<br /> > When you are able
detectordering difference you are able also check if it<br /> > is important for the test or not without any extra
effort.Only what we<br /> > need is put some flag to test that order is not important.<br /><br /></div>Not true.
Sortingthe file is going jumble all the results together.<br /> Since we perform many tests in one file, you're not
goingto be able to<br /> seperate them.<br /><div class="Ih2E3d"><br /> > Regression test MUST BE bulletproof. If
youget a error you must know that<br /> > it is really error (in postgresql or regtest) and must be fixed. When
you<br/> > start to ignore some errors because it can happen sometimes you fall in the<br /> > trap soon.<br
/><br/></div>I think people are misunderstanding. You posted a bunch of diffs with<br /> that comment that they
*appeared*to only be ordering differences. How<br /> good did you check? If an 8 become a 9 chances are you'd miss
it.<br/> Having a second test checking the sorted results would at least<br /> preclude the chance that there really is
somethingwrong.<br /><br /> It was a guide, not a way of getting out of tests.<br /></blockquote></div><br />In the
past,I had faced and tried to work on this exact problem... here's what I had in mind:<br /><br />in the .expected
file,we would demarcate the section of lines we expect to come in any order, by using two special markers. Then, when
comparingthe actual output with expected output, we would take the demarcated group of lines, and the corresponding
linesfrom actual output, and compare them after sorting.<br /><br />    For eg.<br /><br />foo.expected:<br /><br
/>select* from tenk where col1 <= 3 limit 3;<br /><br />col1 | col2 | col3<br />-------------------------<br
clear="all"/>?unsorted_result_start<br />1 | 10 | 100<br />2 | 20 | 200<br />3 | 30 | 300<br /> ?unsorted_result_end<br
/><br/><br />foo.out:<br />select * from tenk where col1 <= 3 limit 3;<br /><br /> col1 | col2 | col3<br />
-------------------------<brclear="all" />3 | 30 | 300<br /> 2 | 20 | 200<br /> 1 | 10 | 100<br /><br /><br />    So,
thediff program should discard the lines beginning with '?' (meta character), and then sort and match exactly the same
numberof lines.<br /><br />    There's another option of putting these '?' lines in a separate file with corresponding
begin/endline numbers of the unsorted group, and using this as a parameter to the diffing program.<br /><br />    Of
course,this needs a change in the (standard) diff that we use from pg_regress!<br /><br />Best regards,<br />-- <br
/>gurjeet[.singh]@EnterpriseDB.com<br/>singh.gurjeet@{ gmail | hotmail | indiatimes | yahoo }.com<br /><br />
EnterpriseDB<a href="http://www.enterprisedb.com">http://www.enterprisedb.com</a><br /><br />Mail sent from my
BlackLaptopdevice