Обсуждение: [HACKERS] Continuous integration on Windows?
Hi hackers, I don't use Windows myself, but I'd rather avoid submitting patches that fail to build, build with horrible warnings or blow up on that fine operating system. I think it would be neat to be able to have experimental branches of PostgreSQL built and tested on Windows automatically just by pushing them to a watched public git repo. Just like Travis CI and several others do for the major GNU/Linux distros, it seems there is at least one Windows-based CI company that generously offers free testing to open source projects: https://www.appveyor.com (hat tip to Ilmari for the pointer). I wonder... has anyone here with Microsoft know-how ever tried to produce an appveyor.yml file that would do a MSVC build and check-world? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On 10/11/2017 11:04 PM, Thomas Munro wrote: > Hi hackers, > > I don't use Windows myself, but I'd rather avoid submitting patches > that fail to build, build with horrible warnings or blow up on that > fine operating system. I think it would be neat to be able to have > experimental branches of PostgreSQL built and tested on Windows > automatically just by pushing them to a watched public git repo. Just > like Travis CI and several others do for the major GNU/Linux distros, > it seems there is at least one Windows-based CI company that > generously offers free testing to open source projects: > https://www.appveyor.com (hat tip to Ilmari for the pointer). I > wonder... has anyone here with Microsoft know-how ever tried to > produce an appveyor.yml file that would do a MSVC build and > check-world? > Interesting. I'm taking a look. A couple of things not in their pre-built images that we'll need are flex and bison. We might be able to overcome that with chocolatey, which is installed, haven't tested yet. getting a working appveyor.yml will take a little while, though. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On 10/12/2017 04:14 PM, Andrew Dunstan wrote: > > On 10/11/2017 11:04 PM, Thomas Munro wrote: >> Hi hackers, >> >> I don't use Windows myself, but I'd rather avoid submitting patches >> that fail to build, build with horrible warnings or blow up on that >> fine operating system. I think it would be neat to be able to have >> experimental branches of PostgreSQL built and tested on Windows >> automatically just by pushing them to a watched public git repo. Just >> like Travis CI and several others do for the major GNU/Linux distros, >> it seems there is at least one Windows-based CI company that >> generously offers free testing to open source projects: >> https://www.appveyor.com (hat tip to Ilmari for the pointer). I >> wonder... has anyone here with Microsoft know-how ever tried to >> produce an appveyor.yml file that would do a MSVC build and >> check-world? >> > > Interesting. > > I'm taking a look. > > A couple of things not in their pre-built images that we'll need are > flex and bison. We might be able to overcome that with chocolatey, which > is installed, haven't tested yet. > > getting a working appveyor.yml will take a little while, though. Actually, that didn't take too long. No testing yet, but this runs a build successfully: <https://gist.github.com/adunstan/7f18e5db33bb2d73f69ff8c9337a4e6c> See results at <https://ci.appveyor.com/project/AndrewDunstan/pgdevel> cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On 2017-10-12 17:57:11 -0400, Andrew Dunstan wrote: > No testing yet, but this runs a build successfully: > <https://gist.github.com/adunstan/7f18e5db33bb2d73f69ff8c9337a4e6c> > > See results at <https://ci.appveyor.com/project/AndrewDunstan/pgdevel> "Time Elapsed 00:04:36.37" I'd expected building would take a lot longer than that... Neat. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Oct 13, 2017 at 10:57 AM, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote: > Actually, that didn't take too long. > > No testing yet, but this runs a build successfully: > <https://gist.github.com/adunstan/7f18e5db33bb2d73f69ff8c9337a4e6c> > > See results at <https://ci.appveyor.com/project/AndrewDunstan/pgdevel> Excellent! Thanks for looking at this, Andrew. It's going to be really useful for removing surprises. It would be nice to finish up with a little library of control files like this for different CI vendors, possibly with some different options (different compilers, different word size, add valgrind, ...). I don't know if it would ever make sense to have standardised CI control files in the tree -- many projects do -- but it's very easy to carry a commit that adds them in development branches but drop it as part of the format-patch-and-post process. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On 10/12/2017 06:46 PM, Thomas Munro wrote: > On Fri, Oct 13, 2017 at 10:57 AM, Andrew Dunstan > <andrew.dunstan@2ndquadrant.com> wrote: >> Actually, that didn't take too long. >> >> No testing yet, but this runs a build successfully: >> <https://gist.github.com/adunstan/7f18e5db33bb2d73f69ff8c9337a4e6c> >> >> See results at <https://ci.appveyor.com/project/AndrewDunstan/pgdevel> > Excellent! Thanks for looking at this, Andrew. It's going to be > really useful for removing surprises. > > It would be nice to finish up with a little library of control files > like this for different CI vendors, possibly with some different > options (different compilers, different word size, add valgrind, ...). > I don't know if it would ever make sense to have standardised CI > control files in the tree -- many projects do -- but it's very easy to > carry a commit that adds them in development branches but drop it as > part of the format-patch-and-post process. > Well, as you can see here the appveyor.yml file can live outside the tree that's being built. What would be good, though, would be to split build.pl into two so you wouldn't need the auxiliary file I had to create from it. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Oct 13, 2017 at 1:42 PM, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote: > Well, as you can see here the appveyor.yml file can live outside the > tree that's being built. Here's a Wiki page where I hope we can collect how-to information on this general topic: https://wiki.postgresql.org/wiki/Continuous_Integration I tried your appveyor.yml, and added: test_script: - cd src\tools\msvc && vcregress check That much I could figure out just by reading our manual and I could see that it worked first time, but to make this really useful I guess we'd have to teach it to dump out resulting regression.diffs files and backtraces from core files (as the Travis CI version accessible from that page does). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On 10/13/2017 08:09 AM, Thomas Munro wrote: > On Fri, Oct 13, 2017 at 1:42 PM, Andrew Dunstan > <andrew.dunstan@2ndquadrant.com> wrote: >> Well, as you can see here the appveyor.yml file can live outside the >> tree that's being built. > Here's a Wiki page where I hope we can collect how-to information on > this general topic: > > https://wiki.postgresql.org/wiki/Continuous_Integration > > I tried your appveyor.yml, and added: > > test_script: > - cd src\tools\msvc && vcregress check > > That much I could figure out just by reading our manual and I could > see that it worked first time, but to make this really useful I guess > we'd have to teach it to dump out resulting regression.diffs files and > backtraces from core files (as the Travis CI version accessible from > that page does). > I'll add some info to the wiki. Unfortunately, the tests fail on the tablespace test because they are running as a privileged user. We need to find a way around that, still looking into it. (See <https://blog.2ndquadrant.com/setting-build-machine-visual-studio-2017/> which describes how I get around that when running by hand). cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
This may seems obvious for you but where is the build result ? Is it stored somewhere to permit to users like me that want to test pg 10 on windows without having to build it ? Regards PAscal -- Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Oct 14, 2017 at 7:47 AM, legrand legrand <legrand_legrand@hotmail.com> wrote: > This may seems obvious for you > > but where is the build result ? Each CI platform has a web page corresponding to your GitHub/BitBucket/... user account that lists builds results. You can also get notifications by various means including email if there is a failure. Here's a randomly selected example build log: https://travis-ci.org/postgresql-cfbot/postgresql/builds/287749152 That particular build happened because a cronjob of mine pushed a Commitfest patch into a branch on github, and I maintain plenty more branches here: https://github.com/postgresql-cfbot/postgresql/branches Here are the resulting builds: https://travis-ci.org/postgresql-cfbot/postgresql/branches Here's a build from a personal development branch of my own, this time on AppVeyor: https://ci.appveyor.com/project/macdice/postgres/build/1.0.3 As Andrew mentioned, AppVeyor builds using his appveyor.yml with the addition of test_script that I showed above current fail in "test tablespace" for some reason that we'll need to sort out, and as I mentioned you can't yet see the regressions.diff output etc... but it's a start. This is actually already useful for me, because I changed a bunch of temporarily file cleanup code and I wanted to confirm that it would work on Window. That's being exercised in one of the tests. So far so good. > Is it stored somewhere to permit to users like me > that want to test pg 10 on windows > without having to build it ? That's the idea. It's quite similar to the build farm, except that it tests your stuff *before* it gets committed to master and everyone shouts at you, and can also be used to test strange experiments and theories without disturbing anyone else. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Oct 14, 2017 at 11:27 AM, Thomas Munro <thomas.munro@enterprisedb.com> wrote: > On Sat, Oct 14, 2017 at 7:47 AM, legrand legrand > <legrand_legrand@hotmail.com> wrote: >> Is it stored somewhere to permit to users like me >> that want to test pg 10 on windows >> without having to build it ? Erm, wait, you said pg 10. That's already released. This thread is about building and automatically testing development code on Windows. If it's pre-built releases of PostgreSQL you're looking for, they're over here: https://www.postgresql.org/download/windows/ -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Oups what a silly boy, I didn't saw that pg10 was released ;o( That's all I need soon (to test Declarative partitioning). Thanks for taking time to answer me in a so detailled and interesting manner. Regards PAscal -- Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Oct 14, 2017 at 3:08 AM, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote: > I'll add some info to the wiki. Unfortunately, the tests fail on the > tablespace test because they are running as a privileged user. We need > to find a way around that, still looking into it. (See > <https://blog.2ndquadrant.com/setting-build-machine-visual-studio-2017/> > which describes how I get around that when running by hand). Thanks for that pointer. "runas" seems to be no good in batch files since it asks for a password interactively, but I managed to get the tablespace test to pass by running it like this: test_script: - net user testuser password1234! /add - psexec.exe -u testuser -p password1234! -w c:\projects\postgres\src\tools\msvc perl.exe vcregress.pl check (It probably goes without saying but I'll say it anyway for anyone who might find this: this plaintext-password-in-script-files stuff is intended for use on self-destructing isolated build bot images only and should never be done on a computer you care about.) Hooray! Now I can go and figure out why my Parallel Hash regression test is failing with file permissions problems on Windows... -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers