Обсуждение: What's the best way to get flex and bison on Windows?

Hello,

On cfbot.cputube.org, we keep seeing random failures on appveyor (the
CI provider it uses for Windows) in this step:

- appveyor-retry cinst winflexbison

"cinst" is the apt/yum/pkg-like tool from the Chocolatey package
system, but unfortunately its repository is frequently unavailable.
"appveyor-retry" was added to cfbot by a pull request from David
Fettter (thanks!) and that reduced the rate of bogus failures quite a
lot by retrying 3 times, but I'm still seeing a sea of red from time
to time so I'd like to find a another source for those tools.

Here's the full set of software that is already installed on the
Windows build images:

https://www.appveyor.com/docs/windows-images-software/

You can see MinGW, MSYS and Cygwin there, and I suspect that one of
those is the answer, but I'm not familiar with them or what else might
be available to install popular F/OSS bits and pieces on that
operating system, because I really only know how to Unix.  Maybe flex
and bison are already installed somewhere or easily installable with a
shell command?  Would someone who knows about development on Windows
like to make a recommendation, or perhaps provide a tweaked version of
the attached patch[1]?

Thanks,

[1]  Instructions: apply to the PG source, push to a public github
branch (or gitlab, kiln, ...), log into appveyor.com with your github
(or ...) account, add the project, watch it build and test.

On Mon, Dec 16, 2019 at 11:25:55AM +1300, Thomas Munro wrote:
> You can see MinGW, MSYS and Cygwin there, and I suspect that one of
> those is the answer, but I'm not familiar with them or what else might
> be available to install popular F/OSS bits and pieces on that
> operating system, because I really only know how to Unix.  Maybe flex
> and bison are already installed somewhere or easily installable with a
> shell command?  Would someone who knows about development on Windows
> like to make a recommendation, or perhaps provide a tweaked version of
> the attached patch[1]?

On my Windows workstations, I use bison and flex bundled in MinGW
which are located under c:\MinGW\msys\1.0\bin\.  (Then, for a MSVC
build, I just append this path to $ENV{PATH} with a semicolon to do
the separation but that's a separate story).
--
Michael

On Mon, Dec 16, 2019 at 8:16 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Mon, Dec 16, 2019 at 11:25:55AM +1300, Thomas Munro wrote:
> > You can see MinGW, MSYS and Cygwin there, and I suspect that one of
> > those is the answer, but I'm not familiar with them or what else might
> > be available to install popular F/OSS bits and pieces on that
> > operating system, because I really only know how to Unix.  Maybe flex
> > and bison are already installed somewhere or easily installable with a
> > shell command?  Would someone who knows about development on Windows
> > like to make a recommendation, or perhaps provide a tweaked version of
> > the attached patch[1]?
>
> On my Windows workstations, I use bison and flex bundled in MinGW
> which are located under c:\MinGW\msys\1.0\bin\.  (Then, for a MSVC
> build, I just append this path to $ENV{PATH} with a semicolon to do
> the separation but that's a separate story).
>

I also have the same setup for flex and bison.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

On Mon, Dec 16, 2019 at 4:31 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Mon, Dec 16, 2019 at 8:16 AM Michael Paquier <michael@paquier.xyz> wrote:
> > On Mon, Dec 16, 2019 at 11:25:55AM +1300, Thomas Munro wrote:
> > > You can see MinGW, MSYS and Cygwin there, and I suspect that one of
> > > those is the answer, but I'm not familiar with them or what else might
> > > be available to install popular F/OSS bits and pieces on that
> > > operating system, because I really only know how to Unix.  Maybe flex
> > > and bison are already installed somewhere or easily installable with a
> > > shell command?  Would someone who knows about development on Windows
> > > like to make a recommendation, or perhaps provide a tweaked version of
> > > the attached patch[1]?
> >
> > On my Windows workstations, I use bison and flex bundled in MinGW
> > which are located under c:\MinGW\msys\1.0\bin\.  (Then, for a MSVC
> > build, I just append this path to $ENV{PATH} with a semicolon to do
> > the separation but that's a separate story).
>
> I also have the same setup for flex and bison.

Thanks Michael and Amit.  Adding SET
PATH=%PATH%;C:\MinGW\msys\1.0\bin\ did the trick, and allows me to
remove the Chocolatey dependency.  I'll apply that change to cfbot
tomorrow.

On Mon, Dec 16, 2019 at 2:04 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>
> On Mon, Dec 16, 2019 at 4:31 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > On Mon, Dec 16, 2019 at 8:16 AM Michael Paquier <michael@paquier.xyz> wrote:
> > > On Mon, Dec 16, 2019 at 11:25:55AM +1300, Thomas Munro wrote:
> > > > You can see MinGW, MSYS and Cygwin there, and I suspect that one of
> > > > those is the answer, but I'm not familiar with them or what else might
> > > > be available to install popular F/OSS bits and pieces on that
> > > > operating system, because I really only know how to Unix.  Maybe flex
> > > > and bison are already installed somewhere or easily installable with a
> > > > shell command?  Would someone who knows about development on Windows
> > > > like to make a recommendation, or perhaps provide a tweaked version of
> > > > the attached patch[1]?
> > >
> > > On my Windows workstations, I use bison and flex bundled in MinGW
> > > which are located under c:\MinGW\msys\1.0\bin\.  (Then, for a MSVC
> > > build, I just append this path to $ENV{PATH} with a semicolon to do
> > > the separation but that's a separate story).
> >
> > I also have the same setup for flex and bison.
>
> Thanks Michael and Amit.  Adding SET
> PATH=%PATH%;C:\MinGW\msys\1.0\bin\ did the trick, and allows me to
> remove the Chocolatey dependency.  I'll apply that change to cfbot
> tomorrow.
>
>


The reason I use chocolatey is to avoid having a dependency on Msys/Msys2 ;-)

If you're going to link to anything it should probably be to the Msys2
binaries, because a) they are likely to be more up to date and b)
unlike msys1 they are available by default for all four VS toolsets
Appveyor provides.

FYI I have captured a complete log of what chocolatey does, at
<https://gist.github.com/adunstan/12e4474c769aa88a584c450548bf2ffa>
Essentially it just downloads a zip from sourceforge and then sets up
shims to the binaries.

cheers

andrew

-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Dec 16, 2019 at 4:49 PM Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
> The reason I use chocolatey is to avoid having a dependency on Msys/Msys2 ;-)

Heh.  Well Chocolatey looks like a really nice project, and I see that
they package PostgreSQL, and their site shows 85,427 downloads.  Neat!
 I just think it's a good idea for this experimental CI to use things
that are already on the image if we can.  FWIW I've had trouble with
other stuff fetched over the network by PostgreSQL CI jobs at times
too (Docbook templates and Ubuntu packages).

> If you're going to link to anything it should probably be to the Msys2
> binaries, because a) they are likely to be more up to date and b)
> unlike msys1 they are available by default for all four VS toolsets
> Appveyor provides.

Ah, OK that sounds like a good plan then.  Any chance you can tell me
what to add to PATH for that?  Changing the 1 to a 2 in the path
mentioned before doesn't work.

On Mon, Dec 16, 2019 at 2:50 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > If you're going to link to anything it should probably be to the Msys2
> > binaries, because a) they are likely to be more up to date and b)
> > unlike msys1 they are available by default for all four VS toolsets
> > Appveyor provides.
>
> Ah, OK that sounds like a good plan then.  Any chance you can tell me
> what to add to PATH for that?  Changing the 1 to a 2 in the path
> mentioned before doesn't work.

The Appveyor page says "MSYS2 (C:\msys64)" so I would try adding
"C:\msys64\bin" to the PATH.

cheers

andrew

-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Dec 16, 2019 at 7:21 PM Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
> On Mon, Dec 16, 2019 at 2:50 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > > If you're going to link to anything it should probably be to the Msys2
> > > binaries, because a) they are likely to be more up to date and b)
> > > unlike msys1 they are available by default for all four VS toolsets
> > > Appveyor provides.
> >
> > Ah, OK that sounds like a good plan then.  Any chance you can tell me
> > what to add to PATH for that?  Changing the 1 to a 2 in the path
> > mentioned before doesn't work.
>
> The Appveyor page says "MSYS2 (C:\msys64)" so I would try adding
> "C:\msys64\bin" to the PATH.

Thanks.  That didn't work but helped me find my way to
C:\msys64\usr\bin.  That version of bison complains about our grammar
using deprecated directives, but that's a matter for another day.

https://ci.appveyor.com/project/macdice/postgres/builds/29560016

Thomas Munro <thomas.munro@gmail.com> writes:
> Thanks.  That didn't work but helped me find my way to
> C:\msys64\usr\bin.  That version of bison complains about our grammar
> using deprecated directives, but that's a matter for another day.

Oh, that's a known issue on late-model bison.  The problem is that the
option syntax it wants us to use doesn't exist at all on older bison
versions.  So far we haven't been willing to break old platforms just
to suppress the warning.  We'll probably have to find another answer
once a decent fraction of PG hackers start using bison versions that
give the warning.

            regards, tom lane