I've been poking at the project of using a reentrant flex scanner for
PG. In order to make this work in a non-klugy way, it turns out that
we have to also upgrade our Bison parser to be a "pure parser" ---
otherwise Bison doesn't want to pass the additional scanner pointer to
yylex(). I haven't finished getting it to work, but we're going to need
at least these additions to gram.y:
***************
*** 140,149 ****
--- 147,160 ---- %}
+ %pure-parser %expect 0 %name-prefix="base_yy" %locations
+ %parse-param {base_yyscan_t yyscanner}
+ %lex-param {base_yyscan_t yyscanner}
+ %union { int ival;
Now what surprised me about this is that it appears to work just fine in
our oldest supported Bison version (1.875), although %parse-param and
%lex-param are not documented in the 1.875 manual. Some digging in the
bug-bison archives (hurrah for open source and the Internet --- it took
only a few minutes to find this information) revealed that they pulled
the documentation at the last minute before releasing 1.875, because
they didn't want users to depend on these features just yet. This
appears to have been due to some argument about whether the API was
right, plus noticing that the documentation wasn't up to date
everywhere.
As best I can tell, they ended up not changing the API, and there is no
reason we shouldn't depend on the feature and continue to claim that we
work with bison >= 1.875. Does anyone feel uncomfortable with that?
(It may be of mostly academic interest anyway, since I bet few people
are still using such old bison versions. The only reason I've got one
is for intentional trailing-edge compatibility testing...)
regards, tom lane