On Mon, Nov 27, 2023 at 9:06 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> At this point one thing that IMO we cannot afford to do, is stop feature
> progress work on the name of parser speed. I mean, parser speed is
> important, and we need to be mindful that what we add is reasonable.
> But at some point we'll probably have to fix that by parsing
> differently (a top-down parser, perhaps? Split the parser in smaller
> pieces that each deal with subsets of the whole thing?)
I was reorganizing some old backups and rediscovered an experiment I
did four years ago when I had some extra time on my hands, to use a
lexer generator that emits a state machine driven by code, rather than
a table. It made parsing 12% faster on the above info-schema test, but
only (maybe) 3% on parsing pgbench-like queries. My quick hack ended
up a bit uglier and more verbose than Flex, but that could be
improved, and in fact small components could be shared across the
whole code base. I might work on it again; I might not.