Обсуждение: question on parsing postgres sql queries
Hello,
I am doing some research on postgres sql query parsing.
I have found the https://github.com/lfittl/libpg_query project which manages to re-use the native postgres server parser. For using this, you need to accept an external dependency on a lib compiled out of the postgres source.
I was wondering what people think of the conformance with regards to the real parser of the documentations on
... and more generally sgmls in https://github.com/postgres/postgres/tree/master/doc/src/sgml/ref
Would it make sense to use these sgml synopsis as some kind of source of truth, parse them, and automatically generate a parser for a specifc language ?
This could enable the creation of parsers for different languages using parser generators based on the synopsis.
I feel like the conformance level of the documentation is high and that the sgml synopis seem to be nearly programmatically sufficient to create parsers.
what do you think ?
Could the parser commiters share some lights on how the documentation process interacts with the parser commits ?
Thanks,
Jerome
Jerome Wagner <jerome.wagner@laposte.net> writes:
> Would it make sense to use these sgml synopsis as some kind of source of
> truth, parse them, and automatically generate a parser for a specifc
> language ?
Probably not. First, it is not uncommon for corner cases (such as
legacy syntaxes) to go unmentioned in the documentation. Second, the
implementation is often encrusted with details we'd just as soon not
expose to users. An example here is the need to be very specific in
the Bison grammar about whether extra parens in a "foo IN ((SELECT ...))"
construct belong to the SELECT or the IN.
It might be nice to have some sort of tool that could check compatibility
of the doc synopses with the actual grammar. But I doubt that trying to
auto-generate either one from the other would be a win.
regards, tom lane
Would it make sense to use these sgml synopsis as some kind of source of truth, parse them, and automatically generate a parser for a specifc language ?
What problem are you trying to solve here? to whit not everything that can be parsed is documented - usually intentionally.
Could the parser commiters share some lights on how the documentation process interacts with the parser commits ?
Commits that modify the parser are expected to have manual modifications to the relevant documentation as well.
David J.
> What problem are you trying to solve here? to whit not everything that can be parsed is documented - usually intentionally.
I am tyring to see whether we could use the documentation as a kind of formal specification of the language but I understand that the devil is
in the details and that even formal specifications can lead to incompatible implementations,
I would have found it nice if the clean documentation of the project could be used as a meta-grammar sufficient to maybe generate the grammar but I will have to dig further into the Bison grammar files.
The project I mentioned that isolates the parser from PostgreSQL binary as a re-usable library is probably the closest you can get currently to a parser matching the real engine.
Otherwise, yes, parsing the synopsis could maybe lead to a sanity check on the fact that the documentation is in line with the grammar. This could lead to warnings or help uncover unexpected corner cases not mentioned in the documentation.
Thanks for your answers
Jerome
On Tue, Jul 26, 2016 at 9:52 PM, David G. Johnston <david.g.johnston@gmail.com> wrote:
Would it make sense to use these sgml synopsis as some kind of source of truth, parse them, and automatically generate a parser for a specifc language ?What problem are you trying to solve here? to whit not everything that can be parsed is documented - usually intentionally.Could the parser commiters share some lights on how the documentation process interacts with the parser commits ?Commits that modify the parser are expected to have manual modifications to the relevant documentation as well.David J.