Re: pgbench - allow backslash-continuations in custom scripts

On 06/21/2015 11:12 AM, Fabien COELHO wrote:
>
> Hello Josh,
>
>>> Add backslash continuations to pgbench custom scripts.
>>> [...]
>>> IMHO this approach is the best compromise.
>>
>> I don't personally agree.  I believe that it it worth breaking backwards
>> compatibility to support line breaks in pgbench statements, and that if
>> we're not going to do that, supporting \ continuations is of little value.
>>
>> As someone who actively uses pgbench to write custom benchmarks, I need
>> to write queries which I can test.  \ continuation does NOT work on the
>> psql command line, so that's useless for testing my queries; I still
>> have to reformat and troubleshoot.  If we added \ continuation, I
>> wouldn't use it.
>>
>> I think we should support line breaks, and require semicolons for
>> end-of-statement.  Backwards-compatability in custom pgbench scripts is
>> not critical; pgbench scripts are neither used in produciton, nor used
>> in automated systems much that I know of.
>>
>> I'm not clear on why we'd need a full SQL lexer.
>
> Attached is a "without lexer" version which does ;-terminated SQL commands
> and \-continuated meta commands (may be useful for \shell and long \set
> expressions).

As Tom pointed out, you need the full lexer to do this correctly. You 
can argue that something that handles the most common cases is enough, 
but realistically, by the time you've handled all the common cases 
correctly, you've just re-invented the lexer.

The home-grown lexer is missing e.g. dollar-quoting support, so this is 
not be parsed correctly:

do $$
begin  ...
end;
$$;

That would be very nice to handle correctly, I've used DO-blocks in 
pgbench scripts many times, and it's a pain to have to write them in a 
single line.

Also worth noting that you can currently test so-called multi-statements 
with pgbench, by putting multiple statements on a single line. Your 
patch seems to still do that, but if we went with a full-blown SQL 
lexer, they would probably be split into two statements.

I think we should either bite the bullet and include the full SQL lexer 
in pgbench, or come up with some new syntax for marking the beginning 
and end of a statement. We could do something like bash here-documents 
or Postgres dollar-quoting, for example:

\set ...
select 1234; -- A statement on a single line, no change here

-- Begin a multi-line statement
\multi-line-statement END_TOKEN
select *  from complicated;
END_TOKEN

- Heikki