Обсуждение: Re: pgbench and timestamps (bounced)

Поиск
Список
Период
Сортировка

Re: pgbench and timestamps (bounced)

От
Tom Lane
Дата:
Fabien COELHO <coelho@cri.ensmp.fr> writes:
> [Resent on hackers for CF registration, sorry for the noise]

For the record, the original thread is at

https://www.postgresql.org/message-id/flat/CAKVUGgQaZVAUi1Ex41H4wrru%3DFU%2BMfwgjG0aM1br6st7sz31Vw%40mail.gmail.com

(I tried but failed to attach that thread to the CF entry, so we'll
have to settle for leaving a breadcrumb in this thread.)

> It requires a mutex around the commands, I tried to do some windows
> implementation which may or may not work.

Ugh, I'd really rather not do that.  Even disregarding the effects
of a mutex, though, my initial idea for fixing this has a big problem:
if we postpone PREPARE of the query until first execution, then it's
happening during timed execution of the benchmark scenario and thus
distorting the timing figures.  (Maybe if we'd always done it like
that, it'd be okay, but I'm quite against changing the behavior now
that it's stood for a long time.)

However, perhaps there's more than one way to fix this.  Once we've
scanned all of the script and seen all the \set commands, we know
(in principle) the set of all variable names that are in use.
So maybe we could fix this by

(1) During the initial scan of the script, make variable-table
entries for every \set argument, with the values shown as undefined
for the moment.  Do not try to parse SQL commands in this scan,
just collect them.

(2) Make another scan in which we identify variable references
in the SQL commands and issue PREPAREs (if enabled).

(3) Perform the timed run.

This avoids any impact of this bug fix on the semantics or timing
of the benchmark proper.  I'm not sure offhand whether this
approach makes any difference for the concerns you had about
identifying/suppressing variable references inside quotes.

            regards, tom lane



Re: pgbench and timestamps (bounced)

От
Fabien COELHO
Дата:
Hello Tom,

>> It requires a mutex around the commands, I tried to do some windows
>> implementation which may or may not work.
>
> Ugh, I'd really rather not do that.  Even disregarding the effects
> of a mutex, though, my initial idea for fixing this has a big problem:
> if we postpone PREPARE of the query until first execution, then it's
> happening during timed execution of the benchmark scenario and thus
> distorting the timing figures.  (Maybe if we'd always done it like
> that, it'd be okay, but I'm quite against changing the behavior now
> that it's stood for a long time.)

Hmmm.

Prepare is done *once* per client, ISTM that the impact on any 
statistically significant benchmark is nul in practice, or it would mean 
that the benchmark settings are too low.

Second, the mutex is only used when absolutely necessary, only for the 
substitution part of the query (replacing :stuff by ?), because scripts 
are shared between threads. This is just once, in an unlikely case 
occuring at the beginning.

> However, perhaps there's more than one way to fix this.  Once we've
> scanned all of the script and seen all the \set commands, we know
> (in principle) the set of all variable names that are in use.
> So maybe we could fix this by
>
> (1) During the initial scan of the script, make variable-table
> entries for every \set argument, with the values shown as undefined
> for the moment.  Do not try to parse SQL commands in this scan,
> just collect them.

The issue with this approach is

   SELECT 1 AS one \gset pref_

which will generate a "pref_one" variable, and these names cannot be 
guessed without SQL parsing and possibly execution. That is why the
preparation is delayed to when the variables are actually known.

> (2) Make another scan in which we identify variable references
> in the SQL commands and issue PREPAREs (if enabled).

> (3) Perform the timed run.
>
> This avoids any impact of this bug fix on the semantics or timing
> of the benchmark proper.  I'm not sure offhand whether this
> approach makes any difference for the concerns you had about
> identifying/suppressing variable references inside quotes.

I do not think this plan is workable, because of the \gset issue.

I do not see that the conditional mutex and delayed PREPARE would have any 
significant (measurable) impact on an actual (reasonable) benchmark run.

A workable solution would be that each client actually execute each script 
once before starting the actual benchmark. It would still need a mutex and 
also a sync barrier (which I'm proposing in some other thread). However 
this may raise some other issues because then some operations would be 
trigger out of the benchmarking run, which may or may not be desirable.

So I'm not to keen to go that way, and I think the proposed solution is 
reasonable from a benchmarking point of view as the impact is minimal, 
although not zero.

-- 
Fabien.



Re: pgbench and timestamps (bounced)

От
Anastasia Lubennikova
Дата:
On 11.09.2020 16:59, Fabien COELHO wrote:
>
> Hello Tom,
>
>>> It requires a mutex around the commands, I tried to do some windows
>>> implementation which may or may not work.
>>
>> Ugh, I'd really rather not do that.  Even disregarding the effects
>> of a mutex, though, my initial idea for fixing this has a big problem:
>> if we postpone PREPARE of the query until first execution, then it's
>> happening during timed execution of the benchmark scenario and thus
>> distorting the timing figures.  (Maybe if we'd always done it like
>> that, it'd be okay, but I'm quite against changing the behavior now
>> that it's stood for a long time.)
>
> Hmmm.
>
> Prepare is done *once* per client, ISTM that the impact on any 
> statistically significant benchmark is nul in practice, or it would 
> mean that the benchmark settings are too low.
>
> Second, the mutex is only used when absolutely necessary, only for the 
> substitution part of the query (replacing :stuff by ?), because 
> scripts are shared between threads. This is just once, in an unlikely 
> case occuring at the beginning.
>
>> However, perhaps there's more than one way to fix this.  Once we've
>> scanned all of the script and seen all the \set commands, we know
>> (in principle) the set of all variable names that are in use.
>> So maybe we could fix this by
>>
>> (1) During the initial scan of the script, make variable-table
>> entries for every \set argument, with the values shown as undefined
>> for the moment.  Do not try to parse SQL commands in this scan,
>> just collect them.
>
> The issue with this approach is
>
>   SELECT 1 AS one \gset pref_
>
> which will generate a "pref_one" variable, and these names cannot be 
> guessed without SQL parsing and possibly execution. That is why the
> preparation is delayed to when the variables are actually known.
>
>> (2) Make another scan in which we identify variable references
>> in the SQL commands and issue PREPAREs (if enabled).
>
>> (3) Perform the timed run.
>>
>> This avoids any impact of this bug fix on the semantics or timing
>> of the benchmark proper.  I'm not sure offhand whether this
>> approach makes any difference for the concerns you had about
>> identifying/suppressing variable references inside quotes.
>
> I do not think this plan is workable, because of the \gset issue.
>
> I do not see that the conditional mutex and delayed PREPARE would have 
> any significant (measurable) impact on an actual (reasonable) 
> benchmark run.
>
> A workable solution would be that each client actually execute each 
> script once before starting the actual benchmark. It would still need 
> a mutex and also a sync barrier (which I'm proposing in some other 
> thread). However this may raise some other issues because then some 
> operations would be trigger out of the benchmarking run, which may or 
> may not be desirable.
>
> So I'm not to keen to go that way, and I think the proposed solution 
> is reasonable from a benchmarking point of view as the impact is 
> minimal, although not zero.
>
CFM reminder.

Hi, this entry is "Waiting on Author" and the thread was inactive for a 
while. I see this discussion still has some open questions. Are you 
going to continue working on it, or should I mark it as "returned with 
feedback" until a better time?

-- 
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company




Re: pgbench and timestamps (bounced)

От
Fabien COELHO
Дата:
> CFM reminder.
>
> Hi, this entry is "Waiting on Author" and the thread was inactive for a 
> while. I see this discussion still has some open questions. Are you 
> going to continue working on it, or should I mark it as "returned with 
> feedback" until a better time?

IMHO the proposed fix is reasonable and addresses the issue.

I have responded to Tom's remarks about it, and it is waiting for his 
answer to my counter arguments. So ISTM that the patch is currently still 
waiting for some feedback.

-- 
Fabien.



Re: pgbench and timestamps (bounced)

От
Tom Lane
Дата:
Fabien COELHO <coelho@cri.ensmp.fr> writes:
>> Hi, this entry is "Waiting on Author" and the thread was inactive for a
>> while. I see this discussion still has some open questions. Are you
>> going to continue working on it, or should I mark it as "returned with
>> feedback" until a better time?

> IMHO the proposed fix is reasonable and addresses the issue.
> I have responded to Tom's remarks about it, and it is waiting for his
> answer to my counter arguments. So ISTM that the patch is currently still
> waiting for some feedback.

It looks like my unhappiness with injecting a pthread dependency into
pgbench is going to be overtaken by events in the "option delaying
queries" thread [1].  However, by the same token there are some conflicts
between the two patchsets, and also I prefer the other thread's approach
to portability (i.e. do it honestly, not with a private portability layer
in pgbench.c).  So I'm inclined to put the parts of this patch that
require pthreads on hold till that lands.

What remains that we could do now, and perhaps back-patch, is point (2)
i.e. disallow digits as the first character of a pgbench variable name.
That would be enough to "solve" the original bug report, and it does seem
like it could be back-patched, while we're certainly not going to risk
back-patching anything as portability-fraught as introducing mutexes.

            regards, tom lane

[1] https://www.postgresql.org/message-id/flat/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de



Re: pgbench and timestamps (bounced)

От
Fabien COELHO
Дата:
Hello Tom,

>>> Hi, this entry is "Waiting on Author" and the thread was inactive for a
>>> while. I see this discussion still has some open questions. Are you
>>> going to continue working on it, or should I mark it as "returned with
>>> feedback" until a better time?
>
>> IMHO the proposed fix is reasonable and addresses the issue.
>> I have responded to Tom's remarks about it, and it is waiting for his
>> answer to my counter arguments. So ISTM that the patch is currently still
>> waiting for some feedback.
>
> It looks like my unhappiness with injecting a pthread dependency into
> pgbench is going to be overtaken by events in the "option delaying
> queries" thread [1].  However, by the same token there are some conflicts
> between the two patchsets, and also I prefer the other thread's approach
> to portability (i.e. do it honestly, not with a private portability layer
> in pgbench.c).  So I'm inclined to put the parts of this patch that
> require pthreads on hold till that lands.

Ok. This is fair enough. Portability is a pain thanks to Windows vs MacOS 
vs Linux approaches of implementing or not a standard.

> What remains that we could do now, and perhaps back-patch, is point (2)
> i.e. disallow digits as the first character of a pgbench variable name.

I'm fine with that.

> That would be enough to "solve" the original bug report, and it does seem
> like it could be back-patched, while we're certainly not going to risk
> back-patching anything as portability-fraught as introducing mutexes.

Sure.

I'm unable to do much pg work at the moment, but this should be eased 
quite soon.

> [1] https://www.postgresql.org/message-id/flat/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de

-- 
Fabien Coelho - CRI, MINES ParisTech



Re: pgbench and timestamps (bounced)

От
Tom Lane
Дата:
Fabien COELHO <coelho@cri.ensmp.fr> writes:
>> What remains that we could do now, and perhaps back-patch, is point (2)
>> i.e. disallow digits as the first character of a pgbench variable name.

> I'm fine with that.

>> That would be enough to "solve" the original bug report, and it does seem
>> like it could be back-patched, while we're certainly not going to risk
>> back-patching anything as portability-fraught as introducing mutexes.

> Sure.

OK.  I've pushed a patch that just does that much, and marked the
commitfest entry closed.  After the other thing lands, please rebase
and resubmit what remains of this patch.

            regards, tom lane