Обсуждение: transactions start time
Hi,
In statistical reports gathered by PgBadger on our PostgreSQL databases almost always we have in "Queries that took up the most time" report table information about transactions start time ('BEGIN;' command). Something like that in example below:
2 3h34m52.26s 48,556,167 0.00s BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.80s | BEGIN;
0.80s | BEGIN;
0.79s | BEGIN;
0.79s | BEGIN;
Databases placed on different hardware, OS - Debian GNU/Linux, PostgreSQL 9.1
So, questions are:
1. Is this a normal situation with transactions start time ( BEGIN method) ?
2. How can we reduce transactions start time if it's possible in principle?
3. What happens in PostgreSQL on transaction starting time? Can someone describe this process in detail? (of course, I saw in PostgreSQL source code, for example, definition such kind functions, like StartTransaction function, but it's not so easy to understand for third-party researcher, that all of these operations mean in real for performance)
Best Regards
Aleksei
In statistical reports gathered by PgBadger on our PostgreSQL databases almost always we have in "Queries that took up the most time" report table information about transactions start time ('BEGIN;' command). Something like that in example below:
2 3h34m52.26s 48,556,167 0.00s BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.80s | BEGIN;
0.80s | BEGIN;
0.79s | BEGIN;
0.79s | BEGIN;
Databases placed on different hardware, OS - Debian GNU/Linux, PostgreSQL 9.1
So, questions are:
1. Is this a normal situation with transactions start time ( BEGIN method) ?
2. How can we reduce transactions start time if it's possible in principle?
3. What happens in PostgreSQL on transaction starting time? Can someone describe this process in detail? (of course, I saw in PostgreSQL source code, for example, definition such kind functions, like StartTransaction function, but it's not so easy to understand for third-party researcher, that all of these operations mean in real for performance)
Best Regards
Aleksei
On 24/07/12 12:14, Aleksei Arefjev wrote:
> Hi,
>
> In statistical reports gathered by PgBadger on our PostgreSQL databases
> almost always we have in "Queries that took up the most time" report
> table information about transactions start time ('BEGIN;' command).
> Something like that in example below:
>
> 2 3h34m52.26s 48,556,167 0.00s BEGIN;
>
> 0.82s | BEGIN;
> 0.82s | BEGIN;
> 0.82s | BEGIN;
> 0.81s | BEGIN;
> 0.81s | BEGIN;
> 0.81s | BEGIN;
> 0.80s | BEGIN;
> 0.80s | BEGIN;
> 0.79s | BEGIN;
> 0.79s | BEGIN;
I'm not sure if I'm reading this right, but are there more than 48
million BEGINs that took 0s each (presumably rounded down) and then a
handful taking about 0.8s?
If so, then it's likely nothing to do with the BEGIN and just that the
machine was busy doing other things when you started a transaction.
> Databases placed on different hardware, OS - Debian GNU/Linux,
> PostgreSQL 9.1
>
> So, questions are:
> 1. Is this a normal situation with transactions start time ( BEGIN method) ?
See above
> 2. How can we reduce transactions start time if it's possible in principle?
Below 0.00? Probably not
> 3. What happens in PostgreSQL on transaction starting time? Can someone
> describe this process in detail? (of course, I saw in PostgreSQL source
> code, for example, definition such kind functions, like StartTransaction
> function, but it's not so easy to understand for third-party researcher,
> that all of these operations mean in real for performance)
Well there are two important things to understand:
1. All* commands run in a transaction
2. I think most of the work in getting a new snapshot etc gets pushed
back until it's needed.
So - the overall impact of issuing BEGIN should be close to zero.
--
Richard Huxton
Archonet Ltd
On 24 July 2012 20:21, Richard Huxton <dev@archonet.com> wrote:
0.00s - this is the average duration parameter column. Them, seems, much more, and those were shown like examples.
Perhaps so, but, at execution time, there were not any problem with performance on those machines.
Yes, I know it.
Probably so, but I wanna know, is there any opportunity to optimize this process.
On 24/07/12 12:14, Aleksei Arefjev wrote:I'm not sure if I'm reading this right, but are there more than 48 million BEGINs that took 0s each (presumably rounded down) and then a handful taking about 0.8s?Hi,
In statistical reports gathered by PgBadger on our PostgreSQL databases
almost always we have in "Queries that took up the most time" report
table information about transactions start time ('BEGIN;' command).
Something like that in example below:
2 3h34m52.26s 48,556,167 0.00s BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.80s | BEGIN;
0.80s | BEGIN;
0.79s | BEGIN;
0.79s | BEGIN;
0.00s - this is the average duration parameter column. Them, seems, much more, and those were shown like examples.
If so, then it's likely nothing to do with the BEGIN and just that the machine was busy doing other things when you started a transaction.
Perhaps so, but, at execution time, there were not any problem with performance on those machines.
See aboveDatabases placed on different hardware, OS - Debian GNU/Linux,
PostgreSQL 9.1
So, questions are:
1. Is this a normal situation with transactions start time ( BEGIN method) ?Below 0.00? Probably not2. How can we reduce transactions start time if it's possible in principle?Well there are two important things to understand:3. What happens in PostgreSQL on transaction starting time? Can someone
describe this process in detail? (of course, I saw in PostgreSQL source
code, for example, definition such kind functions, like StartTransaction
function, but it's not so easy to understand for third-party researcher,
that all of these operations mean in real for performance)
1. All* commands run in a transaction
Yes, I know it.
2. I think most of the work in getting a new snapshot etc gets pushed back until it's needed.
Probably so, but I wanna know, is there any opportunity to optimize this process.
So - the overall impact of issuing BEGIN should be close to zero.
--
Richard Huxton
Archonet Ltd
And yet, repeating the question: What happens in PostgreSQL on transaction starting time? Can someone
describe this process in detail?
Regards
Aleksei
On 24 July 2012 20:21, Richard Huxton <dev@archonet.com> wrote:
If so, maybe using of 'SET TRANSACTION SNAPSHOT' command with the pre-existing transaction exported snapshot by the pg_export_snapshot function could be usefull for reducing transactions start time -
http://www.postgresql.org/docs/9.2/static/sql-set-transaction.html
On 24/07/12 12:14, Aleksei Arefjev wrote:I'm not sure if I'm reading this right, but are there more than 48 million BEGINs that took 0s each (presumably rounded down) and then a handful taking about 0.8s?Hi,
In statistical reports gathered by PgBadger on our PostgreSQL databases
almost always we have in "Queries that took up the most time" report
table information about transactions start time ('BEGIN;' command).
Something like that in example below:
2 3h34m52.26s 48,556,167 0.00s BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.82s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.81s | BEGIN;
0.80s | BEGIN;
0.80s | BEGIN;
0.79s | BEGIN;
0.79s | BEGIN;
If so, then it's likely nothing to do with the BEGIN and just that the machine was busy doing other things when you started a transaction.See aboveDatabases placed on different hardware, OS - Debian GNU/Linux,
PostgreSQL 9.1
So, questions are:
1. Is this a normal situation with transactions start time ( BEGIN method) ?Below 0.00? Probably not2. How can we reduce transactions start time if it's possible in principle?Well there are two important things to understand:3. What happens in PostgreSQL on transaction starting time? Can someone
describe this process in detail? (of course, I saw in PostgreSQL source
code, for example, definition such kind functions, like StartTransaction
function, but it's not so easy to understand for third-party researcher,
that all of these operations mean in real for performance)
1. All* commands run in a transaction
2. I think most of the work in getting a new snapshot etc gets pushed back until it's needed.
If so, maybe using of 'SET TRANSACTION SNAPSHOT' command with the pre-existing transaction exported snapshot by the pg_export_snapshot function could be usefull for reducing transactions start time -
http://www.postgresql.org/docs/9.2/static/sql-set-transaction.html
So - the overall impact of issuing BEGIN should be close to zero.
--
Richard Huxton
Archonet Ltd
Aleksei Arefjev <aleksei.arefjev@nordicgaming.com> writes:
> On 24 July 2012 20:21, Richard Huxton <dev@archonet.com> wrote:
>> I'm not sure if I'm reading this right, but are there more than 48 million
>> BEGINs that took 0s each (presumably rounded down) and then a handful
>> taking about 0.8s?
I'm wondering exactly where/how the duration was measured. If it was at
a client, maybe the apparent delay had something to do with network
glitches? It seems suspicious that all the outliers are around 0.8s.
It would be useful to look to see if there's any comparable pattern
for statements other than BEGIN.
As Richard says, a BEGIN by itself ought to take negligible time.
regards, tom lane
Hi, On Wednesday, July 25, 2012 04:56:20 PM Tom Lane wrote: > Aleksei Arefjev <aleksei.arefjev@nordicgaming.com> writes: > > On 24 July 2012 20:21, Richard Huxton <dev@archonet.com> wrote: > >> I'm not sure if I'm reading this right, but are there more than 48 > >> million BEGINs that took 0s each (presumably rounded down) and then a > >> handful taking about 0.8s? > > I'm wondering exactly where/how the duration was measured. If it was at > a client, maybe the apparent delay had something to do with network > glitches? It seems suspicious that all the outliers are around 0.8s. > It would be useful to look to see if there's any comparable pattern > for statements other than BEGIN. > > As Richard says, a BEGIN by itself ought to take negligible time. He earlier also asked on the IRC-Channel and I got the idea that the problem could be explained by pgbouncer in transaction pooling mode waiting for a free backend connection. Aleksei confirmed that they use pgbouncer in that configuration, so that might be it. Andres -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services