Обсуждение: UUID, UUID-OSSP extension, and ODBC issue

Поиск
Список
Период
Сортировка

UUID, UUID-OSSP extension, and ODBC issue

От
Jeremy Thornton
Дата:
ODBC: pgsql 09.03.0200;
OS: Windows 8.1 Pro;
Postgres version: 9.3.3;

In using pgsqlODBC 09.03.0200 or any version prior that we have tested we seem to be having a performance decrease to only 625 inserts per second when using UUID as the primary key and using the UUID-OSSP to generate the default value of uuid_generate_v1() whereas if we generate the UUID client side for the primary key and send it with the values in the insert transaction we can increase it to 1500 inserts per second bypassing the UUID-OSSP extension. This is consistent and repeatable. The only change being made is Postgres generating the UUID using the extension or us passing the value in the insert statement. If Postgres generates it internally while inserting using a script through PgAdmin then it can generate at near 2000 inserts per second so uuid-ossp shouldn't be the problem since it can keep up inside PgAdmin, but when used over default settings in ODBC it is 2/3 slower. The data being tested is the same in all cases. I am using a default System DSN on Windows 8.1 Pro (no special settings). Is this a bug, expected behavior, or a settings adjustment that needs to be made?

DSN settings:

AB = 0
BI = 0
BoolsAsChar = 1
ByteaAsLongVarbinary = 1
CancelAsFreeStmt = 0
CommLog = 0
Debug = 0
DisallowPremature = 0
Driver = 0903\bin\psqlodbc30a.dll
ExtraSysTablePrefixes = dd_;
FakeOidIndex = 0
Fetch = 100
GssAuthUseGSS = 0
Ksqo = 1
LFConversion = 1
LowerCaseIdentifier = 0
MaxLongVarcharSize = 8190
MaxVarCharSize = 255
Optimizer = 0
Parse = 0
Port = 5432
Protocol = 7.4-1
ReadOnly = 0
RowVersioning = 0
ShowOidColumn = 0
ShowSystemTables = 0
SSLmode = allow
TextAsLongVarchar = 1
TurelsMinus1 = 0
UniqueIndex = 1
UnknownsAsLongVarChar = 1
UnkownSizes = 0
UpdatableCursors = 1
UseDeclareFetch = 0
UseServerSidePrepare = 1
XaOpt = 1


Re: [ODBC] UUID, UUID-OSSP extension, and ODBC issue

От
Heikki Linnakangas
Дата:
On 02/28/2014 06:31 PM, Jeremy Thornton wrote:
> ODBC: pgsql 09.03.0200;OS: Windows 8.1 Pro;Postgres version: 9.3.3;
> In using pgsqlODBC 09.03.0200
>        or any version prior that we have tested we seem to be having a
>        performance decrease to only 625 inserts per second when
>      using UUID as the primary key and using the UUID-OSSP to generate
>      the default value of uuid_generate_v1() whereas if we generate the
>      UUID client side for the primary key and send it with the values in
>      the insert transaction we can increase it to 1500 inserts per second
>      bypassing the UUID-OSSP extension. This is consistent and
>      repeatable. The only change being made is Postgres generating the
>      UUID using the extension or us passing the value in the insert
>      statement. If Postgres generates it internally while inserting using a script through PgAdmin then it can
generateat near 2000 inserts per second so 
>      uuid-ossp shouldn't be the problem since it can keep up inside PgAdmin, but when
>      used over default settings in ODBC it is 2/3 slower. The data being
>      tested is the same in all cases. I am using a default System DSN on
>      Windows 8.1 Pro (no special settings). Is this a bug, expected behavior, or a settings
>      adjustment that needs to be made?

I doubt this is an ODBC issue. I'm guessing that uuid_generate_v1() is
simply quite slow.

The upcoming PostgreSQL 9.4 will contain a new method for generating
UUIDs: the pgcrypto contrib module will include a gen_random_uuid()
function, which creates v4 random UUIDs. In a quick test, that's a lot
faster than the UUID-OSSP based functions.

If you need it bad enough and can't wait for 9.4, you could extract that
into a stand-alone module for older version. It's not a lot of code. Or
you could implement such a function in PL/pgSQL using pgcrypto's
get_random_bytes() function.

- Heikki


Re: [ODBC] UUID, UUID-OSSP extension, and ODBC issue

От
Jeremy Thornton
Дата:
Thanks for the info Heikki. However, my concern is the fact that the same insert when ran inside PgAdmin executes at 2000 inserts per second whereas through ODBC it is at 625 inserts per second. It seems that if uuid_generate_v1() was the only bottleneck it would occur inside PgAdmin. Same information, same uuid_generate_v1() default. Any thoughts on why the performance difference between the two?

- Jeremy

> Date: Fri, 28 Feb 2014 19:25:17 +0200
> From: hlinnakangas@vmware.com
> To: consolity@outlook.com
> CC: pgsql-bugs@postgresql.org; pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] UUID, UUID-OSSP extension, and ODBC issue
>
> On 02/28/2014 06:31 PM, Jeremy Thornton wrote:
> > ODBC: pgsql 09.03.0200;OS: Windows 8.1 Pro;Postgres version: 9.3.3;
> > In using pgsqlODBC 09.03.0200
> > or any version prior that we have tested we seem to be having a
> > performance decrease to only 625 inserts per second when
> > using UUID as the primary key and using the UUID-OSSP to generate
> > the default value of uuid_generate_v1() whereas if we generate the
> > UUID client side for the primary key and send it with the values in
> > the insert transaction we can increase it to 1500 inserts per second
> > bypassing the UUID-OSSP extension. This is consistent and
> > repeatable. The only change being made is Postgres generating the
> > UUID using the extension or us passing the value in the insert
> > statement. If Postgres generates it internally while inserting using a script through PgAdmin then it can generate at near 2000 inserts per second so
> > uuid-ossp shouldn't be the problem since it can keep up inside PgAdmin, but when
> > used over default settings in ODBC it is 2/3 slower. The data being
> > tested is the same in all cases. I am using a default System DSN on
> > Windows 8.1 Pro (no special settings). Is this a bug, expected behavior, or a settings
> > adjustment that needs to be made?
>
> I doubt this is an ODBC issue. I'm guessing that uuid_generate_v1() is
> simply quite slow.
>
> The upcoming PostgreSQL 9.4 will contain a new method for generating
> UUIDs: the pgcrypto contrib module will include a gen_random_uuid()
> function, which creates v4 random UUIDs. In a quick test, that's a lot
> faster than the UUID-OSSP based functions.
>
> If you need it bad enough and can't wait for 9.4, you could extract that
> into a stand-alone module for older version. It's not a lot of code. Or
> you could implement such a function in PL/pgSQL using pgcrypto's
> get_random_bytes() function.
>
> - Heikki

Re: [ODBC] UUID, UUID-OSSP extension, and ODBC issue

От
Heikki Linnakangas
Дата:
On 02/28/2014 07:49 PM, Jeremy Thornton wrote:
> Thanks for the info Heikki. However, my concern is the fact that the
> same insert when ran inside PgAdmin executes at 2000 inserts per
> second whereas through ODBC it is at 625 inserts per second. It seems
> that if uuid_generate_v1() was the only bottleneck it would occur
> inside PgAdmin. Same information, same uuid_generate_v1() default.
> Any thoughts on why the performance difference between the two?

Oh, I misunderstood the issue then. How exactly are you executing the
inserts? Do you run all the inserts as a single transaction (that's
faster than separate transactions)? If you can create a self-contained
ODBC test program and post it to the list, I can take a closer look...

- Heikki


Re: [ODBC] UUID, UUID-OSSP extension, and ODBC issue

От
Jeremy Thornton
Дата:
After further testing it appears just to be the slow uuid-ossp extensions. Here is some additional results. Because of the difference between the v4/v1 extension times via ODBC I do not think now that ODBC is the problem either. I look forward to when Postgres will have it own native uuid v1 and v4 functions that are faster (more like newid() or newsequentialid() in SQL server). I mean 1,149 and 763 rows per second using the uuid-ossp extension may not be slow by some standards, but we are running a multi-tenant application that requires as much speed as we can pull out of it. 

The insert is ran as one transaction.

UUID v4 when generated client side postgres inserts 100,000 rows in 49 seconds (2040 inserts per sec[ips]),
UUID v1 when generated client side postgres inserts 100,000 rows in 49 seconds (2040 ips),
UUID v4 when generated by uuid_generate_v4() uuid-ossp extension postgres inserts 100,000 rows in 87 seconds (1,149 ips),
UUID v1 when generated by uuid_generate_v1() uuid-ossp extension postgres inserts 100,000 rows in 131 seconds (763 ips).

- Jeremy