Обсуждение: Does Postgres support bookmarks (tuples ids)?
I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.
Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.
I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?
The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.
Would you recommend a solution?
Thank you
Konstantin
Perhaps you want to use the ctid. You can query it like any other column:
http://www.postgresql.org/docs/9.0/static/ddl-system-columns.html
The ctid is not permanent. An alternative is to create tables with OID values.
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Konstantin Izmailov
Sent: Tuesday, November 30, 2010 9:50 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] Does Postgres support bookmarks (tuples ids)?
Dear experts,
I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.
Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.
I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?
The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.
Would you recommend a solution?
Thank you
Konstantin
Is your application by chance using OLEDB?
If that is the case, then just get a PostgreSQL OLEDB provider that supports bookmarks.
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Konstantin Izmailov
Sent: Tuesday, November 30, 2010 9:50 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] Does Postgres support bookmarks (tuples ids)?
Dear experts,
I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.
Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.
I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?
The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.
Would you recommend a solution?
Thank you
Konstantin
On Wed, Dec 1, 2010 at 12:57 AM, Dann Corbit <DCorbit@connx.com> wrote: > The ctid is not permanent. An alternative is to create tables with OID > values. Creating OIDs gives you zero benefit over having a PK. They would be roughly equivalent if you added a unique index on the OID column, but that is by no means automatic.
On Wed, 2010-12-01 at 09:55 -0500, Vick Khera wrote: > On Wed, Dec 1, 2010 at 12:57 AM, Dann Corbit <DCorbit@connx.com> wrote: > > The ctid is not permanent. An alternative is to create tables with OID > > values. > > Creating OIDs gives you zero benefit over having a PK. They would be > roughly equivalent if you added a unique index on the OID column, but > that is by no means automatic. It is also a really bad idea considering the global nature of an OID. JD > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt
it is a very good suggestion! How does OLEDB implement the bookmarks? Is there a specific Postgres feature that is used for the bookmarks?
Thank you!
Konstantin
Is your application by chance using OLEDB?
If that is the case, then just get a PostgreSQL OLEDB provider that supports bookmarks.
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Konstantin Izmailov
Sent: Tuesday, November 30, 2010 9:50 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] Does Postgres support bookmarks (tuples ids)?
Dear experts,
I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.
Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.
I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?
The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.
Would you recommend a solution?
Thank you
Konstantin