Обсуждение: generating dynamic queries using pl/pgsql

Поиск
Список
Период
Сортировка

generating dynamic queries using pl/pgsql

От
sarlav kumar
Дата:
Hi All,
 
Is it possible to "generate" dynamic queries using pl/pgsql?
 
I am trying to archive a set of tables: affiliate_event, affiliate_batch, affiliate_batch_details, affiliate_daily_batch, affiliate_daily_batch_details.
 
The following steps are necessary for each table:
 
 1) select the data from the table based on the date (and other clause), and Move that data to a temporary table 
 2) Dump the temp table to a file
 3) Delete the data from the original table
 4) Delete the temporary table.
 
Is it possible for pl/pgsql to dynamically generate the SQL select queries needed in step 1??
What information/input would it need in that case?
 
These are the step 1 queries I am using:
 
create table temp1 as select * from affiliate_batch where tx_dt < '12/31/2003';
..
delete from temp1;
--------------------------------------------------------------------------------------------------------------------------------
create table temp1 as select bd.* from affiliate_batch_details bd join affiliate_batch b on bd.batch_id=b.id where  b.tx_dt < '12/31/2003' order by bd.batch_id;
..
delete from temp1;
--------------------------------------------------------------------------------------------------------------------------------
create table temp1 as select adb.* from affiliate_daily_batch adb join affiliate_batch b on adb.monthly_batch_id=b.id where b.tx_dt < '12/31/2003' order by adb.monthly_batch_id;
..
delete from temp1;
---------------------------------------------------------------------------------------------------------------------------------
create table temp1 as select adb.* from affiliate_daily_batch_details adb where exists (select ad.id from affiliate_daily_batch ad join affiliate_batch b on ad.monthly_batch_id=b.id where adb.batch_id=ad.id and b.tx_dt < '12/31/2003' order by ad.monthly_batch_id) order by adb.batch_id;
..
delete from temp1;
---------------------------------------------------------------------------------------------------------------------------------
create table temp1 as select * from affiliate_event where dt<'12/31/2003';
..
delete from temp1;
---------------------------------------------------------------------------------------------------------------------------------
 
Thanks in advance!
 
Saranya
 
 
 

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

Re: generating dynamic queries using pl/pgsql

От
"K Anderson"
Дата:
----- Original Message -----From: "sarlav kumar" <SARLAVK@YAHOO=
.COM>To: pgsqlnovice Subject: [NOVICE]=
 generating dynamic queries using pl/pgsqlDate: Tue, 18 Jan 2005 10:34:=
11 -0800 (PST)
Hi All,
 
Is it possible to "generate" dynamic queries using pl/pgsql?
 
I am trying to archive a set of tables: affiliate_event, affiliate_bat=
ch, affiliate_batch_details, affiliate_daily_batch, affiliate_daily_batch_d=
etails.
 
The following steps are necessary for each table:
 
 1) select the data from the table based on the date (and ot=
her clause), and Move that data to a temporary table 
 2) Dump the temp table to a file
 3) Delete the data from the original table
 4) Delete the temporary table.
 
Is it possible for pl/pgsql to dynamically generate the SQL =
select queries needed in step 1??
What information/input would it need in that case?
 
These are the step 1 queries I am using:
 
create table temp1 as select * from affiliate_ba=
tch where tx_dt < '12/31/2003';
..
delete from temp1;
----------------------------------------------------------------------=
----------------------------------------------------------
create table temp1 as select bd.* from affiliate=
_batch_details bd join affiliate_batch b on bd.batch_id=3Db.id where  =
b.tx_dt < '12/31/2003' order by bd.batch_id;
..
delete from temp1;
----------------------------------------------------------------------=
----------------------------------------------------------
create table temp1 as select adb.* from affiliat=
e_daily_batch adb join affiliate_batch b on adb.monthly_batch_id=3Db.id whe=
re b.tx_dt < '12/31/2003' order by adb.monthly_batch_id;
..
delete from temp1;
----------------------------------------------------------------------=
-----------------------------------------------------------
create table temp1 as select adb.* from affiliat=
e_daily_batch_details adb where exists (select ad.id from affiliate_daily_b=
atch ad join affiliate_batch b on ad.monthly_batch_id=3Db.id where adb.batc=
h_id=3Dad.id and b.tx_dt < '12/31/2003' order by ad.monthly_batch_id) or=
der by adb.batch_id;

..
delete from temp1;
----------------------------------------------------------------------=
-----------------------------------------------------------
create table temp1 as select * from affiliate_ev=
ent where dt<'12/31/2003';

..
delete from temp1;
----------------------------------------------------------------------=
-----------------------------------------------------------
 
Thanks in advance!
 
Saranya
 
----------------
 
Yes, it's possible.
 
<A href=3D"http://www.postgresql.org/docs/7.4/interactive/plpgsql-stat=
ements.html#PLPGSQL-STATEMENTS-EXECUTING-DYN">http://www.postgresql.org/doc=
s/7.4/interactive/plpgsql-statements.html#PLPGSQL-STATEMENTS-EXECUTING-DYN<=
/A>
 
The manual has some great information.
--=20
____________________________________________________
Get your free email from <a href=3D"http://www.kittymail.com" target=3D"_bl=
ank">http://www.kittymail.com

Powered by Outblaze

Re: generating dynamic queries using pl/pgsql

От
sarlav kumar
Дата:
>Yes, it's possible.
>The manual has some great information.
 
I read through the manual. But I am still not clear how to actually generate the "queries" dynamically. In my case, some of the "select queries" that I want to dynamically generate have joins and some of the "select queries" are just from one table.
 
I am new to pl/pgsql. I came up with the following code to start with. This one just has all the queries hardcoded.  I want to dynamically generate the "select query part" in the "create table temp1" statement. I am totally lost:(. It would be great if someone can help me!
 
Also, is it possible to dump a table within pl/pgsql?
 
 
create function try1(date,text) returns integer as '
declare
arch_date alias for $1;
filename alias for $2;
begin
 
create table temp1 as select * from affiliate_batch where tx_dt < arch_date;
-- Here, I would like to dump the table "temp1" to 'filename' before deleting it
drop table temp1;
 
create table temp1 as select bd.* from affiliate_batch_details bd join affiliate_batch b on bd.batch_id=b.id where  b.tx_dt < arch_date order by bd.batch_id;
-- Here, I would like to dump the table "temp1" to 'filename' before deleting it
drop table temp1;
 
create table temp1 as select * from affiliate_daily_batch adb where exists (select b.id from affiliate_batch b where b.tx_dt < arch_date and adb.monthly_batch_id=b.id) order by adb.monthly_batch_id;
-- Here, I would like to dump the table "temp1" to 'filename' before deleting it
drop table temp1;
 
create table temp1 as select adb.* from affiliate_daily_batch_details adb join affiliate_daily_batch ad on ad.id=adb.batch_id join affiliate_batch b on ad.monthly_batch_id=b.id where b.tx_dt < arch_date order by ad.monthly_batch_id;
-- Here, I would like to dump the table "temp1" to 'filename' before deleting it
drop table temp1;
 
create table temp1 as select * from affiliate_event where dt< arch_date;
-- Here, I would like to dump the table "temp1" to 'filename' before deleting it
drop table temp1;
 
create table temp1 as select * from clickthrough where date < arch_date||''23:59:59'';
-- Here, I would like to dump the table "temp1" to 'filename' before deleting it
drop table temp1;
 
return 1;
end;
' language 'plpgsql';
 
 
Thanks a lot for all the help.
 
Saranya


Do you Yahoo!?
Yahoo! Search presents - Jib Jab's 'Second Term'

Re: generating dynamic queries using pl/pgsql

От
Sean Davis
Дата:
On Jan 20, 2005, at 12:40 PM, sarlav kumar wrote:

> >Yes, it's possible.
>
> >http://www.postgresql.org/docs/7.4/interactive/plpgsql-
> statements.html#PLPGSQL->STATEMENTS-EXECUTING-DYN
> >The manual has some great information.
>  
> I read through the manual. But I am still not clear how to actually
> generate the "queries" dynamically. In my case, some of the
> "select queries" that I want to dynamically generate have joins and
> some of the "select queries" are just from one table.
>

The point of section 37.6.4 (see the manual again) is that you can
build a query from a combination of text and variables.  If you have
pieces of your query, you need to build up the full query using the ||
operator (concatenation).  Then you can EXECUTE it.

As for "dumping" the table, look at:

http://www.postgresql.org/docs/7.4/interactive/sql-copy.html

You can use COPY to do this.

HTH,
Sean


Re: generating dynamic queries using pl/pgsql

От
sarlav kumar
Дата:
Hi Sean,
 
Thanks, for the help. I got the dynamic query generation part to work.
The only thing left to do is to get the dump of the temporary table.
 
When I try to use COPY inside the pl/pgsql function, I get the following error:
 
 COPY temp1 to ''aff.txt'';
 
WARNING:  Error occurred while executing PL/pgSQL function try2
WARNING:  line 38 at SQL statement
ERROR:  Relative path not allowed for server side COPY command
 
Then I dropped the function, and recreated the function with the following command:
 
COPY temp1 to ''/home/developers/ss2/aff.txt'';

WARNING:  Error occurred while executing PL/pgSQL function try2
WARNING:  line 38 at SQL statement
ERROR:  COPY command, running in backend with effective uid 501, could not open file '/home/developers/ssivakumar/aff.txt' for writing.  Errno = No such file or directory (2).
How can I get the copy command to work from within the pl/pgsql?
I even tried using \copy, it still gives me the above error.
 
Is there anyway to use \! pg_dump command inside pl/pgsql? I tried it, but it does not allow the character "!". Is there a way to escape "!"?
 
Thanks in advance,
Saranya
 


>http://www.postgresql.org/docs/7.4/interactive/sql-copy.html

>You can use COPY to do this.

>HTH,
>Sean


Do you Yahoo!?
Yahoo! Search presents - Jib Jab's 'Second Term'

Re: generating dynamic queries using pl/pgsql

От
Sean Davis
Дата:
On Jan 21, 2005, at 11:00 AM, sarlav kumar wrote:

> Hi Sean,
>  
> Thanks, for the help. I got the dynamic query generation part to work.
> The only thing left to do is to get the dump of the temporary table.
>  
> When I try to use COPY inside the pl/pgsql function, I get the
> following error:
>  
>  COPY temp1 to ''aff.txt'';
>  
> WARNING:  Error occurred while executing PL/pgSQL function try2
> WARNING:  line 38 at SQL statement
> ERROR:  Relative path not allowed for server side COPY command
>  
> Then I dropped the function, and recreated the function with the
> following command:
>  
> COPY temp1 to ''/home/developers/ss2/aff.txt'';
>
> WARNING:  Error occurred while executing PL/pgSQL function try2
> WARNING:  line 38 at SQL statement
> ERROR:  COPY command, running in backend with effective uid 501, could
> not open file '/home/developers/ssivakumar/aff.txt' for writing. 
> Errno = No such file or directory (2).
> How can I get the copy command to work from within the pl/pgsql?
>

The tricky part about COPY is that it is executed by the SERVER!
Therefore, the tables can only be written to somewhere writable by the
user running the server process.  If, for example, you have a user
named postgres, you could set up a directory that is owned by postgres
and use that for the dumps.  /tmp is another place.  Of course, all
this has to be done on the SERVER machine; it can't be done locally to
a file.  I imagine that is the issue, but others can correct me if I am
wrong on this.

Another option is to COPY to STDOUT and then capture the output.

Sean


Re: generating dynamic queries using pl/pgsql

От
sarlav kumar
Дата:

Hi Sean,
 
The problem is that I dont have permission to create directories as a postgres user.
If I can get the \copy command or the \! pg_dump command to work, that would be great.
 
Thanks,
Saranya

Sean Davis <sdavis2@mail.nih.gov> wrote:

On Jan 21, 2005, at 11:00 AM, sarlav kumar wrote:

> Hi Sean,
>  
> Thanks, for the help. I got the dynamic query generation part to work.
> The only thing left to do is to get the dump of the temporary table.
>  
> When I try to use COPY inside the pl/pgsql function, I get the
> following error:
>  
>  COPY temp1 to ''aff.txt'';
>  
> WARNING:  Error occurred while executing PL/pgSQL function try2
> WARNING:  line 38 at SQL statement
> ERROR:  Relative path not allowed for server side COPY command
>  
> Then I dropped the function, and recreated the function with the
> following command:
>  
> COPY temp1 to ''/home/developers/ss2/aff.txt'';
>
> WARNING:  Error occurred while executing PL/pgSQL f unction try2
> WARNING:  line 38 at SQL statement
> ERROR:  COPY command, running in backend with effective uid 501, could
> not open file '/home/developers/ssivakumar/aff.txt' for writing. 
> Errno = No such file or directory (2).
> How can I get the copy command to work from within the pl/pgsql?
>

The tricky part about COPY is that it is executed by the SERVER!
Therefore, the tables can only be written to somewhere writable by the
user running the server process. If, for example, you have a user
named postgres, you could set up a directory that is owned by postgres
and use that for the dumps. /tmp is another place. Of course, all
this has to be done on the SERVER machine; it can't be done locally to
a file. I imagine that is the issue, but others can correct me if I am
wrong on this.

Another option is to COPY to STDOUT and then capture the output.

Sean

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

Re: generating dynamic queries using pl/pgsql

От
Sean Davis
Дата:
Saranya,

Just my personal suggestion, but I would move on to a scripting
language like perl.  Once you learn a bit of it, doing stuff like what
you are proposing is trivial.  Unfortunately, I don't have more
suggestions on making psql work for you.  If you want to automate the
whole process, then you will benefit from learning a scripting
language.  Others can correct me if they don't share my sentiments.

Sorry,
Sean

On Jan 21, 2005, at 11:44 AM, sarlav kumar wrote:

>
> Hi Sean,
>  
> The problem is that I dont have permission to create directories as a
> postgres user.
> If I can get the \copy command or the \! pg_dump command to work, that
> would be great.
>  
> Thanks,
> Saranya
>
> Sean Davis <sdavis2@mail.nih.gov> wrote:
>
> On Jan 21, 2005, at 11:00 AM, sarlav kumar wrote:
>
> > Hi Sean,
> >  
> > Thanks, for the help. I got the dynamic query generation part to
> work.
> > The only thing left to do is to get the dump of the temporary table.
> >  
> > When I try to use COPY inside the pl/pgsql function, I get the
>  > following error:
> >  
> >  COPY temp1 to ''aff.txt'';
> >  
> > WARNING:  Error occurred while executing PL/pgSQL function try2
> > WARNING:  line 38 at SQL statement
> > ERROR:  Relative path not allowed for server side COPY command
> >  
> > Then I dropped the function, and recreated the function with the
>  > following command:
> >  
> > COPY temp1 to ''/home/developers/ss2/aff.txt'';
> >
> > WARNING:  Error occurred while executing PL/pgSQL function try2
> > WARNING:  line 38 at SQL statement
> > ERROR:  COPY command, running in backend with effective uid 501,
> could
>  > not open file '/home/developers/ssivakumar/aff.txt' for writing. 
>  > Errno = No such file or directory (2).
> > How can I get the copy command to work from within the pl/pgsql?
> >
>
> The tricky part about COPY is that it is executed by the SERVER!
>  Therefore, the tables can only be written to somewhere writable by the
>  user running the server process. If, for example, you have a user
>  named postgres, you could set up a directory that is owned by postgres
>  and use that for the dumps. /tmp is another place. Of course, all
> this has to be done on the SERVER machine; it can't be done locally to
> a file. I imagine that is the issue, but others can correct me if I am
>  wrong on this.
>
> Another option is to COPY to STDOUT and then capture the output.
>
> Sean
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com


Re: generating dynamic queries using pl/pgsql

От
John DeSoi
Дата:
On Jan 21, 2005, at 11:44 AM, sarlav kumar wrote:

> The problem is that I dont have permission to create directories as a
> postgres user.
> If I can get the \copy command or the \! pg_dump command to work, that
> would be great.
>  

The commands starting with \ are psql commands. They are only executed
on the client side by psql. Stored procedures of course run on the
server, and don't know anything about psql \commands. What you can do
is setup a script for psql that first calls your stored procedure and
then uses \copy or \!pg_dump. But you can't put these commands in the
stored procedure.

John DeSoi, Ph.D.
http://pgedit.com/
Power Tools for PostgreSQL