Обсуждение: pgloader - Can't find textreader/csvreader
I'm trying to see if pgloader will make my work easier for bulkloads. I'm testing it out and I'm stucked basically because it can't find the module TextReader or CSVreader. Googling doesn't help as there seems to be no reference to a module named textreader or csvreader. I'm on Python 2.4.4 Thanks.
Le lundi 27 août 2007, Ow Mun Heng a écrit : > I'm trying to see if pgloader will make my work easier for bulkloads. > I'm testing it out and I'm stucked basically because it can't find the > module TextReader or CSVreader. > > Googling doesn't help as there seems to be no reference to a module > named textreader or csvreader. It's a pgloader provided module, and the error arise because I forgot to make sure you can use pgloader without installing it properly in the system. The only installation supported now is via the debian package, if you're using debian either use the .deb provided on pgfoundry or issue a debuild command. You can also tweak the makefile to add some mkdir -p commands before it installs files in non-existing directories, too, and then install pgloader with make install. (debian packaging cares about system dirs, hence not the provided makefile) I'll see about having pgloader usable without any system-wide installation... Regards, -- dim
Вложения
On Mon, 2007-08-27 at 12:22 +0200, Dimitri Fontaine wrote: > Le lundi 27 août 2007, Ow Mun Heng a écrit : > > I'm trying to see if pgloader will make my work easier for bulkloads. > > I'm testing it out and I'm stucked basically because it can't find the > > module TextReader or CSVreader. > > > It's a pgloader provided module, and the error arise because I forgot to make > sure you can use pgloader without installing it properly in the system. > The only installation supported now is via the debian package, if you're using > debian either use the .deb provided on pgfoundry or issue a debuild command. I use gentoo.. so.... > > You can also tweak the makefile to add some mkdir -p commands before it > installs files in non-existing directories, too, and then install pgloader I Downloaded the 2.2.0 tarball and can't find any reference to it from this tarball except for the import statement. I'll try again with the (is there are 2.2.1, I thought I saw it) or take a peek at the deb package.. Thanks. ps : (no internet at the momemt.. only email access)
Le lundi 27 août 2007, Ow Mun Heng a écrit : > > It's a pgloader provided module, and the error arise because I forgot to > > make sure you can use pgloader without installing it properly in the > > system. After some testing, it seems pgloader is still usable without system installation at all. But I forgot some new files in the 2.2.0 release. > I'll try again with the (is there are 2.2.1, I thought I saw it) or take > a peek at the deb package.. We've just made some tests here with 2.2.1 and as this release contains the missing files, it works fine without any installation. Regards, -- dim
Вложения
On Mon, 2007-08-27 at 11:27 +0200, Dimitri Fontaine wrote: > We've just made some tests here with 2.2.1 and as this release contains the > missing files, it works fine without any installation. Yep.. I can confirm that it works.. I am using the csv example. Goal : similar functionality much like mysql's mysqlimport --replace (overwrite any rows which has duplicate primary keys) $ psql pgloader < csv/csv.sql $ ../pgloader.py -Tvc examples/pgloader.conf csv pgloader=# alter table csv add primary key (a,b,c); pgloader=# \d csv Table "public.csv" Column | Type | Modifiers --------+--------------+----------- a | bigint | not null b | bigint | not null c | character(2) | not null d | text | Indexes: "csv_pkey" PRIMARY KEY, btree (a, b, c) pgloader=# select * from csv; a | b | c | d ----------+----------+----+---------------- 33996344 | 33996351 | GB | United Kingdom 50331648 | 68257567 | US | United States 68257568 | 68257599 | CA | Canada 68257600 | 68259583 | US | United States 68259584 | 68259599 | CA | Canada $cat csv/csv.data "2.6.190.56","2.6.190.63","33996344","33996351","GB","Error Kingdom" "4.17.143.0","4.17.143.15","68259584","68259599","CA","new Country" <Note : only columns 3 to 6 are taken for loading) $ psql pgloader < csv/csv.sql $ ../pgloader.py -vc pgloader.conf csv Using pgloader.conf configuration file Will consider following sections: csv [csv] parse configuration Notice: reject log in /tmp/csv.rej.log Notice: rejected data in /tmp/csv.rej [csv] data import Notice: COPY csv data Error: Please check PostgreSQL logs HINT: double check your client_encoding, datestyle and copy_delimiter settings $sudo tail -f /var/log/pglog/postgresxx-xx-xx.log ERROR: duplicate key violates unique constraint "csv_pkey" CONTEXT: COPY csv, line 1: "33996344,33996351,Error Kingdom,GB" STATEMENT: COPY csv (a, b, d, c) FROM stdin USING DELIMITERS ',' So.. doesn't really solve my issue. Dang it..