Обсуждение: TPC H data

Hello,

We are trying to import the TPC-H data into postgresql using the COPY
command and for the larger files we get an error due to insufficient
memory space.

We are using a linux system with Postgresql-7.3.4

Is it that Postgresql cannot handle such large files or is there some
other possible reason.

Thanks
Shalu Gupta
NC State University.

Shalu Gupta wrote:

>Hello,
>
>We are trying to import the TPC-H data into postgresql using the COPY
>command and for the larger files we get an error due to insufficient
>memory space.
>
>We are using a linux system with Postgresql-7.3.4
>
>Is it that Postgresql cannot handle such large files or is there some
>other possible reason.
>
>Thanks
>Shalu Gupta
>NC State University.
>
>
>

Shalu,

I loaded the largest TPC-H table (lineitem, roughly 6 million rows) the
other day into a completely untuned 7.5devel PostgreSQL instance running
on RH 9, and it didn't raise a sweat. I delayed creating the indexes
until after the load. Data load took roughly 10 minutes, index creation
took a further 35 minutes (there are 13 of them).

HTH. (I'm just down the road from NCSU, would be happy to help out)

cheers

andrew

On Wed, 21 Apr 2004, Shalu Gupta wrote:

> Hello,
>
> We are trying to import the TPC-H data into postgresql using the COPY
> command and for the larger files we get an error due to insufficient
> memory space.
>
> We are using a linux system with Postgresql-7.3.4
>
> Is it that Postgresql cannot handle such large files or is there some
> other possible reason.

what method(s) are you using to load the data?

What scale factor TPC H are you importing?

additionally -  might be worth giving the specs of the machine you are
doing this on.

(I seem to recall trying this with Pg 7.2 a while ago without this
issue, mind you -  think I had ~1G of Ram and used the scale fact 1
dataset, i.e 1G)

regards

Mark


Shalu Gupta wrote:

>Hello,
>
>We are trying to import the TPC-H data into postgresql using the COPY
>command and for the larger files we get an error due to insufficient
>memory space.
>
>We are using a linux system with Postgresql-7.3.4
>
>Is it that Postgresql cannot handle such large files or is there some
>other possible reason.
>
>Thanks
>Shalu Gupta
>NC State University.
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: Have you checked our extensive FAQ?
>
>               http://www.postgresql.org/docs/faqs/FAQ.html
>
>
>
>