Обсуждение: populating a time series database

I am writing an app where I have to populate file content into a database.

The file looks like this,

epoch,name, orderid, quantaty.


There are about 100k rows like this. I am curious is it best to
process each line 1 at a time or do a mass INSERT of 100k and then do
a execute?


Another question,

Since the data is going to be large, are there any tricks or tips
anyone knows how to handle time series data into a database? Is
postgresql the right tool or is there something else out there?

TIA

On 18 May 2010 11:51, Mag Gam <magawake@gmail.com> wrote:
> I am writing an app where I have to populate file content into a database.
>
> The file looks like this,
>
> epoch,name, orderid, quantaty.
>
>
> There are about 100k rows like this. I am curious is it best to
> process each line 1 at a time or do a mass INSERT of 100k and then do
> a execute?

Use COPY: http://www.postgresql.org/docs/current/static/sql-copy.html

You may wish to import it into a temporary table first before
converting the date into a useable format.

> Another question,
>
> Since the data is going to be large, are there any tricks or tips
> anyone knows how to handle time series data into a database? Is
> postgresql the right tool or is there something else out there?
>

Depends on what you're doing with the data.  If you're querying date
ranges, you'll need to index based on date.

Regards

Thom