Re: [HACKERS] \dt and disk access

>
> On Sat, 21 Jun 1997, Bruce Momjian wrote:
>
> > > ExecSort calls psort() to do the work. psort() uses a Knuth algorithm.
> > >
> > > I removed the psort() call from ExecSort, and the execution time was:
> > >
> > >     Select with order by using psort():        0.30
> > >
> > >     Select with order by NOT using psort():        0.07
> > >
> > > Now of course, not using psort() causes the data not to be sorted, but
> > > there must be some inefficienty if sorting one row takes 0.23 seconds on
> > > a PP200 with 64MB of RAM.  I suspect the other 0.06 seconds is caused by
> > > the creation of the temp table and moving the rows around.
> >
> > It looks like psort()'s use of disk files to do the sorting is causing
> > the slowdown, with the multiple create's and unlink's.  I am going to
> > modify psort() to use memory buffers instead of disk files for sorting
> > small SELECT results.  psort() is only called by ExecSort, which is used
> > by ORDER BY's.
> >
> > Permission to put this in 6.1p1?
>
> Can you make the size of the result set above which diskfiles will be used
> configurable? That way ppl with loads of RAM can use huge buffers, and ppl
> with little RAM can keep that RAM free for other processes.

Good idea.  I think the final code will not use the buffer cache, but
directly palloc the memory it needs and copy the rows to be sorted into
there.

I also hope to eliminate the copy to a temp table.  Currently, the code
copies to a temp table, loads the rows into "tape" files, then writes to
a new table, which seems like a waste.  Paul's FASTBUILD btree code uses
ftruncate() after he has loaded the rows from the table, before he
writes them back.

I will contact him directly for comments.

- --
Bruce Momjian
maillist@candle.pha.pa.us

------------------------------