Re: Building full-text index

Поиск
Список
Период
Сортировка
От Tom Lane
Тема Re: Building full-text index
Дата
Msg-id 4320.1195250436@sss.pgh.pa.us
обсуждение исходный текст
Ответ на Building full-text index  ("Sean Davis" <sdavis2@mail.nih.gov>)
Ответы Re: Building full-text index  ("Sean Davis" <sdavis2@mail.nih.gov>)
Список pgsql-novice
"Sean Davis" <sdavis2@mail.nih.gov> writes:
> I am trying to build a full-text index (gin(to_tsvector('english',
> title || abstract))) on about 18 million abstracts and titles from
> medical literature.  However, I keep getting out-of-memory errors.  (I
> am on a 32Gb linux system with maintenance_work_mem set to 20Gb and
> shared buffers at 4Gb; postgres 8.3beta).  Does creation of a
> full-text index require that the entire index fit into memory?

I looked closer at this and discovered that there's an overflow problem
in the GIN index build code: with maintenance_work_mem above 8Gb, it
miscalculates how much space it's used and never realizes when it's
reached the intended limit.  So indeed you were seeing it try to create
the entire index in memory :-(.

This will be fixed in the next beta, but in the meantime set
maintenance_work_mem to something less than 8Gb.

            regards, tom lane

В списке pgsql-novice по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: Lock and unlock registry
Следующее
От: "Sean Davis"
Дата:
Сообщение: Re: Building full-text index