Re: tsearch profiling - czech environment - take 55MB

2010/3/11 Tom Lane <tgl@sss.pgh.pa.us>:
> Pavel Stehule <pavel.stehule@gmail.com> writes:
>> There are some wrong in our implementation NISortDictionary. After
>> initialisation is ts_cache memory context 55MB long and pg takes
>> 190MB.
>
> What's your tsearch configuration exactly?
>

files: http://www.pgsql.cz/data/czech.tar.gz

configuration:

CREATE TEXT SEARCH DICTIONARY cspell  (template=ispell, dictfile = czech, afffile=czech, stopwords=czech);
CREATE TEXT SEARCH CONFIGURATION cs (copy=english);
ALTER TEXT SEARCH CONFIGURATION cs  ALTER MAPPING FOR word, asciiword WITH cspell, simple;

then try: select * from ts_debug('cs','Příliš žluťoučký kůň se napil
žluté vody');

with some time (used fce clock())

cspell: 1024 total in 1 blocks; 136 free (1 chunks); 888 used
After dictionary loading 320000
cspell: 3072 total in 2 blocks; 568 free (5 chunks); 2504 used Ispell dictionary init context: 27615288 total in 13
blocks;7710864 
free (12 chunks); 19904424 used
After AffFile loading 330000
cspell: 816952 total in 78 blocks; 18072 free (18 chunks); 798880 used Ispell dictionary init context: 27615288 total
in13 blocks; 7710864 
free (20 chunks); 19904424 used
After stop words loading 330000
cspell: 816952 total in 78 blocks; 13360 free (13 chunks); 803592 used Ispell dictionary init context: 27615288 total
in13 blocks; 7710864 
free (20 chunks); 19904424 used
****** 1 ******
cspell: 816952 total in 78 blocks; 9240 free (12 chunks); 807712 used Ispell dictionary init context: 27615288 total in
13blocks; 7710864 
free (20 chunks); 19904424 used
****** 2 ****** 380000
cspell: 825144 total in 79 blocks; 8440 free (10 chunks); 816704 used Ispell dictionary init context: 27615288 total in
13blocks; 7710864 
free (20 chunks); 19904424 used
****** 2.5 ****** 490000
// mkSPNode
cspell: 825144 total in 79 blocks; 8440 free (10 chunks); 816704 used Ispell dictionary init context: 27615288 total in
13blocks; 7710864 
free (20 chunks); 19904424 used
****** 3 ****** 580000
cspell: 55706480 total in 6775 blocks; 140200 free (1728 chunks); 55566280 used Ispell dictionary init context:
27615288total in 13 blocks; 7710864 
free (20 chunks); 19904424 used
After dictionary sort 580000
cspell: 55706480 total in 6775 blocks; 140200 free (1728 chunks); 55566280 used Ispell dictionary init context:
27615288total in 13 blocks; 7710864 
free (20 chunks); 19904424 used
After Affixes sort 580000
cspell: 55853736 total in 6789 blocks; 130208 free (1553 chunks); 55723528 used Ispell dictionary init context:
27615288total in 13 blocks; 7710864 
free (34 chunks); 19904424 used
final 580000
cspell: 55853736 total in 6789 blocks; 130208 free (1553 chunks); 55723528 used Ispell dictionary init context:
27615288total in 13 blocks; 7710864 
free (34 chunks); 19904424 used
executor start



>                        regards, tom lane
>