I have been trying to get keyword searching quicker and have now decided to make smaller tables.
I was working with tables of about 50 000 000 rows but now will use about 1.5million rows, I am using a TSearch2 to find keywords in the titles column of the rows. I have indexes using a gist index on the TSearch2 using 'default'.
I wanted to know if it would be quicker to change my words in the title into crc32 bit integers and then look for them using an index which would be on the crc numbers. I could then cluster on the crc numbers and this should result in a fast search but I just want to know if you know of better ways of doing keyword searching, I need it to be about 1-4 seconds and return usually around 1-5 thousand rows.
I know places like google and ebay do seaches and only return 200 or 100 max rows, this can be done on our current system very fast as you just limit the number returned but for our purposes I need to return all the rows,
Thanks
smokedvw