Sub-selects taking way too long..

Ok.. I know its provably something im doing dumb..
but here it goes..

I have 2 tables that are the same:

"temp_table" and "table"

"temp _table" has 7,761 rows and "table" is empty

the columns for both tables are: ID (primary key sequence), index, column1,
column2

when i run:

Insert Into table
select index, column1, column2
from temp_table
where index NOT IN (select index from table)

it takes 40 MINUTES to execute..
i dont know what im doing wrong here both tables have "index" indexed ..
help...