In addition to what Mark pointed out, there is the possibility that a
query
is running which is scanning a large table or otherwise bringing in a
large number of pages from disk. That would first use up all available
unused cache space, and then may start replacing some of your
frequently used data. This can cause slowness for some time after the
process which flushed the cache, as pages are reread and recached.
Keep in mind that the cache could be flushed by some external process,
such as copying disk files.
The use of free memory for caching is not slowing you down; but if it
coincides with slowness, it could be a useful clue.
-Kevin