Which version were you originally seeing it in? I still see that behavior (or something close to it) in 9.4dev.
It stabilizes after doing a "vacuum analyze" (not just "analyze"), but that is because it then switches to Index Only Scan.
I haven't pinned it down exactly, but it seems that there are two issues.
One is that there are two estimates of the row returned, one for the whole query (which is visible in EXPLAIN, and is spot on) and one for the index patch, which is very wrong for 0000% case (the estimate was 1 row rather than 100, visible only in the debugger or with special logging code added). The other is that the correlation between index/column order and table order is thought to be 0 when it is really 1.