Yes, the smaller instance you choose - the slower ebs will be.
EBS lives separately from EC2, they are communicating via network. So small instance = low network bandwidth = poorer disk performance.
But still strong recommendation to pre-warm your ebs in any case, especially if they created from snapshot.