So.. I would like to understand the "why" of this behaviour, ie. the change of order when I do the cast.
I believe the "why" is immaterial here. Your queries do not contain order by so your results are unordered - even if there appears to be an apparent ordering for any particular result.