"I go down to the second floor."
Translation:Lemegyek a második emeletre.
It's just how it is.
I'm on the first floor. = Az első emeleten vagyok.
He fell from the fifth floor. = Leesett az ötödik emeletről.
I guess you've already been told about this, but I'm repeating it here: Hungarian floors work like British ones. So usually when you enter a building, you're on the groundfloor (földszint), above which there may be a first floor, second floor &c (első emelet, második emelet stb.)
Not the most common word order, but definitely correct. I can imagine it being followed by "...de tovább nem" -- "...but not any further"