does elmegyek only mean going away, or should going out or going over be accepted too. going away to the horse seems awkward to me. I can imagine going away on the horse.
I would just say "going", without "over", without "away". The preverb "el-" here really only serves as an indicator of a complete action, goal will be achieved, I will see the horse. That is all. So,
"Yes I am going to (visit) the horse!"
Like vvsey said, completeness of the action. And sounds a bit better.
Elmegyek a lóhoz. - I am going to the horse and I'll definitely get there. I really want to pet it. Also has a neutral word order, stating what you're doing.
Megyek a lóhoz. - I am on my way to the horse. Not sure whether I'll actually arrive or if the sheep ends up being much cuter. Has a bit of a strained word order, emphasising that you're walking to the horse.