Actually it can mean both.
"Péter eljön tőled, majd eljön hozzám." -- "Péter comes away from you, then comes over to me."
I guess that in both cases "el-" means "away from the location the subject was originally at".
I do not understand the meaning of "come away." Is it: Peter goes away from you? Peter geht weg von dir? or Peter comes from you? Peter kommt von dir?