Coming and going is all from the perspective of the speaker. If I am with you and we plan to go to another place, I would say "You go with us." If I am in another place from you, but I want you to come to us to be with us; then I would say "You come with us." even if you are just across the room from us or simply not standing next to us. It can get confusing, because when someone asks "Are you coming?" we usually answer "Yes, I'm coming." rather than "Yes, I am going.".
"Are you coming now?" could be answered with "Yes, Í am coming right now." (using the perspective of the person asking the question) or "Yes, I am leaving now. I should be there soon." or "I am going now! I will see you as soon as I can." (both from the perspective of the person answering)