"You and me" is only correct when "we" are the direct/indirect object of the sentence; when "we" are the subject, it has to be "you and I".
"You and I punched a gorilla. The gorilla punched you and me back."
If you take out the word "you", you get:
"I punched a gorilla. The gorilla punched me back."
Removing "you" from your sentence makes it "Me am leaving" instead of "I am leaving".
That being said, "You and me are..." is just about the most common grammatical error I can think of, and doesn't sound the least bit strange when I hear it in day to day speech.