It can mean both. For example "to have" in Swahili, is "kuwa na", (literally "to be with"). In present tense it is shortened to e.g. "Nina rafiki" (I have a friend, or literally, I am with a friend).
You would not use the word "all" in english in this context so it should not be marked wrong if you don't include it
"You aren't playing with me" not accepted. The correct answer, "You are all not playing with me", is not how I would express myself.