One can think of it as the distinction between "to speak" and "to talk". As in English, there are both similarities and differences between the two.
Here are a couple webpages that might help:
In German, we prefer to talk "with someone" - it feels more like a conversation than talking at someone or "to someone" which sounds more as if you're just throwing your words at the person and not caring about what they say :)
Er spricht zu uns brings up in my mind a public speaker - who speaks "to the crowd" but not with them, not an actual conversation.
In English, you can talk to someone or talk with someone. It can have the same connotation. If I say "I had a talk with him," it implies that there was a discussion. If I say that "I talked to him about it" there's more of an indication that I wanted to get my point across.