why isn't 'talked with' accepted?

I think 'spoken to' should also be accepted, since the meaning is Kind of the same. There is no actuall difference between talking with someone and talking to someone, is there? If I'm wrong, please correct me.

Would "he had never spoken with my sister" work for this?

Why not "talked to"