I will absolutely grant that "whom" is becoming less common in casual speech, but I maintain that grammatically "whom" is correct. It is the object, not the subject, and the preposition you cite is still in the sentence. It's still "you" talking "to whom," no matter what order you arrange the words in.
While colloquial English often drops the "whom," if I am learning a language, I want to learn what's correct, not what will pass in many circumstances. :-)
Historically, 'whom' is the correct form, describing the accusative (object-case) of 'who'. Like 'him' is the accusative of 'he'. But it's increasingly less used, and it's generally accepted that you use 'who' for whichever case, so do as you like.
Just don't say "whom are you?" or the Boogieman will come for you.
In your sentence, the 'you' is the object ('who' did the talking), but in the sentence above it's the subject ('you' did the talking). The difference between nominative you (subject) and accusative you (object) isn't apparent in English, but it is in Danish: du as subject, dig as object.
"Who talked with you?" would be "Hvem snakkede med dig?"