In this sentence, "privat" is neither dative nor accusative - it's an adverb. Adverbs describe other adjectives (i.e. He was awfully nice) or verbs (e.g. He ran quickly). Here, the adverb "privat" describes the verb "sprechen": How do we speak? Privately.
In English, adjectives used as adverbs normally add the ending -ly. In German, adjectives used as adverbs remain unmarked, i.e. they do not get a special ending.
The case thing is only relevant for attributive adjectives, i.e. "Wir besuchten seine private Wohnung" (We visited his private appartment).
"We are speaking private." is incorrect English.
It would have to be either "We are speaking privateLY" because in this case "privately" is modifying the verb "speaking" which makes it an adverb needing the "ly". - As in "we are speaking badly, we are speaking quickly ..." - Adverbs frequently have an "ly" ending. Though not always. One would not say "we are speaking goodly" but "we are speaking well." But generally, adverbs are easily spotted by the "ly" ending.
If you did not want to use it as an adverb the sentence would have to be "We are speaking IN private." But I am not sure if Duo would accept this since it is a somewhat different sentence/meaning.