This is just my opinion.
"Niet doen" is a general correct way of translating this, it means correcting, stopping or warning the person at the time they are doing something you don't like or if they are not supposed to do it.
"Doe niet" Is the literal translation of "do not" (which are joined in English as don't) , in Dutch there is no combination/joining of these words like that. It is not often used in speaking or writing as far as I know.
"Doe het niet" is more of a sound advise after someone makes their intent known and the subject is already really clear beforehand. Like "I am going to steal from this shop". The "het" in this translation points to a subject. So it is literally "Don't do it"