- Ela esperava mas ele não voltava
She was expecting, or she used to expect, she kept waiting, and "ele não voltava" stresses the "at every moment" checking over and over if he would come.
- Ela esperou mas ele não voltou: she has waited, but he hasn't come back.
Now, mixing both tenses in one sentence is kind of unusual.
I put She waited but he never returned which wasn't accepted. In spoken English it means the same as She waited but he did not return (in this context).
The difference was in the word "never":
"She waited but he never returned", it can be translated as:
"Ela esperava, mas ele nunca voltava."
"Ela esperava, mas ele não voltava." - original
I guess the difference of both phrases are the same as you said in English form, very subtle, but with the word "never", it's a more emphatic phrase
In DL's English sentence, "return" describes a one-time event.
She was waiting/waited, but he didn't return.
The simple past is a versatile tense which describes single, limited events as well as repetitive and habitual actions.