They should include a lesson on how to differentiate both, since in french most of the times they all sound the same or very similar. I made many mistakes putting something in singular when it was plural while doing lessons... Le chat / les chats sounds the same to me!
They sound quite different once you are used to hearing them. To me in my aussie accent, "le" rhymes with "the", while "les" rhymes with "heh". But I think I misunderstood your original suggestion- if you have trouble hearing the difference between le and les, then I agree, there definitely should be a lesson on that.
It does take a little getting used to, but it is possible to distinguish them, usually. Moreover, remember that when you're actually speaking French with someone, the context will usually make it clear whether they're speaking about one person (il) or many people (ils). It doesn't help much for the Duolingo lessons, but it'll help in real life.
I know what you mean and it is discouraging to get a question wrong because you thought you heard something different, I usually end up doing that especially recently with questions like "le chien vs les chiennes" what I would suggest is that you know what you meant and just keep practicing they usually ask the same questions so you know what they're asking for later in the lesson. (they repeat lessons)