I'm a native English speaker and have never differentiated between the two.
Now that you mention it, it might be kinda tricky to explain. The rough explanation is that jackets are rather short, more casual, while coats are more longer and might be more elegant. Try doing a Google image search on „płaszcz” and „kurtka”.
Jackets are lightweight, coats are heavier. Whether or not one thinks or cares about the difference between the two in everyday life probably depends on how cold it gets in the winter where they live and whether or not they're doing a foreign language exercise. :)
Both płaszcz and kurtka are "outdoor clothes" , both can be very casual, sport or elegant. But kurtka is short and płaszcz is long, so when in doubt long "kurtka"/ short "płaszcz(yk)" more elegant garments are called płaszcz, while sport ones are "kurtka" casual - let your clothes producer decide :)
I am a native Polish speaker aand i have never said "Płaszcz" meaning coat. I always said it like it would mean "robe" in English. Otherwise i'd say "Kurtka" would be a coat
Robe? In American English it is strictly "szlafrok" = "podomka". I did not even hear it in the meaning of "szata".
On the app I had to put together "a overcoat". Should it not be "an overcoat"?
I guess Google Graphics may be the best way to see it. Both seem acceptable for "płaszcz", although I'd go with 'overcoat'.
W innych kursach jest fajne rozgraniczenie: kurtka -> a coat [lub a jacket] (zależnie od kontekstu), a płaszcz -> a longcoat (a overcoat, czasami pokrywa się z płaszczem, a czasem z marynarką)