Duolingo is a tool so it makes more sense to learn Romanian "with" Duolingo. Learning romanian "at" Duolingo refers to a place.
Perhaps it refers to the website as a virtual place? (Even though "with" would make more sense.)
If it were "cu" that would make more sense, but with and on/at have a slightly different meaning. One is using it as a tool, and the other - like you said - refers to the site (as in where it's being learned).
Oh come on! "They learn Romanian language at Duolingo." is not accepted, with the correction: They learn Romanian at Duolingo... Seriously!