I'm not 100% sure, but as a translation to the sentence, probably not. I think they're talking about Duolingo the bird (their mascot), not the website. So it's as if you're learning it alongside the bird.
But if you want to say you're learning it via the website, then I think that's fine
Hallo! "studieren" is more like majoring in a subject at a university while "lernen" is more general purpose.
http://www.thegermanprofessor.com/studieren-vs-lernen/ is a good reference
"Mit" is together with something, or by means of (ich fahre mit dem bus, i am going by bus). "Bei" is more like near sth, at some place (wir sehen uns bei dem konzert, see you at the concert, ich bin bei meinen eltern, i am at my parents'), or if you are in posession of something (ich habe mein handy immer bei mir, i always have my phone with me).