ej means the same as inte (= not) but it's hardly ever used in speech, only in some cases in formal texts and on signs.
Yes. Sentences start with a capital letter and names are capitalized (well, except for the kind of names that aren't supposed to be capitalized, like the von in Carl von Linné). In English for some reason you also capitalize the words for languages, nationalities, days of the week, months etc, but in Swedish we don't (meaning that if you want to write correctly, it is wrong to capitalize those).
In informal writing you can of course ignore all that and write for instance emails in all lowercase, or angry notes to your neighbors in all uppercase. :)
In this course, the spellchecker disregards capitalization and punctuation, so you don't need to use it here.
Are exclamation marks used a lot in swedish? Or is this just duolingo's thing?
I wouldn't say we use it more than any other language, so I think it's just Duolingo.