to teach in Finnish = å undervise på finsk
to teach Finnish = å undervise (i) finsk

The former would mean that the subject was being taught in Finnish, and the latter would mean that the subject being taught was Finnish.