What's the best way to configure my keyboard so I'm not coming up to the mouse every time I need an umlaut?
Just type "e" after the letter with the umlaut! That's how umlauts are spelled out when you don't have access to the correct keyboard (my wife writes like this when emailing her parents from her USA laptop, etc). Ü is ue, Ö is oe, Ä is ae. Duolingo accepts this without any problems.
In X, you can use xmodmap to add AltGr and dead-key combinations. In Windows, you can do the Alt+code method, or you can download a driver to enable AltGr (I have a Dvorak+German driver for those rare occasions that I boot into Windows). For other operating/windowing systems, check the documentation.