Being a bit of a nerd I wrote a script to check this hypothesis ;-). It mostly holds up. These are the exceptions that I've found (database of 20000 words):
gift, fett, finger, garn, land
oh, and all latin words ending in -um:
akvarium, auditorium, decennium, diarium, imperium, jubileum, kollegium, kompendium, kranium, krematorium, kriterium, laboratorium, medium, museum, mysterium
For the Latin loanwords, no. Words that end in -ium or -eum drop the -um and add -er instead. So akvarium -> akvarier and museum -> museer for example.
For Latin loanwords that end on just -um, the plural works the same as with other ett-words,, i.e. it's the same as the singular form but without indefinite article.
A few people prefer the latin -a plural instead, but it looks very snobby and a bit unnatural. After all, we speak Swedish with loanwords, not Latin.
"Garn" and "land" are only half exceptions. They each have two meanings, and one is different in plural, the other the same.
Garner is plural of yarn for knitting, garn is plural for a type of fishnets.
Länder is plural for countries, land is plural for garden patches etc, similar meaning to land in English.
It shouldn't be too easy :-)
It is a very simple python script that uses the publicly available LEXIN dictionary what has inflection information. Checkout the notebook if you want:
That's how I've learned it: a) ending in a consonant - Ett hus (a house), huset (the house), hus (houses), husen (the houses) b) ending in a vowel - Ett äpple (an apple), äpplet (the apple), äpplen (apples), äpplena (the apples). At least this is the basic rule. I don't know if there are exceptions in ett-words. (Some en-words certainly have irregular plurals)
Edit: Corrected 'the houses' -> 'husen'