I'm still very newborn in russian language, but as I understand so far the difference is the "gender" of things. Моя stands for my femenine things [e.g. моя машина] and мой for masculine things [e.g. мой папа]. I hope I nailed the correct answer and thus it can help you or someone else.
Моя = my (feminine), мой = my (masculine) in the first person nominative case. There is a nice chart on http://www.russianlessons.net/grammar/pronouns.php
It's not really a case of 'adding' it, it's part of the spelling. Occasionally in some cases (genitive plural comes to mind) you might replace a vowel with a soft sign, but otherwise it's just part of the word. In English, we have lots of letters that seem extraneous; in Russian it does at least have a function! ;) It softens/palatalises the consonant it follows. But basically, you learn it as part of the word.
You will find that most (though not all) verbs end in the мягкий знак when in the infinitive, so it can be handy in determining that an unknown word might be a verb. Endings like ить ать овать etc are often (though not always) verbs.