I forgot the rule - what do I do in the case of "I give it to him"?
Is it "Yo se lo doy a él"?
If the object you are giving to him is a masculine noun (for example, a book), you use lo. "Yo se LO doy a él (EL libro)"
If the thing is feminine (for example, an apple), you use la. "Yo se LA doy a él (LA manzana)"
But you never ever use "le" in this case ("Yo se le doy a él" doesn't exist) If there is no conext, and you don't know what are you giving to him, just use "lo".
Hope I'm clear, let me know otherwise.
Yes, the confusion was about if you say "Yo le lo doy a él", because it would be "Yo le doy un libro a Martin" (I give a book to Martin).
I see, I misinterpreted your question. But yes, you're correct, it is "Yo se lo doy a él", not "Yo le lo doy a él". Direct/indirect objects in Spanish can be kind of tricky at first, and in your example you have both together in the same sentence. This page explains that particular case better than I could do in case you're interested. http://www.rocketlanguages.com/spanish/resources/direct_indirect_object_pronouns.php
Your sentence is correct. You could shorten it even further to "Se lo doy" and mean the same thing, but it is common to keep "a él", since se is so ambiguous.
One thing to add to ryaton's comment about "le" -- le and se are indirect object pronouns, i.e. they substitute "to him". Use le if there is no direct object pronoun (lo), but change it to se if there is.
Yo le doy el libro a él Yo se lo doy a él