Could mi be used in place of dhomh in this context?
No. mi only works as a subject or a direct object; never as an indirect object. You need the preposition do to, for for that.
So dhomh literally means "to me"? Give your hand to me?