Kone og barn / Mann og barn are set phrases, and it's assumed you mean your own children/kid and spouse. The spouse part is not up for questioning, but even if it is a known fact that your spouse brought a child of his/her own into the relationship, I think most would use the same phrase. As it's the people in your household you are referring to. If you are in fact a part-time dad/mom, saying you are going home to kone og barn will just be a way of letting your listener know you have the kids home as well as your wife.
So, yes. Context. But feel free to ask if you wonder, asking about stuff like that could be considered a bit impolite and an invation of the personal sphere, but that door is wide open after letting out the "kone og barn".
The possessive is implied here, but if you were to write it out it would have to be in one of the following combinations: "sin kone" "si kone" "konen sin" "kona si".
The reason you would use "si/sin" rather than "hans" is that the possessor is the subject of the sentence ("han").