Those are the Hanja from which the word is derived, yes, but the word doesn't refer to sweet potatoes in Korean. Those are called 고구마. Sometimes loan words don't mean the same thing as in the original language, for a number of reasons:
- Maybe there was a misunderstanding during the transfer and the loaning speakers thought it meant something different in the source language than it actually did.
- Maybe the meaning of the word changed in the loaning language or the source language (or both) after the borrowing.
- Maybe the word wasn't originally borrowed as a whole at all but only it's components were. And later both languages independently combined the components to a new word, but with different meanings for the result.
Or any combination of those.
For potatoes (including sweet potatoes) it is also important to remember that they are native to the Americas, so they only arrived in Asia some 400 years ago - long after the major borrowing waves of Sino-Korean vocabulary. So in this case I suspect that 3. played a role.