I said "A cute, book-reading boy" but I guess a cute boy reading a book makes sense.
This is a really great question though and something I really struggle with, so I'm going to explain it further. Sean is correct. I think of it like this: Adjectives work the opposite way in Korean. To be brief, in Korean when you look up descriptive words (adjectives) in the dictionary (so in their standard form) they are always verbs. In Korean, the "to be" is always attached. However in English, these descriptive words are just adjectives already. If you want an adjective to be a verb, you need to add "to be". SO in Korean the default is "to be cute" and in English the default is "cute".
With this in mind, it makes sense that if going from adjective -> verb form requires adding something in English, the reverse is true in Korean. To take a descriptive verb (to be cute) into an adjective (cute) we must add something. In Korean it is called a modifier. The Korean adjective modifier is ~(으)ㄴ. So 귀엽다 (to be cute) becomes 귀여운 (cute).
In your sentence "The boy reading a book is cute" You have used cute in its "to be" form which would be 책을 읽는 귀여운 남자아이 귀여워요. Do you see the difference? :)
The last thing I will add is you can use this adjective modifier with action verbs as well, which is what is used in the first half of the sentence "책을 읽는" this is literally making "book reading" an adjective. You could think of this sentence "책을 읽는 귀여운 남자아이" as literally "The cute, book-reading boy", as mentioned above in an earlier comment. But if this last paragraph confuses you, ignore it. Hope this was helpful.
Talk to me in Korean Level 3 lessons 13 and 14 cover adjectives in Korean for both descriptive verbs and action verbs.
That's a different word order where "cute" becomes a descriptive verb instead of an adjective:
책을 읽는 남자아이가 귀여워요/귀엽습니다. The last word means "is cute".
"A cute boy is reading a book" is wrong? I only added "is", but surely that's not actually altering too much?
It is altering too much, since there is no verb in this exercise. Your suggestion would be: 귀여운 남자아이가/는 책을 읽고 있어요. [읽고 있다] is the present progressive "to be reading".