"What is the continuation of the book?"
Translation:Qual é a continuação do livro?
Is it wrong to use "O que" instead of "Qual" for the sentence "Qual é a continuação do livro?"
Yes, it is wrong. Basically, "o que" = "what" and "qual" = "which". E.g.:
Qual desses você quer? (Which one do you want?) Não entendo o que você quer dizer. (I don't understand what you mean.) O que você quer? (What do you want?)
But we have a few exceptions. E.g.:
What is your name? = Qual é o seu nome?
I don't know how to explain it to you widely. It's one of those things you learn practicing.
For what it's worth, the Modern Brazilian Portuguese books says to use Qual when you are asking for something from a range of possibilities (Qual e o seu nome / what is your name), and use o que when asking for a definition (O que sao celulas-tronco / what are stem cells.