"What is the continuation of the book?"
Translation:Qual é a continuação do livro?
Yes, it is wrong. Basically, "o que" = "what" and "qual" = "which". E.g.:
Qual desses você quer? (Which one do you want?) Não entendo o que você quer dizer. (I don't understand what you mean.) O que você quer? (What do you want?)
But we have a few exceptions. E.g.:
What is your name? = Qual é o seu nome?
I don't know how to explain it to you widely. It's one of those things you learn practicing.