Sluta is only transitive in very few situations such as "sluta skolan" (Finish school). In this case "sluta" cannot be transitive, but you could say "Han vill sluta läsa boken".
If he wants to give up the book I would suggest using that very sentence, i.e. "Han vill sluta läsa boken"
I just want to add that it seems like English "quit" is the same way. "He wants to quit work", "He wants to quit school" are the only two that come to mine. Similarly to "sluta", you can't say "He wants to quit the book", but you can say "He want's to quit reading the book"
I think (though I'm not positive) that it's the same distinction between "to stop" and "to finish." I.e., "He wants to stop smoking" vs. "He wants to finish his cigarette.". Or here, "he wants to stop reading" e.g. for the night, vs. "He wants to finish the book" as in, he wants to complete it.
It's interesting. I can almost see the commonality, after looking at each one. https://en.wiktionary.org/wiki/Category:Swedish_words_prefixed_with_av-