I am afraid I don't agree with Paul. As a native english speaker i would use either to write or writing in this context to mean the same thing. "To write" together with "began" is in general usage in the UK.
Oh, ok, so "start" can be followed by either infinitive or gerund with no difference in meaning? Good to know :-).
if you want to use the verb start, then it should be: "When did he start WRITING that book?"
Sorry began to write is accepted. I noticed a t had found its way in front of he in my answer and i didn't notice it. Sorry Paul I also misread your post. Put it down to the vagaries of old age.