I agree, but if you play around with the emphasis I think "...speaks only..." can be made palatable. I wouldn't go so far as to say it sounds completely wrong.
I also agree that "only speaks" sounds more natural. However, "speaks only" has the same meaning and seems completely acceptable to me.
Before I thought about it I almost put "speaks only" since it is the same word sequence as the original sentence.