"Come to me" also works
And it is accepted.
"Come toward me." is identical to "Come towards me." and should be accepted.
This alternative has been added, as Ngrams agrees.