"It is neither fast nor slow."
If my understanding is correct, おそい is an adjective and おそく is an adverb. This being said, おそい describes something (such as in "the car is slow", in which slow is the adjective). On the other hand, おそく describes the verb (such as in "the car drives slowly", in which slowly is the adverb). It appears as though the negative form is just an exception and uses おそく instead of the expected おそい, since it can be thought of as being the adjective for ない.
you can do this with contextual particles (も、は) because they are not considered grammatical case markers like が or を, so basically they can appear in weird places like this.
You can also say that the word 遅くない is not really a full word, is a combination of the connective form of 遅い and ～ない which is a pseudo-adjective, usually the inverse of the verb ある、translated as "It is not". So saying 遠くもない is like this:
「遠くも」"even the concept of fast"
「遠くもない」"even the concept of fast, it is not"
「速くも、遅くも、ない」"even the concept of fast, even the concept of slow, it is not"