I think "that's not right at all" and "that's not entirely correct" (which is what I put) are two different things. The first one implies that whatever "that" is (an answer, for example) is completely wrong. The second one implies that a portion of the answer is correct, but a portion is also wrong.
When you say that something is not really correct you are usually actually saying that it is totally wrong, although you understand why the person thought it might be right and you don't want to hurt their feelings. However the sentence in this exercise is saying that something is mostly correct but not entirely correct.
i don' t think it is correct in English (at least not Australian English) to describe something as "very" correct. It is either correct or it isn't.... though one could say "not quite correct" if we are encouraging someone to come up with an answer closer to the required answer.
Actually, correct can have different degrees, as this sentence shows. (you can be 75% correct, partially correct, not entirely correct). "Very correct" is correct, although not very used.