"Answer to" is generally used when we are talking about issues of accountability. That is to say, for example, I as an employee answer to my boss - I am accountable to him, and I must be ready at any moment to give him a rundown of everything I have done through the day. Another example would be a politician answering to his voters for why he behaved in a particular way.
However, when we are just talking about general questions, we can just say, "Can you answer me?"
Edit: I had initially approached my answer to you from an English grammar point of view. However, in thinking this over, I can see how the Russian construction of using мне, the dative/indirect object, would lead to an English translation of, "answer to me." In that vein, you could use "answer to me" while not implying accountability but simply an answer to a question; but you would say a much wordier, "Could you give an answer to me?" which would still keep the spirit of the Russian dative usage. In English, for simplicity's sake, I would stick to, "Could you answer me?"
This seems to be the trickiest part of English. I had a foreign co-worker that spoke perfect English, except that he would always ask, "Could you reply my email?" I had thought about trying to politely let him know (because he was the type of person that would want to know, if they were saying something wrong, so he could correct it). I started thinking about it and I realized I don't actually know the rules either (native speaker). For example, why is it "answer my email", but "reply TO my email". I had to look it up and it has to do with verbs being transitive or intransitive. Unfortunately, the only way to know that is to memorize it.
Отвеча́ть is imperfective. Отве́тить is perfective.
Using отвечать would lend a flavor of requesting a repeated or continuous answer, kind of along the lines of, “Can you be answering me?”
Using ответить highlights that the person speaking the sentence would like a single result if вы можете: an answer.
Yeah, more or less.
Отвечать мне, ему, ей, Ване, маме. (dative)
Отвечать требованиям (to meet the requirements). (dative)
Отвечать на вопрос, на звонок (answer the call), на оскорбление (respond to an insult). (accusative)
Отвечать за пожарную безопасность (be responsible for fire safety), за последствия (answer for the consequences), за свои поступки (take responsibility for your actions). (accusative)
Отвечать смехом (to respond with a laugh), взаимностью (to return smb.'s feelings), вопросом на вопрос (to answer a question with a question). (instrumental)