I am not sure what you mean by "infinitive case". Can you please elaborate or give a few examples?
Any chance you are referring to the continuous or progressive tenses?
Yes, Hungarian has only simple present and simple past, the progressive sense is not expressed grammatically.
"To walk" - is the infinitive. The Hungarian equivalent is "sétálni". The "-ni" suffix makes it the infinitive.
I like to walk. - Szeretek sétálni.
"I walk" - is the simple present tense.
I walk. - Sétálok.
"I am walking" - is the present continuous or progressive tense.
I am walking. - Sétálok.
Yes, these last two are not grammatically separated in Hungarian. There are other ways to express the difference, when needed.