Is egy necessary here? I really have a hard time with pronouns, ie wheter or not they have to be there.
No, absolutely not necessary here. In fact, I would prefer it without "egy".
But you could say "I am only a man". Then it is more natural to say "Én csak egy férfi vagyok".