To use feminine - surely possible. To say "вибачаюсь" - well, that's not a preferable way to say that you are sorry. Ending "сь" makes the verb reflexive, so it's like you are trying to forgive youself. This form does exist colloquially, but I would advise against using it. You've just started learning the language, so why learn something that is not really right :)

Well, it is not a recommended form, AFAIK. The correct forms are «вибачте», «пробачайте», «прошу вибачити» and so on.

Thank you both! It is just amazing how fast one gets replies to questions in this course. And with an explanation :) I just can't stop studying... I was already thinking that it's time to stop for this evening but since I keep getting replies, I will continue for a few more lessons :D