As I understand it, "anniversaire", by itself, really does just mean "birthday"; no one would imagine you mean anything else. But if, as you say, you wanted to be very precise, perhaps you might say "l'anniversaire du jour de ma naissance" ? (Not a native speaker, just guessing here)
For a francophone, the word "fête" is sometimes used to sound like it's a "birthday", but it really refers to what is called a "saint's day" or "name day". In some catholic traditions, days are organized according to the names of different saints. When the day for "Saint Gustav" comes up (assuming there is such a day), that would be your "fête". Some celebrate it almost like it is a birthday, but it is really your "name day". "Fête" also means party or celebration. But "anniversaire" means "birthday" (not a birthday party) and also "anniversary" when it's used with a modifier, e.g., anniversaire de mariage (wedding anniversary).
The 1st translation for birthday is "anniversaire", ie the date of your birth day and it occurs every year. But if someone wants to celebrate it and invites you to its party, usually you would say "je suis invité à l'anniversaire de..." and not "je suis invité à la fête d'anniversaire de...". So the word birthday can be translated either by "anniversaire" or by "fête d'anniversaire".
"Fête" can refer to a "saint's day" or "name day", a practice observed by many in France. In that culture, all the days of the year are organized according to the name of a saint. When the day comes around that has the same name as yourself, it is your name day or saint's day. It is celebrated in the same way that others celebrate a birthday, but it is not your birthday. https://en.wikipedia.org/wiki/Calendar_of_saints