"Finally they open the synagogue."
Translation:סוף סוף הם פותחים את בית הכנסת.
They're close, but they're not exactly synonyms in my opinion. לבסוף means something like "in the end" - and if you write it in this sentence instead of סוף סוף it sounds like you're talking about something that already happened. It can also be used as "finally" in the sense of "firstly, secondly.. finally" when writing an essay. סוף סוף means finally as in "after a long time", so it fits a lot better here.