Can אלא also mean 'but rather' - could this sentence be translated as 'It is not fear, but rather hatred'?
Yes, that's a perfect translation, and helps differentiate אלא from אבל.
Yes, it's like "sino" in Spanish
Yeah: sino = אלא, pero = אבל.
Or like "sondern" in German.
Ze lo pachad, ela sin'a