Perhaps a silly question: If I am not mistaken, łazienki is the genitive. Does potrzebujemy always require a genitive object?
Yes. There are some verbs that always go with genotive.
It's another case that's not difficult understand in English because you can think of potrzebujemy as meaning "we have need of".
"Dużej" doesnt mean bigger? Or am i making a mistake with some grammatical declination or sr else?
"dużej" is Genitive of "duża".
But you're not totally far, because -ej ending is used for the comparative form of adverbs.
"We need a big bath" - why not?
"bath" is a bit too ambiguous, I think, since it can refer to a bathroom (łazienka), a bathtub (wanna), or an act of bathing (kąpiel).
And it can also refer to a public bath – „łaźnia”.
Łazienka is a bathroom in a house?
Yes, but I'd say it can easily be used for a toilet as well.