"I do not have a sponge at my house."
Good question. Generally, the French look at it with logic.
A story about sponge(s) is rather neutral, because at the opposite, you could have one or several sponge(s) at home. Therefore, you are free to say "pas d'éponge" or "pas d'éponges".
Other cases could be more obviously singular or plural: - "je n'ai pas de voiture" should certainly remain singular, because otherwise you would have 1 - "je n'ai pas de gants" (gloves) should certainly be plural, since otherwise you would have 2