Am I correct to think that "mit Zucker" is dative? I'm trying so hard to get a grasp on these cases. It's making me crazy.
Yes. Mit goes with the indirect object, and is always dative.
«"Mit das" gibt es nicht», as I am always told. ;-)
Why isn't it Unsere Tee ?
Because "Tee" is masculine, and as it is under accusative, you should add -en to "unser"
Why cant I say: Wir trinken unseren Tee mit dem Zucker? Aus, bei, mit, nach, von, zu is connected with Dativ (der, des, Dativ - dem, den). Is it then like with that sugar?
"dem" is unnecessary.