The accusative endings comes only to the definite objects. You can think of it as the definite article "the" but it's only used with the objects.
In this sentence, it's not a specific tea but any tea in uncertain quantity. Just like you don't say We drink the tea in English, you don't need the accusative. Also the aorist tense suggests it's a habitual action so we are talking about tea in general.
However, if you cooked a specific type of tea or talked about that tea before then you would say "Biz çayı içiyoruz." -> We are drinking the tea. A specific tea that we talked about before.