How formal/informal is "doei"? I was offered "later", but it was marked incorrect.
It's generally considered very informal, even moreso than jij'ing somebody. I personally don't think it's quite as bad as later, but I still wouldn't end a business call with it. People can take a lot of offense to it.
But, just as with jij vs. u, it really depends on the situation. When in doubt, dag is always safe. :-)
It is considered to be pretty informal, but, like with most things in the Netherlands, the way you say it is essential. I personally never use 'doei', except sarcastically. In my personal opinion the word sounds a bit rough. In informal settings 'doei' often gets turned into the somewhat softer sounding 'doeg'. To be polite I often say: "Nog een fijne dag gewenst" or "Nog een fijne dag verder" (I wish you a good day), that usually does the trick.
I wouldn't use it to end a business meeting, but it's a common way to say bye to your friends. I don't understand why people would find it rude.