'так' means 'so' in russian so i would assume that the more literal translation would be 'something is not so'. Please correct me if I'm wrong.
Just like in Russian, «не так» in this context means ‘not right, wrong’.
Yes, «так» means ‘so’ (or ‘yes’ in Ukrainian) in some other contexts. But Duolingo doesn’t accept literal translations.
I find it interesting that tak can be used for "right". Is this the literal translation?
It's hard to say what's the most literal meaning. Both meanings are very common.
I think the original meaning was ‘this way’, like in «Я роблю́ це так» ‘I do it this way’, and the meanings ‘yes’ and ‘correctly’ appeared later. But I might be mistaken.