That would have to be plural, since we don't use nouns in the singular without an article like that. You don't in English either; you wouldn't say I haven't answer or I don't have answer.
So if you'd say Jag har inte svar, that would be like I don't have answers or I haven't answers in English. It's a bit strange in both languages: you might find a context where it would work, but usually we would prefer to say Jag har inga svar = I have no answers or Jag har inte några svar = I don't have any answers.
"Jag har nej svar." is incorrect, yes. You need to see, that there are two types of "no"s. 1. no = nej: That type of "no" is able to stand alone in a sentence and is mainly used for small talk. For example "Are there any eggs left? - No." 2. no/none = ingen/inget/inga: This "no" is another form of "none". It describes the amount of something. For example "How many eggs are left? - None. - What, there are no eggs left?" I hope you get my point, since I'm neither a native English speaker, nor a native Swedish speaker xD If someone thinks I'm wrong, please tell me.
For anyone else having my problems:
svar: answer (verb form: svara)
svår: hard, difficult; svårt for ett-words
svär: Swear (both senses of "I swear it's true" and "He was swearing and cursing"). Also used as a prefix in, for example, svärmor (mother-in-law).