"Esperanto" as a language is capitalized only to differentiate it from "esperanto" - "one who hopes".
If you it look up in the on-line PIV, the entry for the language is "urdu/o", with a lower-case letter. Similarly with "amhar/o" for Amharic.
I tend to capitalise languages and similar cultural adjectives and nouns, at least sometimes, at the suggestion of Bertilo Wennergren, to indicate that it's a cultural term rather than a "core" part of the vocabulary - so if someone doesn't know what "li estas Buruŝasko" means, at least they can guess that it's probably a national designation rather than a word they might be expected to know. But that's not a must and many people don't capitalise such words. Though "Esperanto" is often capitalised.
On example where it may help is "Tio estas Kuba domo" (this is a Cuban house) vs. "Tio estas kuba domo" (this is a cubic house).