"This jacket is small."
Translation:Ta kurtka jest mała.
There are two nouns in the expression used as a definition, one is a subject and the other is a noun in a position of the direct object. The verb JEST/SĄ requires this noun (and adjective, if there is one there) in instrumental case:
A mouse IS a small animal - Mysz JEST małym zwierzęciem (instrumental)
In the sentence "This jacket is small" there is no definition. There is only one noun in it, "the jacket" (kurtka) in nominative, because it is the subject. The adjective "small" (mały, mała, małe) describes the jacket (the subject). The Polish adjective has to agree with the noun as to the gender, number, and case, so the adjective is feminine, singular, and in nominative case here:
This jacket is small - Ta kurtka jest mała (nominative)
Well, "żakiet" is an accepted answer, it's a piece of women's clothing (you may use Google Graphics).
My first guess would be that perhaps you used the wrong form of "this" ("ten" instead of "ta") and then the algorithm usually shows the closest answer... looking at the letters from the very beginning. If you used "ta", perhaps it was just a bug... that happens sometimes.
I'm not exactly sure if I understand the question - you do not "use" a female, male or neuter noun, the noun just is male or female or neuter. "kurtka" is always feminine and nothing will change it. Doesn't matter if it's Matt's jacket or Kate's jacket. I'm not sure if that is what you meant.
The rule of thumb is: look at the basic, Nominative singular form of the noun:
if it ends with a consonant, it's probably masculine
if it ends with -a, it's probably feminine
if it ends with -o, -e, -ę or -um, it's neuter.
There will be exceptions, but that should be your first assumption.