Can be confusing.
First, remember that две/два requires a genitive singular.
Second, remember that in all singular cases of окно́, the stress is on the second syllable. So the genitive singular sounds окна́. The English ear hears two 'a's, I suppose.
For the sake of completion, note that all cases of the plural have the stress on the first syllable. So the nominative plural may look identical to the genitive singular, but it sounds о́кна. You hear 'o' and 'a.'
The number 2 has a collective numeral form, «дво́е», which looks very similar to dvaja. While you're not required to always use it when talking about animate nouns, it doesn't work too well with inanimate nouns (except for pluralia tantum nouns). There are also some other restrictions, like you can't use it with feminine nouns, although the collective numeral «оба/обе» works fine with them.
«Два учи́теля» and «двое учителе́й» are very similar in meaning, but there is a subtle distinction in whether you view them as a single group or as a collection of individuals (hopefully that makes sense).