हाथी is a masculine noun. Whatever rules you might have read to predict noun-genders based on word endings are only rough guidelines and all of them have many exceptions. My suggestion is that any time you come across a new noun, learn it along with an adjective (like बड़ा कुत्ता, बड़ी बिल्ली). This helped me a lot while learning French.

The plural form '-iyan' is only for feminine nouns ending with -ī. Since हाथी is masculine, its plural form remains हाथी