I think it could also be translated as "Other men don't interest me.". Do you agree?
Se is in the second position here. In the first position is the whole phrase "o další muže". This phrase can§t really be divided, it is just a preposition and an adjective with the main noun.