Sympathy for English speakers here please. "Pants" in English means something completely different from "pants" in American. Surely "pantaloni" really means "trousers", doesn't it? Pants = underwear in English English.
The English should be: "You seem/look/appear uglier IN pants." "With pants" is not English (unless what is mean is that the person is holding a pair of pants, not wearing them, and that makes him or her look uglier).