I would rather say "The towns are not small" and not "The towns are not little".
We tend to think of cities as larger than towns. More importantly, we expect cities to have more amenities such as theaters, restaurants, a transportation system, an active business district etc. Cities usually control their municipal services such as police, fire fighting, schools, etc. The legal definitions vary. In Spanish, we say "ciudad" for city and "pueblo" for town (and sometimes for village).
In the US, cities are usually chartered by the State Legislature and have greater autonomy than towns. Towns are subdivisions of larger territories called counties and may depend on the county for many of their services. Town residents pay taxes to the town and the county. City residents pay taxes the city. Some states have somewhat different structures but this is a general outline of the difference between towns and cities in the US.