"Did you visit it?"
When the object - it, in this context - isn't specified, how do we know which object infix to use?
Because the verb "to visit" relates to a place and the object infix for place (mahali) is usually "-pa-" (a definite place). It could also be "-ku-" (an indefinite place), but because it can then be confused with the object infix 2nd person singular it is not used very often. The object infix "-m-" (for an inside place) is almost never used.
... in which case you would use the object infixes for persons: -ni-, -ku-, m(w)-, -tu-, -wa-, -wa-. Although in this example it would be funny to combine the verb visiting with the first person singular.
Yes, but you wouldn't call people 'it', so that suggests this sentence is not about people. But I'm not sure that the word couldn't take another object prefix, like if you visited a statue or a grave. I would hardly call those "places", at least not in the strictest sense. I don't know how it is in Swahili though.