Caltech has two forms of off-campus housing. There is what students call "off-campus housing," which implies living off the Caltech campus but still on Caltech property. Caltech owns much of the property around the school, and since there's guaranteed housing for undergrads and not enough spaces on campus for all the students, they lease or rent adjoining properties to students. These properties include houses and apartment buildings. The school has a housing lottery every year. The lottery includes all school housing, including Avery and Marks, but not the seven houses, because of Rotation which is like a residence hall version of frat rushing. Caltech is responsible for utility bills, Internet connections, and cleaning service in off-campus housing. This is a very convenient way of living near campus, without the actual dorm atmosphere, but with all the advantages of school housing where there is only one bill to pay.
The other mode of off-campus housing is referred to by students as "off-off-campus housing." When a student lives off-off campus, he or she is not living in Caltech-owned property. Not very many students live off-off campus because Los Angeles is expensive, and even a two-bedroom apartment can be extremely overpriced. For those who want to live off-off campus after freshman year (students are required to live on campus for the first two terms, and they usually just stay the third term to finish off the year), there are always graduating seniors who will leave behind empty apartments and owners of apartment buildings who are usually accepting in replacing one Caltech lessee with another.