Research on asthma frequently recruits patients from clinics because the ready pool of patients leads to easy access to patients in office waiting areas, emergency departments, or hospital wards. Patients with other chronic conditions, and with mobility problems, face exposures at home that are not easily identified at the clinic. In this article, we describe the perspective of the community health workers and the challenges they encountered when making home visits while implementing a research intervention in a cohort of low-income, minority patients. From their observations, poor housing, often the result of poverty and lack of social resources, is the real elephant in the chronic asthma room. To achieve a goal of reduced asthma morbidity and mortality will require a first-hand understanding of the real-world social and economic barriers to optimal asthma management and the solutions to those barriers.