The efficacies of various exercise modalities in improving older adults’ sleep quality remain unclear. Therefore, in this study, network meta-analysis was performed by comparing the efficacies of various exercise regimens in improving sleep quality in this age group. Six electronic databases were searched for relevant studies between the date of database creation and August 13, 2021. Only randomized controlled trials reporting the effects of exercise on sleep quality in this population were included. Random-effects network meta-analysis based on a frequentist framework was conducted. In total, 35 trials involving 3519 older adults were included. Cognitive behavioral therapy for insomnia, muscle endurance training combined with walking, Tai chi, Baduanjin, resistance training combined with walking, and resistance training significantly improved sleep quality to a greater degree than did usual care (P < 0.05). Muscle endurance training combined with walking led to significantly higher sleep quality than did regimens involving sleep hygiene, Pilates, only walking, health education, resistance training, Taichi, resistance training combined with walking, or yoga. Muscle endurance training combined with walking was identified as the optimal exercise program (88.9%) for enhancing sleep quality in older adults. The results of this study support the claim that exercise can improve sleep quality in this population. PROSPERO registration number: CRD42020178209.