Wrist actigraphy (ACT) may overestimate sleep and underestimate wake, and the agreement may be lower in people with chronic conditions who often have poor sleep and low activity levels. The purpose of this systematic review is to compare the agreement between ACT and polysomnographic (PSG) measures of sleep in adults without chronic conditions and sleep complaints (healthy) and with chronic conditions. We conducted a systematic review and meta-analysis using PRISMA guidelines. We searched PubMed, OVIDEMBASE, OVIDMEDLINE, OVIDPsycINFO, CENTRAL, CINAHL, ClinicalTrials.gov, International Clinical Trials Registry, and Open Grey. We included 96 studies with a total of 4134 participants, of whom 762 (18.4) were healthy adults and 724 (17.5%) were adults with chronic conditions. Among adults with chronic conditions, ACT overestimated TST, compared to PSG [M = 22.42 min (CI 95%: 11.92, 32.91 min)] and SE [M = 5.21% (CI 95%: 1.41%–9.00%)]. ACT underestimated SOL [M = −7.70 min (CI 95%: −15.22, −0.18 min)], and WASO [M = −10.90 min (CI 95%: −26.01, 4.22 min)]. These differences were consistently larger between ACT and PSG sleep measures compared to healthy adults. Research is needed to better understand factors that influence the agreement between ACT and PSG among people with chronic conditions.