Genetic risk for coronary artery disease (CAD) is commonly measured with polygenic risk scores (PRS); yet, the relationship of atherosclerotic burden with PRS in healthy individuals not at high clinical risk for CAD (ie, without a high pooled cohort equations [PCE] score) is unknown. Here, we implemented a novel recall-by-PRS strategy to measure coronary artery calcium (CAC) scores prospectively in 53 healthy individuals with extreme high PRS (median [IQR] PRS = 94% [83-98]) and low PRS (median [IQR] PRS = 3.6% [1.2-10]). The high PRS group was associated with a 2.8-fold greater CAC than the low PRS group, adjusted for age, sex, BMI, smoking, and statin use, and had a 6.7-fold greater proportion of individuals with CAC exceeding 300 HU. These findings reveal that extreme PRS tracks with CAD risk even in those without high clinical risk and demonstrate proof of principle for recall-by-PRS approaches that should be assessed prospectively in larger trials.