To assess whether the US News and World Report (USNWR) Urology specialty ranking methodology accurately captures and classifies complications following elective outpatient urology procedures.

We conducted electronic health record chart review of n = 80 elective, outpatient urology procedures with complications from 2019-2023 across 4 hospitals in our integrated US health system. We used the Solventum AM-PPC software and USNWR methodology to determine eligibility and measure complications. For each complication identified by the software, we assessed: (1) whether the procedure was performed by a urologist; (2) whether the adjudicator agreed with the complication type; and (3) whether the complication was a clinically related sequelae of the index procedure. We reported Clavien-Dindo severity of each complication.

Our adjudication agreed on complication type in 62/80 (78%) complications, and 64/80 (80%) complications were clinically related to the index urology procedure. Combined, 57/80 (71%) complications were concordant on both complication type and clinical relatedness. However, 38/80 (48%) index procedures were conducted by interventional radiologists, not urologists. Furthermore, 11/80 (13.8%) complications were false positive urinary tract infections (UTIs).

The USNWR methodology for elective outpatient urology procedural complications showed reasonable clinical validity but detected several false positive UTIs. Further, USNWR should clarify the extent to which procedures performed by interventional radiologists belong in urology rankings.