To investigate the diagnostic accuracy, inter-rater and intrarater agreement of grade predictions based on the visual appearance of papillary upper tract urothelial carcinoma (UTUC) during digital ureteroscopy.

Nine urologists predicted the histopathologic grade of 64 papillary UTUC (low-grade vs high-grade) by assessing the visual appearance of the tumors in videos from digital ureteroscopy. The diagnostic accuracy was estimated by comparing the grade predictions with the histopathology from colocalized biopsies. Inter-rater agreement was assessed by pairwise inter-rater percentage agreement and Fleiss Kappa analysis. The videos were rated in a random order again 30 days after the first assessment to evaluate the intrarater percentage agreement.

Low-grade tumors were predicted correctly in 37%-85% of the cases with a median concordance of 59% for questionnaire 1 and 66% for questionnaire 2. High-grade tumors were predicted correctly in 26%-91% of the cases with a median concordance of 52% and 61% for each questionnaire. The median pairwise inter-rater percentage agreement was 66% for both questionnaires with a Fleiss Kappa of 0.29 and 0.38, respectively. The median intrarater percentage agreement was 81%.

The histopathologic grade of UTUC is essential to the risk-stratification for treatment selection. Predictions of the histopathologic grade based on the visual appearance of papillary UTUC with digital ureteroscopy are often incorrect in comparison with biopsy results and yield low inter-rater agreement. Urologists must be aware of these limitations in the assessment of the tumor grade during digital ureteroscopy to warrant good clinical practice.