To evaluate the reliability and accuracy of nonaneurysmal perimesencephalic subarachnoid hemorrhage (NAPSAH) on Noncontrast Head CT (NCCT) between numerous raters.

45 NCCT of adult patients with SAH who also had a catheter angiography (CA) were independently evaluated by 48 diverse raters; 45 raters performed a second assessment one month later. For each case, raters were asked: 1) whether they judged the bleeding pattern to be perimesencephalic; 2) whether there was blood anterior to brainstem; 3) complete filling of the anterior interhemispheric fissure (AIF); 4) extension to the lateral part of the sylvian fissure (LSF); 5) frank intraventricular hemorrhage; 6) whether in the hypothetical presence of a negative CT angiogram they would still recommend CA. An automatic NAPSAH diagnosis was also generated by combining responses to questions 2–5. Reliability was estimated using Gwet's AC1 (κ_G), and the relationship between the NCCT diagnosis of NAPSAH and the recommendation to perform CA using Cramer's V test. Multi-rater accuracy of NCCT in predicting negative CA was explored.

Inter-rater reliability for the presence of NAPSAH was moderate (κ_G = 0.58; 95%CI: 0.47, 0.69), but improved to substantial when automatically generated (κ_G = 0.70; 95%CI: 0.59, 0.81). The most reliable criteria were the absence of AIF filling (κ_G = 0.79) and extension to LSF (κ_G = 0.79). Mean intra-rater reliability was substantial (κ_G = 0.65). NAPSAH weakly correlated with CA decision (V = 0.50). Mean sensitivity and specificity were 58% (95%CI: 44%, 71%) and 83 % (95%CI: 72 %, 94%), respectively.

NAPSAH remains a diagnosis of exclusion. The NCCT diagnosis was moderately reliable and its impact on clinical decisions modest.