Contraception counseling and provision is an essential preventative service. Real-time assessment of these services is critical for quality improvement and comparative study. Direct observation is not feasible on a large scale, so indirect measures (such as chart review) have been determined to be acceptable tools for this assessment. Computer-aided chart review has significant benefits over manual chart review as far as greater efficiency and ease of repeated measurements. The wide use of electronic medical records provides an opportunity to create a data extraction algorithm for computer-aided chart review that is sharable among institutions. We provide a useful schema for others who use electronic medical record systems and are interested in real-time assessment of contraception counseling and provision for the purposes of baseline assessment of services and quality improvement.

The purpose of this study was to create a comprehensive and accurate data extraction algorithm that is useful in the assessment of contraception counseling and provision rates in the outpatient setting.

We included all visits between August 2015 and May 2016 at 8 outpatient clinics that are affiliated with a large, urban academic medical center in which nonpregnant women who were 14–45 years old were seen by a nurse practitioner, physician’s assistant, or physician. Contraception-related prescriptions, International Classification of Diseases codes, current procedural terminology codes, and search-term capture were extracted with the use of structured query language from electronic medical record data that were stored in a relational database. The algorithm’s hierarchy was designed to query prescription data first, followed by International Classification of Diseases and current procedural terminology codes, and finally search-term capture. Visits were censored when the first positive evidence of contraceptive service was obtained. Search terms were selected based on group discussion of investigators and providers. This algorithm was then compared with manual chart review and refined 3 times until high sensitivity and specificity, when compared with manual chart review, were achieved.

There were 22,134 visits of reproductive-aged women who our inclusion criteria. Electronic medical record evidence of contraception counseling or provision was found in 56.9% of these visits. Of these, 21.3% were captured by prescriptions; 8.9% were captured by International Classification of Diseases codes, and 69.7% were captured by search-term capture with the use of our algorithm. Among visits with evidence of contraception counseling without provision, 15.7% were captured by diagnosis codes and 84.3% were captured by search-term capture. When compared with manual chart review, sensitivity and specificity improved from 0.79 and 0.85 to 0.99 and 0.98, respectively, over the 3 rounds of testing and revision.

Data extraction algorithms can be used effectively for computer-aided chart review of contraception counseling and provision measures, but testing and refinement are extremely important. Search-term capture from unstructured data is a critical component of a comprehensive algorithm, especially for the capture of instances of contraception counseling without provision. The algorithm that we developed here could be used by others with an electronic medical record system who are interested in real-time assessment, quality improvement, and comparative study of the delivery of contraceptive services. The ease of execution of this algorithm also allows for its repeated use for ongoing assessments over time.