Increased interest in disseminating and implementing psychological treatments has focused on the need for evidence-based training programs for providers, especially those without specialized training. To evaluate provider-training programs, validated outcome measures are necessary; however, the scalable measurement of training outcomes has been largely overlooked. Current methods of assessing providers’ ability to deliver psychological treatments are generally time-consuming and costly, representing a major bottleneck in scaling up mental health care for commonly occurring disorders such as depression. The present study describes the development and initial validation of a scalable measure for assessing provider competence in delivering a brief behavioral activation treatment for depression, called the Healthy Activity Program, adapted for primary care settings. The measure focuses on testing knowledge about the treatment and applied knowledge regarding how to skilfully deliver the treatment, both essential features of competence. The measure was tested on a sample of 531 respondents with a variety of educational levels and professional backgrounds, and found to meet the requirements of the Rasch model. Three versions of the measure each of equal difficulty were derived to allow repeat testing of training outcomes over time. A scalable measure of provider competence is an essential first step towards supporting the wider dissemination and implementation of brief psychological interventions for depression, especially in low-resource settings.