Description: We are leading the independent evaluation of VA’s mental health care and suicide prevention programs mandated by the Clay Hunt Suicide Prevention for American Veterans (SAV) Act. We are evaluating nine programs using administrative intake and follow-up data provided by VA’s Northeast Program Evaluation Center (NEPEC), VHA’s Medical Cost Accounting (MCA) System, and new data collected through a redesigned Veterans Outcome Assessment (VOA) survey. Our evaluation has consisted of the following components: screening VA mental health programs for effectiveness, including cost-effectiveness; Veteran satisfaction; and identifying and summarizing best practices in mental health/behavioral health and suicide prevention. The scope of the Clay Hunt SAV Act was expanded to include an evaluation of opioid prescribing and safety as well as identifying programs that are most effective for women veterans.
Project Management. We are performing to our comprehensive Project Management Plan, led by our Program Manager. Our agile management toolkit and engagement approach combine industry best-practice standards from the Project Management Body of Knowledge (PMBOK) tailored to VA policies and processes. Our Program Manager liaises with the VA clients and our Data Team while maintaining the work breakdown structure, issue log, lessons learned, project schedule, etc. We also create and maintain Weekly Status Reports and Monthly Progress Reports.
Survey Design. The evaluation team redesigned the VOA survey using questions from seven survey instruments. We prepared the sampling plan, which features an equal allocation strategy. Using this strategy, the plan provides for a consistent margin of error and the necessary reliability and precision among all evaluated programs. The equal allocation sampling plan also provides the statistical power sufficient to analyze programs separately and then to compare them against one another, if needed.
Data Analysis, Results, and Reporting. Our team has analyzed the data using effectiveness screening models that compare a pre-treatment value of an outcome variable to its post-treatment value. Statistically significant improvements from pre-treatment to post-treatment indicate that the program might be effective (i.e., the treatment is causing the improvement). Programs that screen positive will be subjected to in-depth analyses, such as by comparing program results to a control group. The cost-effectiveness analyses identify the cost of a program and then relate these costs to specific measures of program effectiveness for each mental health program evaluated. The interim results which have been integrated into two Interim Reports (FY2016 and FY2017) that are submitted to Congress, have identified best practices including the delivery of evidence-based treatments and organizational, administrative, and implementation-related principles that improve participant engagement in and/or outcomes for the treatment or prevention of mental health conditions or self-directed violence. Annual Reports to Congress are submitted to the Secretary of VA for submission to Congress by 01DEC of each year. Our 2018, 2019, and 2020 Annual Reports to Congress were very well received.