Evaluation of a dialectical behavior therapy training model in residential programs
Dialectical behavior therapy is a cognitive theory-based intervention that has previously demonstrated effectiveness in treating a range of behavioral disorders. However, DBT can be challenging to implement in community-based settings. Little guidance is available on models to evaluate the effectiveness or sustainability of training and implementation efforts. Residential treatment programs have much to gain from introduction of evidence-based practices, but present their own challenges in implementation, including few resources, high staff turnover rates, and lack of flexibility in staff schedules.
We piloted a process evaluation model to assess multi-stage DBT training in five residential programs serving adults with mental health and substance use disorders. The model targets staff and organizational factors associated with successful implementation of evidence-based practices and matches low-resource data collection to the four stages of the DBT training model (Staff Training, Service Design, Implementation, and Maintenance). The evaluation aimed to assess staff knowledge retention post-DBT training, the role of organizational culture in implementation, and the fidelity of services delivered.
In Phase One, administrative and line staff in each program received 24 hours of team-based training, with group facilitators receiving an additional 3 hours of training. Following completion of the initial training, program staff complete a knowledge assessment that includes items on the core components and skills modules of DBT.
In Phase Two, the trainer facilitates the design of DBT service delivery in conjunction with the administrative and supervisory staff. This includes planning for ongoing fidelity monitoring, clarifying staff roles, and ensuring that adequate resources are provided for maintenance. Practical barriers and aspects of organizational culture that may present challenges to successful implementation are assessed.
Staff begin provision of DBT services in Phase Three, including individual skills coaching and group facilitation, along with DBT-informed clinical supervision and staff meetings, and fidelity monitoring activities. The trainer assists in troubleshooting operational and clinical issues, and provides initial intensive fidelity monitoring. The trainer titrates consultation based on the assessed level of service quality and fidelity to the DBT model. The evaluation activities in this phase focus on fidelity assessment, through direct observation of staff facilitating the DBT skills group, and measuring therapeutic alliance with clients.
In the fourth and final phase of implementation, a monthly team consultation meeting brings together staff from the previously-trained programs to enhance and reinforce their DBT skills and knowledge, and to prevent “drift” from the DBT model. These meetings function to extend clinician skills, support motivation, and to monitor burnout. Evaluation activities focused on elucidating the ways in which these team consultation meetings supported staff work with clients and the impact of DBT training on staff retention. Qualitative interviewing was utilized to explore the processes and impact of the meetings, along with analysis of secondary data from the agency human resources department to monitor staff turnover.
One of the main strengths of the evaluation model is that it maximizes the existent resources of a community agency, increasing the feasibility and sustainability of DBT. By matching data collection to each stage of the training and implementation, community providers receive timely feedback that allows them to address common pitfalls in the implementation process. While the lack of standardized instruments available in the published literature specific to DBT implementation represents a significant weakness in our assessment, we outline one example of a low-cost evaluation model to inform implementation of dialectical behavior therapy in residential treatment settings.
Amber M Holbrook
West Chester University, USA
PublicationImplementation of Dialectical Behavior Therapy in Residential Treatment Programs: A Process Evaluation Model for a Community-Based Agency.
Holbrook AM, Hunt SR, See MR
Community Ment Health J. 2018 Oct