Methods of Evaluating Practice and Programs: SOWK 4900

Title
Program Evaluation on Mental Health Center of Denver.
Methods of Evaluating Practice and Programs: SOWK 4900
August 11, 2020
INTRODUCTION
The Mental Health Center of Denver (MHCD) is a community for recovery, resilience, and awareness of overall well-being through the use of innovative and evidence-based practices. Their program is known to be effective in community behavioral health care that links deep cultural concepts to understand people's everyday life challenges. Their mission is “enriching lives and minds by focusing on strengths and well-being” (Mental Health Center of Denver, 2020). The MHCD founded its mission on the philosophy that people can recover from different mental illnesses with proper guidance and treatment that works. Their mission statement can be extended into three aspects, including in making a difference, in providing mental health literacy, and expanding mental health access. In making a difference, they believe that their work can make a difference in helping the well-being of tens of thousands of people every year. In providing mental health literacy, they can increase the awareness of the large community about mental health issues. In expanding access, the MHCD make sure that their services can reach the people using an effective and compassionate treatment (Mental Health Center of Denver, 2020).
They believe that their growing success is directly connected to the mutual trust of patients to the mental health institution. The MHCD's staff are trained in addressing the issues of children, young adults, adults, and families. They enable the children to be more resilient and improve the family's overall health using an individualized community-based approach. The issues of teens and young adults are addressed using the specifically designed transition guidance from adolescence to adulthood. Adult recovery services focus on their strengths to help them improve their well-being. Because the MHCD has multiple community sites including the Emerson St. for teens and young adults and the Wellshire behavioral services for children and adults, access to public schools, and collaborations with other organizations, they can provide mental health treatment that reaches a greater number of patients (Mental Health Center of Denver, 2020).
The MHCD provides experts in leading the innovation of their services, best practices, and medical intervention. The MHCD experts can learn to apply new evidence-based practices in the health care service while maintaining their daily routine in serving the mental health center. As time pass, best practices are adopted in the workplace. Today, newly implemented practices include, but not limited to, integrated physical and behavioral health, Integrated Dual Disorder Treatment (IDDT) and Dialectical Behavioral Therapy (DBT) for the best possible treatment of mental illness and improve the quality of life of the people. The successful implementation of these programs is only possible with the support and guidance of MHCD leaders that take turns their vision to reality (Mental Health Center of Denver, 2020).
Purpose and Research Questions
The objective of this paper is to evaluate the effectiveness of implementing the MHCD's community-based approach in promoting healthcare improvement in the community. Specifically, this paper aims to answer if there is a significant difference between the community-approach implementation and the individualized evidence-based clinical approach in promoting better mental healthcare? What are the benefits of using the community-approach? What are the challenges of the community-approach? What are the individualized evidence-based methods that the MHCD is using? What is the effectiveness of these evidence-based methods?
Background Information on the Social problem Addressed by the Program
According to the 2018 Depression Report of Denver Public health, one in every eight residents in Denver reported that they experienced mental health problems that affect their mood, thinking, sleeping patterns, eating patterns, and overall productivity. This data was from individuals ages 5yrs and older about their mental health over the past month when the survey was conducted. The Denver Public Health authorities also reported that symptoms of depression have higher frequencies among increasing age groups, where 15 to 17-year-old were reported to have the highest frequency among the age group of children and adolescents. Untreated depression can lead to chronic health conditions, substance abuse, premature death, and suicide (Denver Public Health, 2017). Additionally, about 99% of adolescents and young adults (19-29 yrs.) are hesitant seeking mental health care because of the cost of treatment. Additionally, the article of Brown (2016) at the Denver Post reported that teenagers and adolescents are not comfortable talking to a therapist and making a treatment plan for their therapy. Most adult women are also not likely to seek treatment because of the cost and the scope of their insurance while adult males are more likely to not be comfortable in talking about their mental health issues (Denver Public Health, 2017).
LOGICAL MODEL
For the inputs, the resources and barriers, which can potentially limit the program’s effectiveness are divided into three groups, which include the external inputs, the MHCD mental health staff, and the assistance staff. Each input has a distinct role and an overlapping role with another input group that fosters collaboration for the community-based mental health support of the MHCD. The external inputs, which include the consumers, financial supporters, and local government officers, will be responsible for the activities to reach the greatest amount of people where they can provide additional resources and support in the activities since they know more about the community than the medical health professionals.
The MHCH staff trained with the necessary leadership skills and evidence-based practices will spearhead the overlapping activities of the external input group and the assistance staff. The expected outcome of their group is the effective implementation of evidence-based practices. Although their main activity is to implement the use of evidence-based practices, they will also work together with the other groups for the success of the program. The overlapping role of the external input group and the mental health staff in gathering the participation of children, young adults, adults, and families can lead to increased understanding and effectively addressing the mental health issues of each individual and family. The overlapping role of the assistance staff and the mental health staff in developing a community health information network can promote easy access of individuals to mental health support.
The last input group, the logistics, and technical assistance staff are involved in evaluating the effectiveness of the practices and reports the outcomes of the program. This information is relevant to further improve the program in the future. At the end of the model, the arrows converge into the impact of the expected result. In this program, the overall health status of the community is expected to improve.
LITERATURE REVIEW
How effective are the evidenced-based practices that MDCH implemented to the improvement of the people’s overall health status?
Effectiveness of Community-based approach to Healthcare Improvement
In the article of Aveling, Martin, Herbert, and Armstrong (2017), the researchers made a comparative study on how different community-based approach helps in improving healthcare. They studied two clinical community practice to see the difference between the Michigan Keystone ICU program in the United States and the Improving Lung Cancer Outcomes Project in the UK. Three main issues were studied to see the effectiveness of the community-based approach in mobilizing diverse communities, establishing a strong sense of community, and drive implementation. Their findings show that both US and UK community-based approach was effective in satisfying the three issues presented by the researchers for an effective improvement of community healthcare; however, many challenges arise from community-based approaches.
Their comparison suggests that the effectiveness of community-based practices requires leaders with credibility and skill to synergize the organizations, teams, or families, which are involved in the program. The clinical community-based practice had a dynamic and constant tension between the vertical and horizontal forces that affect the difficulty in implementing a community-based approach (Aveling et al., 2017). In the MHCD program, since there are multiple parties involved in making a community-based social work, leadership would be an underlying issue. The leaders should be open to collaboration between organizations to maximize the effectiveness of the approach.
Effectiveness of Integrated Dual Diagnosis Treatment (IDDT)
In the article by Kikkert et al. (2018), the researchers studied the use of an integrated approach in treating mental health issues with substance abuse. IDDT is an innovative approach in combining substance abuse treatment with psychiatric treatment for a more effective approach in addressing patients suffering from substance abuse. However, the use of IDDT requires professional skills and specific knowledge of the implementation. Kikkert et al. (2018), included 37 clinicians to undergo a 3-day training of IDDT. After the IDDT implementation, about 154 patients were treated with the new practice. Although the implementation of IDDT found a significant reduction in the use of alcohol or drugs, the researchers did not find any improvement in the secondary well-being outcomes of the patients, which includes psychopathology, motivation to change, and functioning. Kikkert et al. (2018), also reported that the training is not sufficient in improving the clinician’s knowledge and attitudes in disseminating IDDT protocol. It was noted in the conclusion that the clinicians are limited in the time of training and IDDT implementation has underlying complexities that overwhelmed the trainees.
Effectiveness of Dialectical Behavior Therapy (DBT)
In the article by McMain et al. (2018), the researchers studied the effectiveness of DBT for the treatment of borderline personality disorder (BPD). They explained that the common version of DBT spans for about 12-months of therapy to treat BPD. However, this limits its adoption to mental health care facilities because the demands for the therapy exceed more than one year of healthcare's resources. McMain et al. (2018) aim to see if a 6-month version of DBT, which is more cost-effective and easier to implement, has a significant difference against the 12-month version. The researchers concluded that the 12-month version of DBT spends too many resources, which makes it less practical in clinical practice. The 6-month version of DBT would be an excellent alternative because is cost-effective and more people are to be treated.
Effectiveness of Integrated Physical Health Needs of People with Mental Illness
In the article by Rodgers et al. (2018), they studied the effectiveness of integrating both physical and mental health care for people with severe mental illnesses. People with mental illness have lower life expectancy because of their poor physical condition during their struggles with mental health (Rodgers et al., 2018). They found evidence that health systems lack in integrating their healthcare since there is fragmentation in the specializations of medical professions. Rodgers et al. (2018), emphasized that mental health and physical health are linked but traditional mental health treatment separates the two services. Evidence from different literature suggests that people diagnosed with severe mental illnesses required both arrangements in addressing the physical health and mental health (Rodgers et al., 2018). The outcome will result in a more effective approach in overall health using an integrated care system. Communication is a vital element in implementing an integrated approach. The researchers explained the need for the effective sharing of information to gather support from different specializations. There will also be clarity in taking accountability and responsibility for physical treatment and mental treatment that promotes a collaborative environment in health care services (Rodgers et al., 2018).
Evaluation Design
This paper aims to answer if there is a significant difference between the community-approach implementation and the individualized evidence-based clinical approach in promoting better healthcare? This evaluation design will be a quasi-experimental design that will use quantitative data to help the evaluators to see the big picture of the program evaluation. A quasi-experimental design is appropriate because I will gather data from those people that are enrolled in the intervention that they need rather than assigning them into random interventions. There will be a summative evaluation at the end of therapy to see if the actual outcome is the same as expected outcomes of the logic model of the program.
Population and Sample
The adolescent and adult (21 yrs. old and above) patients that are enrolled in the community-based therapy of MHCD, which includes the Dahlia Campus for Health and Well-being (for child and family), the Emerson St. (for teens and young adults), and the recovery center (for adults), and the patients using the evidenced-based treatment plans for individuals in the MHCD Wellshire Behavioral Services that are undergoing IDDT, DBT, and Integrated Physical-Mental Therapy will constitute the population of the evaluation. The evaluators will acquire as much as many willing respondents from the community-based therapy and individualized treatment plans.
Measures/Data Collection Tool
Because the outcome and the impact in the logic model expect an overall improvement of the mental health status of the participants (21 yr. old and above), the preferred data collection tool should measure the overall psychological well-being of a person. The 42-item Ryff's Psychological Well-being Scale (PWS) that was developed by Carol D. Ryff will be used to test the general well-being of adult participants. The general characteristics of the scale integrated the six-dimensions of well-being, which includes self-acceptance, positive relations with others, autonomy, environmental mastery, purpose in life, and personal growth (Shryock & Meeks, 2018). According to Ryff (2014), autonomy is the ability of a person to be independent and resist social pressures where a person can regulate their behavior from their virtue. Environmental mastery is the ability of a person to cope with and manage their immediate environment to match their personal needs (Ryff, 2014). Personal Growth is the ability of a person to continue to develop and optimize their ability to reach their fullest potential (Ryff, 2014). Positive relationships with others are the ability of a person the be in a trusting relationship with others. This also includes the ability of a person to be concerned about the well-being of other people (Ryff, 2014). Purpose in life is the ability of a person to make their goals and persistence towards achieving those goals. The purpose in life gives a person an intention to live life with direction and meaning (Ryff, 2014). Self-acceptance is the ability of a person to have a realistic perception of themselves, which includes both good and bad, and can still accept oneself regardless of their flaws (Ryff, 2014).
Ryff’s PWS data collection tool is a 7-item Likert scale type instrument where the participants will choose their best response depending on the question. Additionally, the 42 item Ryff's PWS is short enough that data collection is easier to do. The participants won't take a long time in answering the questions, which is more convenient than other scales.
In the study of Shryock and Meeks (2018), the internal consistency and the validity of the 42-item Ryff's Psychological Well-being scale (PWS) among different age groups, which include older, middle-aged, and younger adults. Their finding shows that the average Cronbach’s alphas were .71, .78, and .77 respectively, which show an acceptable internal consistency for research (Shryock and Meeks, 2018). Additionally, the Ryff's PWS got a reasonable validity for the 6-factor model, which measures the self-acceptance, positive relations with others, autonomy, environmental mastery, purpose in life, and personal growth (Shryock and Meeks, 2018). Shryock and Meeks (2018) concluded from the results that the Ryff's PWS is a reliable measurement scale for across different adult age groups. Because Ryff's PWS is specialized for different adult age groups, the participants are limited to adults. Other age-specific tools are more appropriate for other age groups.
Sampling Technique
The data collection will be limited to surveys using Ryff’s PWS. Because the data that will be collected is purposive towards those people enrolled in different interventions, the data collection method will be non-probabilistic convenience sampling. The evaluators will conduct convenience sampling in the community-based therapy of MHCD, which includes the Dahlia Campus for Health and Well-being (for child and family), the Emerson St. (for teens and young adults), and the recovery center (for adults), and the in the MHCD Wellshire Behavioral Services. The evaluators will only contact the adults that are willing to participate in the evaluation. We will need the help of the mental health staff or professionals that are assigned to the intervention for the increased success in getting responses from willing participants.
Questionnaire Administration
The questionnaire could be admitted during or after the sessions of the different community-based therapy and individualized evidence-based therapy in the MHCD. The mental professional staff will be instructed to introduce the evaluator, then the evaluator will explain the objectives and specific instructions in answering the questionnaire.
Data Analysis
Because there is no realistic estimate on how many people will participate in the evaluation using convenience sampling, the data will be combined to form only two groups, which include the community-based groups and the individual-based groups, to maximize the quality of data assessment. In this evaluation, the data acquired from the Dahlia Campus for Health and Well-being (for child and family), the Emerson St. (for teens and young adults), and the recovery center (for adults), will be combined into one group, “community-based”. The data acquired from the MHCD Wellshire Behavioral Services that are undergoing IDDT, DBT, and Integrated Physical-Mental Therapy will be combined into another group, “individual-based”.
After calculating the scores in the Ryff's Psychological Wellbeing, the data gathered will be analyzed using the latest version SPSS Statistics Program to conduct the independent sample T-test to find if there is a significant difference between the community-approach implementation and the individualized evidence-based clinical approach in promoting better mental healthcare. The T-test is an appropriate measure because there are only two independent groups that will be compared. The T-test is also appropriate because the standard deviation within the groups is unknown. However, because the sampling method will utilize a convenience sampling approach, the analysis would need the additional power of the nonparametric Mann-Whitney U Test to see a significant difference between groups.
Conclusion
The evaluation proposal is designed to evaluate the effectiveness of the community-based program of the MHDC compared to individualized therapy. The approach is appropriate to see if there is a significant difference or improvement between the psychological well-being of the two groups. However, convenience sampling decreases the reliability of the results. The proposal addressed this issue by using the Mann-Whitney U test for the non-parametric approach of data analysis. Another problem is the number of participants because there are no realistic expectations on how many people are willing to take the evaluation of the effectiveness of their intervention. This can be addressed by combining all the participant’s data in only two different groups: the community-based group and the individualized group. The evaluation plan is easy to implement and measurable. The measurement scale used is simple and it generalizes the psychological well-being of adults. It is also cost-effective and the results are relevant to the stakeholders of the MHCD. The results of this evaluation are important in determining the decisions to improve the activities or remove those that are not working. Overall, the evaluation can examine the difference between the expected outcome of the program compared to the actual results.
References
Aveling, E., Martin, G., Herbert, G., & Armstrong, N. (2017). Optimizing the community-based approach to healthcare improvement: Comparative case studies of the clinical community model in practice. Social Science and Medicine, 173, 96-103. https://doi.org/10.1016/j.socscimed.2016.11.026
Brown, J. (2016, November 1). Mental health center’s new house for youth is the “cool” version of a mental health clinic. Retrieved August 10, 2020, from /2016/11/01/emerson-street-cool-mental-health-center/
Denver Public Health. (2017). New Report: Depression in Denver-Through the Lifespan. /news/2018/11/new-report-dep...