Principal Researchers: Dr Will Dobud (Charles Sturt University) & Dr Daniel Cavanaugh (University of Washington Bothell)
Funding: The Adventure Therapy Outcome Monitoring (ATOM) study was awarded $10,000 USD in 2022 which was spent on guest lecturing, research consultation with Dr Scott D Miller, Professor Nevin J. Harper, and Steve Javorski.
Free training and consultation will be delivered with Scott D Miller throughout 2023 with practitioners involved in the ATOM study. You can register for the training here.
Useful Links:
Participant Information Sheet
Practitioner Demographics
Feedback-Informed Treatment Measures
Video Introducing the ATOM (Password: ATOM)
Podcast about the ATOM (Spotify)
Interested in taking part? Email [email protected] to get in touch!!
Funding: The Adventure Therapy Outcome Monitoring (ATOM) study was awarded $10,000 USD in 2022 which was spent on guest lecturing, research consultation with Dr Scott D Miller, Professor Nevin J. Harper, and Steve Javorski.
Free training and consultation will be delivered with Scott D Miller throughout 2023 with practitioners involved in the ATOM study. You can register for the training here.
Useful Links:
Participant Information Sheet
Practitioner Demographics
Feedback-Informed Treatment Measures
Video Introducing the ATOM (Password: ATOM)
Podcast about the ATOM (Spotify)
Interested in taking part? Email [email protected] to get in touch!!
ADVENTURE THERAPY OUTCOME MONITORING STUDY PROJECT DESCRIPTION
Background
Outdoor therapy (Harper & Doherty, 2020) is an umbrella term used to describe the range of therapies intentionally using adventure and experiential techniques to engage people in psychotherapy services. This study will invite outdoor/adventure therapy practitioners, such as those practicing adventure, wilderness, surf, or nature-based therapy, to integrate Feedback-Informed Treatment (FIT), a “pantheoretical approach for evaluating and improving the quality and effectiveness of behavioral health services” (Bertolino, Bergmann, & Miller, 2012, p. 2), into their practice. The FIT Manuals utilised to inform this research project and methods are endorsed by the United States’ Substance Abuse and Mental Health Services Administration’s National Registry of Evidence-Based Programs and Practices. FIT involves administering two self-report, ultra-brief measures to evaluate the quality of psychotherapy services. The Outcome Rating Scale (ORS; Miller et al., 2003), administered at the start of each session, is a measure of general wellbeing. Administered in the last five minutes of a session, the Session Rating Scale (SRS; Duncan et al., 2003) is a measure of quality of the therapeutic relationship. This study aims to explore the potential and feasibility for routine outcome monitoring in outdoor/adventure therapy practice.
Literature review
Data from meta-analyses and systemic reviews support the use of outdoor and adventure therapies (Bowen & Neill, 2013), wilderness therapy (Bettman et al, 2016), forest therapy (Lee et al., 2017), among the many other forms of outdoor therapies. While outdoor and adventure therapy practitioners are delivering outcomes on par with more ‘traditional’ modes of psychotherapy (Dobud & Harper, 2018), a major gap in the literature is practice-based research and the implementation of routine outcome monitoring, namely through FIT (Dobud, Cavanaugh, & Harper, 2020).
As mentioned above, FIT involves the use of two ultra-brief psychometric measures, the ORS and SRS (See Appendix A). The ORS, which is available in child form and more than 20 languages, is an outcome measure used at each therapy encounter, or at the start of each week of residential treatment, to determine the client’s perception of progress in therapy. The SRS is a measure of the therapeutic alliance; one of the most potent factors for predicting client progress in psychotherapy. The SRS is administered at the end of each therapy session, or at the end of each week in residential treatment. Both scales take less than one minute to administer, which increased FIT’s feasibility in real world practice (Schuckard, Miller, & Hubble, 2017).
The available evidence from more than a dozen meta-analyses and randomised clinical trials suggests that practitioners using these two measures to elicit feedback on their client’s perception of progress and the therapeutic alliance have the potential to improve client outcomes. According to Miller et al. (2015), this routine monitoring of outcome may
(a) double the effect size of treatment and increase the proportion of clients with reliable significant change; (b) cut dropout rates in half; (c) reduce the risk of deterioration by one third; (d) shorten the length of treatment by two thirds; and (e) drive down the cost of care. (p. 449)
FIT works across therapeutic approaches, such as cognitive-behavioural or psychodynamic therapy, by bringing the practitioner’s attention to a therapy client’s progress, or lack thereof, and their perception of the quality of the therapeutic alliance, which includes a relational bond, agreement on the purpose or goals of therapy, and consensus on the means or methods to achieve those goals (Bordin, 1979). The longer a client remains in therapy without experiencing a positive outcomes, the more likely a client is to deteriorate, or dropout without receiving benefit (Garcia & Weisz, 2002; Hansen, 2002). The most predictive factors for deterioration and dropout are issues in the therapeutic alliance (Bertolino et al., 2012) and the research indicates that practitioners are unlikely to identify these clients at-risk of negative or null outcomes (Hannan et al., 2005).
Rationale/Justification (i.e. how the research will fill any gaps, contribute to the field of research or contribute to existing or improved practice)
While limited research exists in the outdoors therapies exploring outcomes throughout the treatment process, these studies did not implement routine outcome monitoring, in which practitioners work with their clients to systematically explore the quality of the therapeutic alliance and determining which clients are at-risk of negative outcome (Dobud et al., 2020). The present study will invite practitioners to implement FIT, using the ORS and SRS, into their outdoor/adventure therapy practice. By creating a ‘database’ of outcomes, this research project will explore the variance in practitioner outcomes, based on a range of factors, such as years of experience, qualification, caseload, gender identity, ethnicity, among others described below. Additionally, by tracking outcomes each week, the study will be the most comprehensive exploration of the dose-effect relationship in these particular therapies.
Research question
This study is informed by the research question: Is feedback-informed treatment an effective tool for measuring effectiveness in the outdoor therapies?
Aims
The aim of this study is to explore the outcomes of outdoor/adventure therapy practitioners using FIT to best determine what can improve client outcomes.
Objectives
The objectives of the present study is to invite outdoor therapy practitioners to routinely monitor their outcomes through FIT. This data will be used to explore the correlation of FIT’s ultra-brief outcome and alliance measures, the dose-effect relationship in the outdoor therapies, and the variance in practitioners’ outcomes.
Expected outcomes
By inviting practitioners to implement FIT, a framework for routine outcome monitoring, in their outdoor/adventure therapy practice, this study can examine dose-effect, deterioration rates, variance in practitioners’ outcomes, and correlation between outcome and alliance measures. These findings can inform practitioners about the warning signs when therapy is off track. Additionally, there are possibilities to inform social policy about the possibility of taking therapy outdoors.
Project Design
Methodological approach
The methods utilised for this research project are informed by previous studies utilising FIT and the exploration of therapist effects on outcomes outside of the outdoor and adventure therapies, such as Goldberg et al. (2016) and Lutz et al. (2007).
Rationale for choices of method/s (tied to project aims/objectives)
The purpose of utilising these methods for outcome monitoring and exploring the variance of therapist effects is linked closely to the objectives of the study.
Research Participants
The research participants invited to engage with this study are therapy ‘practitioners’, such as social workers, psychologists, counsellors, psychiatrists, marriage and family therapists, youth workers, or other paraprofessionals, like the field staff employed at wilderness therapy programs, who provide a range of the outdoor therapies described by Harper and Dobud (2020).
Description and number
I aim to construct a sample of a minimum of thirty practitioners to improve the generalisability of the findings. Ideally, each practitioner will routinely monitor the outcomes for a minimum of 30 outdoor therapy clients to improve confidence in the findings.
Inclusion and exclusion criteria
Inclusion criteria includes professional therapeutic practitioners providing outdoor/adventure therapy services aimed at improving wellbeing and psychological functioning. While this study is focused on the ‘outdoor’ therapies, there are practitioners ‘bringing the outdoors in’ (Harper, Rose, & Siegel, 2019) who are invited to participate. For example, adventure therapy practitioners may utilise an indoor rock climbing gyms to conduct their session. When recording their outcomes, practitioners will record where their session took place. Additionally, there may be practitioners conducting group therapy in a residential setting that also utilise outdoor spaces, such as a small courtyard or hiking trail.
Volunteers, summer camp programs, outdoor education, and other outdoor services which do not contain a therapeutic rationale, such as wellness retreats or adventure-based recreation, will be excluded from this study.
Justification of sample size
The aim for a minimum of 30 practitioners is to improve the generalisability and confidence in my findings. According Seidel and Miller (2012) in Manual 4 of the FIT manuals, measuring a practitioner’s outcomes
can be influenced by random variations in a clinician’s caseload, so they are likely to be “unstable” or unreliable with caseloads of fewer than 30 clients. Caseloads of 60 or more are likely to yield effect sizes that are stable unless systematic changes in therapist functioning or case occur; and caseloads of 100 or more clients will provide especially robust or predictive effect sizes. (p. 28).
This research project will aim to include at least 30 practitioners who have routinely monitored the outcomes from a caseload of at least 30.
Description of participant recruitment strategies and timeframes, including measures to be used to ensure informed consent
Information sheets and consent forms will be emailed to potential practitioners through their organisation’s website. The information sheet will be shared on social media pages, such as the International Adventure Therapy Facebook group and Ecopsychology Facebook group, among others. The chief researcher will host virtual information sessions via Zoom for those interested in learning more about the study.
Data Collection/Gathering: What information are you going to collect/gather?
Practitioners volunteering to engage will be directed to download the free licensing for the FIT measures (available here) and provided training on how to collect and input data into the pre-made Excel spreadsheet.
Page 1 of the excel spreadsheet, titled Practitioner Demographics will capture each practitioner’s name, the country they are currently practicing, gender identity, date of birth, ethnicity, profession, level of education, workload, years of experience, theoretical orientation and their current employment. These factors can be used, as they have in previous studies (see Chow et al., 2015), to explore the variance in practitioner outcomes.
Page 2 of the spreadsheet, titled Your Caseload, is designed for practitioners to input their routinely monitored outcomes. For each client, the practitioner will provide a de-identified Client ID only they can identify. They will mark that informed consent was signed by the client, and/or the client’s legal guardian in the case of working with youth, as well as a client’s gender identity, ethnicity, date of birth, referral source, presenting problem, marital status, and case status (i.e. open, treatment completed, client drop out, referred onward, client relocated, health/medical issues). After each session, or phase of residential programming, the practitioner will input the client’s ORS and SRS scores, describe the outdoor/adventure therapy initiative that took place, such as hiking, rock climbing, or nature-based medication, and the location in which the session took place, like a nearby park, beach, home visit, or community centre. No identifiable information from the client will be collected.
Outcome data will be collected using the self-report ORS (Miller et al., 2003), which was developed as an alternative to the Outcome Questionnaire 45.2 (OQ; Lambert et al., 1996). The ORS is preferred over the OQ management system due to its brevity. The OQ, and Youth-OQ, contain 45 questions, and while the OQ is used widely in psychotherapy outcome research, it is not feasible like the ultra-brief ORS, which can take less than one minute to administer. The “specific items on the ORS were adapted from the three areas of client functioning assessed by the OQ-45.2; specifically, individual, relational, and social” (Miller et al., 2003, p. 93) wellbeing. Regarding this research project, the ORS is useful given that the OQ measures have been widely used in the outdoor and adventure therapy literature (Norton et al., 2014).
The data from outcome measures will be correlated with data from the SRS (Duncan et al., 2003), a measure of the client’s perception of the therapeutic alliance. Using a similar ultra-brief, visual analogue as the ORS, the SRS measures a client’s perception of the relational bond, goal consensus, the methods used by the practitioner, and an overall rating of the session; the core components of the therapeutic alliance originally conceptualised by Bordin (1979). Duncan et al. (2003) found the SRS to have strong internal consistency reliability, concurrent validity, and feasibility in comparison to longer, commonly used alliance measures, such as the Working Alliance Inventory (Horvath & Greenberg, 1989).
Data collection/gathering techniques: How will you collect/gather the information?
Practitioners will provide their clients with the information sheet about FIT and the present study to obtain informed consent. From the first meeting, the practitioner will administer the ORS and SRS, and each session thereafter, or phase of residential treatment. Practitioners will store their Excel spreadsheet on a password protected hard drive. They will email their spreadsheet to the chief researcher quarterly for outcomes to be tracked overtime, similar to the Goldberg et al. (2016) study.
Impact of and response to participant withdrawal
There will be no change in service delivery if a client decides to withdrawal from the study and their practitioner can delete their outcome and alliance data from the spreadsheet at any time. Practitioners will provide this information to their clients upon seeking informed consent. If a practitioner elects to withdrawal from the study, the chief researcher will delete their spreadsheet from the password protected hard drive storing the practitioner outcomes.
Description of the data analysis process/es
Each practitioner’s outcome data will be used to calculate their overall effect size using Glass’s delta (). This data will be used to explore the variance in the practitioners’ outcomes. An effect size will be calculated quarterly to determine if practitioners’ outcomes change with time.
Bivariate correlation (Pearson’s r) will be used to determine if the many factors collected in the Excel spreadsheet correlate to the outcomes of these outdoor and adventure therapy practitioners’ outcomes. For example, alliance data from the SRS will be used to determine if, like in other studies, the client’s perception of the therapeutic alliance correlates with outcome in the outdoor and adventure therapies.
Results, Outcomes and Future Plans
Data from this pilot study will be published in peer-reviewed journals.
Plans for return of results of research to participants
The chief researcher will send quarterly reports to the participating practitioners containing an update about the progress of the research project and preliminary findings.
Plans for dissemination and publication of project outcomes
Data from this pilot study will be published in peer-reviewed journals.
Other potential uses of the data at the end of the project
If this pilot study reveals potential for ongoing research, the project could develop into a randomised clinical trial focused on what steps, such as ongoing training or modes of supervision, help practitioners improve their outcomes.
References
Bettmann, J. E., Gillis, H. L., Speelman, E. A., Parry, K. J., & Case, J. M. (2016). A meta-analysis of wilderness therapy outcomes for private pay clients. Journal of Child and Family Studies, 25(9), 2659–2673.
Bertolino, B., Bergmann, S. & Miller, S. D. (2012). What works in therapy: A primer. In B. Bertolino & S. D. Miller (Eds.). ICCE Manuals on Feedback-Informed Treatment (FIT) (Manual 1). International Center for Clinical Excellence.
Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16(3), 252–260.
Bowen, D. J., & Neill, J. T. (2013). A meta-analysis of adventure therapy outcomes and moderators. The Open Psychology Journal, 6(1), 28–53.
Chow, D. L., Miller, S. D., Seidel, J. A., Kane, R. T., Thornton, J. A., & Andrews, W. P. (2015). The role of deliberate practice in the development of highly effective psychotherapists. Psychotherapy, 52(3), 337–345.
Duncan, B. L., Miller, S. D., Sparks, J. A., Claud, D. A., Reynolds, L. R., Brown, J., & Johnson, L. D. (2003). The Session Rating Scale: Preliminary psychometric properties of a “working” alliance measure. Journal of Brief Therapy, 3(1), 3–12.
Dobud, W. W., Cavanaugh, D. L., & Harper, N. J. (2020). Adventure Therapy and Routine Outcome Monitoring of Treatment: The Time Is Now. Journal of Experiential Education, 43(3), 262–276.
Dobud, W. W., & Harper, N. J. (2018). Of Dodo birds and common factors: A scoping review of direct comparison trials in adventure therapy. Complementary Therapies in Clinical Practice, 31, 16–24.
Garcia, J. A., & Weisz, J. R. (2002). When youth mental health care stops: Therapeutic relationship problems and other reasons for ending youth outpatient treatment. Journal of Consulting and Clinical Psychology, 70(2), 439–443.
Goldberg, S. B., Rousmaniere, T., Miller, S. D., Whipple, J., Nielsen, S. L., Hoyt, W. T., & Wampold, B. E. (2016). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63(1), 1–11
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002). The psychotherapy dose‐response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9(3), 329–343.
Harper, N. J. & Dobud, N. J. (Eds.). (2020). Outdoor therapies: An introduction to practice, possibilities and critical perspectives. Routledge.
Harper, N. J. & Doherty, N. J. (2020). An introduction to outdoor therapies. In N. J. Harper & W. W. Dobud (Eds.) Outdoor therapies: An introduction to practice, possibilities, and critical perspectives (pp. 3–16). Routledge.
Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 64, 223–233.
Lambert, M. J., Hansen, N. B., Umphress, V., Lunnen, K., Okiishi, J., Burlingame, et al. (1996). Administration and scoring manual for the OQ-45.2. Stevenson, MD: American Professional Credentialing Services.
Lee, I., Choi, H., Bang, K. S., Kim, S., Song, M., & Lee, B. (2017). Effects of forest therapy on depressive symptoms among adults: A systematic review. International Journal of Environmental Research and Public Health, 14(3), 1–18.
Lutz, W., Leon, S. C., Martinovich, Z., Lyons, J. S., & Stiles, W. B. (2007). Therapist effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counseling Psychology, 54(1), 32–39.
Miller, S. D., Duncan, B. L., Brown, J., Sparks, J. A., & Claud, D. A. (2003). The outcome rating scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. Journal of Brief Therapy, 2(2), 91–100.
Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449–457.
Norton, C. L., Tucker, A., Russell, K. C., Bettmann, J. E., Gass, M. A., Gillis, H. L., & Behrens, E. (2014). Adventure therapy with youth. Journal of Experiential Education, 37(1), 46–59.
Seidel, J. & Miller, S. D., (2012). Documenting change: A primer on measurement. In B. Bertolino & S. D. Miller (Eds.). ICCE Manuals on Feedback-Informed Treatment (FIT) (Manual 4). International Center for Clinical Excellence.
Schuckard, E., Miller, S. D., Hubble, M. A. (2017). Feedback-informed treatment: Historical and empirical foundations. In D. S. Prescott, S. D. Miller, & C. L. Maeschalck (Eds.). Feedback-informed treatment in clinical practice: Reaching for excellence (pp. 13–36). International Center for Clinical Excellence
Background
Outdoor therapy (Harper & Doherty, 2020) is an umbrella term used to describe the range of therapies intentionally using adventure and experiential techniques to engage people in psychotherapy services. This study will invite outdoor/adventure therapy practitioners, such as those practicing adventure, wilderness, surf, or nature-based therapy, to integrate Feedback-Informed Treatment (FIT), a “pantheoretical approach for evaluating and improving the quality and effectiveness of behavioral health services” (Bertolino, Bergmann, & Miller, 2012, p. 2), into their practice. The FIT Manuals utilised to inform this research project and methods are endorsed by the United States’ Substance Abuse and Mental Health Services Administration’s National Registry of Evidence-Based Programs and Practices. FIT involves administering two self-report, ultra-brief measures to evaluate the quality of psychotherapy services. The Outcome Rating Scale (ORS; Miller et al., 2003), administered at the start of each session, is a measure of general wellbeing. Administered in the last five minutes of a session, the Session Rating Scale (SRS; Duncan et al., 2003) is a measure of quality of the therapeutic relationship. This study aims to explore the potential and feasibility for routine outcome monitoring in outdoor/adventure therapy practice.
Literature review
Data from meta-analyses and systemic reviews support the use of outdoor and adventure therapies (Bowen & Neill, 2013), wilderness therapy (Bettman et al, 2016), forest therapy (Lee et al., 2017), among the many other forms of outdoor therapies. While outdoor and adventure therapy practitioners are delivering outcomes on par with more ‘traditional’ modes of psychotherapy (Dobud & Harper, 2018), a major gap in the literature is practice-based research and the implementation of routine outcome monitoring, namely through FIT (Dobud, Cavanaugh, & Harper, 2020).
As mentioned above, FIT involves the use of two ultra-brief psychometric measures, the ORS and SRS (See Appendix A). The ORS, which is available in child form and more than 20 languages, is an outcome measure used at each therapy encounter, or at the start of each week of residential treatment, to determine the client’s perception of progress in therapy. The SRS is a measure of the therapeutic alliance; one of the most potent factors for predicting client progress in psychotherapy. The SRS is administered at the end of each therapy session, or at the end of each week in residential treatment. Both scales take less than one minute to administer, which increased FIT’s feasibility in real world practice (Schuckard, Miller, & Hubble, 2017).
The available evidence from more than a dozen meta-analyses and randomised clinical trials suggests that practitioners using these two measures to elicit feedback on their client’s perception of progress and the therapeutic alliance have the potential to improve client outcomes. According to Miller et al. (2015), this routine monitoring of outcome may
(a) double the effect size of treatment and increase the proportion of clients with reliable significant change; (b) cut dropout rates in half; (c) reduce the risk of deterioration by one third; (d) shorten the length of treatment by two thirds; and (e) drive down the cost of care. (p. 449)
FIT works across therapeutic approaches, such as cognitive-behavioural or psychodynamic therapy, by bringing the practitioner’s attention to a therapy client’s progress, or lack thereof, and their perception of the quality of the therapeutic alliance, which includes a relational bond, agreement on the purpose or goals of therapy, and consensus on the means or methods to achieve those goals (Bordin, 1979). The longer a client remains in therapy without experiencing a positive outcomes, the more likely a client is to deteriorate, or dropout without receiving benefit (Garcia & Weisz, 2002; Hansen, 2002). The most predictive factors for deterioration and dropout are issues in the therapeutic alliance (Bertolino et al., 2012) and the research indicates that practitioners are unlikely to identify these clients at-risk of negative or null outcomes (Hannan et al., 2005).
Rationale/Justification (i.e. how the research will fill any gaps, contribute to the field of research or contribute to existing or improved practice)
While limited research exists in the outdoors therapies exploring outcomes throughout the treatment process, these studies did not implement routine outcome monitoring, in which practitioners work with their clients to systematically explore the quality of the therapeutic alliance and determining which clients are at-risk of negative outcome (Dobud et al., 2020). The present study will invite practitioners to implement FIT, using the ORS and SRS, into their outdoor/adventure therapy practice. By creating a ‘database’ of outcomes, this research project will explore the variance in practitioner outcomes, based on a range of factors, such as years of experience, qualification, caseload, gender identity, ethnicity, among others described below. Additionally, by tracking outcomes each week, the study will be the most comprehensive exploration of the dose-effect relationship in these particular therapies.
Research question
This study is informed by the research question: Is feedback-informed treatment an effective tool for measuring effectiveness in the outdoor therapies?
Aims
The aim of this study is to explore the outcomes of outdoor/adventure therapy practitioners using FIT to best determine what can improve client outcomes.
Objectives
The objectives of the present study is to invite outdoor therapy practitioners to routinely monitor their outcomes through FIT. This data will be used to explore the correlation of FIT’s ultra-brief outcome and alliance measures, the dose-effect relationship in the outdoor therapies, and the variance in practitioners’ outcomes.
Expected outcomes
By inviting practitioners to implement FIT, a framework for routine outcome monitoring, in their outdoor/adventure therapy practice, this study can examine dose-effect, deterioration rates, variance in practitioners’ outcomes, and correlation between outcome and alliance measures. These findings can inform practitioners about the warning signs when therapy is off track. Additionally, there are possibilities to inform social policy about the possibility of taking therapy outdoors.
Project Design
Methodological approach
The methods utilised for this research project are informed by previous studies utilising FIT and the exploration of therapist effects on outcomes outside of the outdoor and adventure therapies, such as Goldberg et al. (2016) and Lutz et al. (2007).
Rationale for choices of method/s (tied to project aims/objectives)
The purpose of utilising these methods for outcome monitoring and exploring the variance of therapist effects is linked closely to the objectives of the study.
Research Participants
The research participants invited to engage with this study are therapy ‘practitioners’, such as social workers, psychologists, counsellors, psychiatrists, marriage and family therapists, youth workers, or other paraprofessionals, like the field staff employed at wilderness therapy programs, who provide a range of the outdoor therapies described by Harper and Dobud (2020).
Description and number
I aim to construct a sample of a minimum of thirty practitioners to improve the generalisability of the findings. Ideally, each practitioner will routinely monitor the outcomes for a minimum of 30 outdoor therapy clients to improve confidence in the findings.
Inclusion and exclusion criteria
Inclusion criteria includes professional therapeutic practitioners providing outdoor/adventure therapy services aimed at improving wellbeing and psychological functioning. While this study is focused on the ‘outdoor’ therapies, there are practitioners ‘bringing the outdoors in’ (Harper, Rose, & Siegel, 2019) who are invited to participate. For example, adventure therapy practitioners may utilise an indoor rock climbing gyms to conduct their session. When recording their outcomes, practitioners will record where their session took place. Additionally, there may be practitioners conducting group therapy in a residential setting that also utilise outdoor spaces, such as a small courtyard or hiking trail.
Volunteers, summer camp programs, outdoor education, and other outdoor services which do not contain a therapeutic rationale, such as wellness retreats or adventure-based recreation, will be excluded from this study.
Justification of sample size
The aim for a minimum of 30 practitioners is to improve the generalisability and confidence in my findings. According Seidel and Miller (2012) in Manual 4 of the FIT manuals, measuring a practitioner’s outcomes
can be influenced by random variations in a clinician’s caseload, so they are likely to be “unstable” or unreliable with caseloads of fewer than 30 clients. Caseloads of 60 or more are likely to yield effect sizes that are stable unless systematic changes in therapist functioning or case occur; and caseloads of 100 or more clients will provide especially robust or predictive effect sizes. (p. 28).
This research project will aim to include at least 30 practitioners who have routinely monitored the outcomes from a caseload of at least 30.
Description of participant recruitment strategies and timeframes, including measures to be used to ensure informed consent
Information sheets and consent forms will be emailed to potential practitioners through their organisation’s website. The information sheet will be shared on social media pages, such as the International Adventure Therapy Facebook group and Ecopsychology Facebook group, among others. The chief researcher will host virtual information sessions via Zoom for those interested in learning more about the study.
Data Collection/Gathering: What information are you going to collect/gather?
Practitioners volunteering to engage will be directed to download the free licensing for the FIT measures (available here) and provided training on how to collect and input data into the pre-made Excel spreadsheet.
Page 1 of the excel spreadsheet, titled Practitioner Demographics will capture each practitioner’s name, the country they are currently practicing, gender identity, date of birth, ethnicity, profession, level of education, workload, years of experience, theoretical orientation and their current employment. These factors can be used, as they have in previous studies (see Chow et al., 2015), to explore the variance in practitioner outcomes.
Page 2 of the spreadsheet, titled Your Caseload, is designed for practitioners to input their routinely monitored outcomes. For each client, the practitioner will provide a de-identified Client ID only they can identify. They will mark that informed consent was signed by the client, and/or the client’s legal guardian in the case of working with youth, as well as a client’s gender identity, ethnicity, date of birth, referral source, presenting problem, marital status, and case status (i.e. open, treatment completed, client drop out, referred onward, client relocated, health/medical issues). After each session, or phase of residential programming, the practitioner will input the client’s ORS and SRS scores, describe the outdoor/adventure therapy initiative that took place, such as hiking, rock climbing, or nature-based medication, and the location in which the session took place, like a nearby park, beach, home visit, or community centre. No identifiable information from the client will be collected.
Outcome data will be collected using the self-report ORS (Miller et al., 2003), which was developed as an alternative to the Outcome Questionnaire 45.2 (OQ; Lambert et al., 1996). The ORS is preferred over the OQ management system due to its brevity. The OQ, and Youth-OQ, contain 45 questions, and while the OQ is used widely in psychotherapy outcome research, it is not feasible like the ultra-brief ORS, which can take less than one minute to administer. The “specific items on the ORS were adapted from the three areas of client functioning assessed by the OQ-45.2; specifically, individual, relational, and social” (Miller et al., 2003, p. 93) wellbeing. Regarding this research project, the ORS is useful given that the OQ measures have been widely used in the outdoor and adventure therapy literature (Norton et al., 2014).
The data from outcome measures will be correlated with data from the SRS (Duncan et al., 2003), a measure of the client’s perception of the therapeutic alliance. Using a similar ultra-brief, visual analogue as the ORS, the SRS measures a client’s perception of the relational bond, goal consensus, the methods used by the practitioner, and an overall rating of the session; the core components of the therapeutic alliance originally conceptualised by Bordin (1979). Duncan et al. (2003) found the SRS to have strong internal consistency reliability, concurrent validity, and feasibility in comparison to longer, commonly used alliance measures, such as the Working Alliance Inventory (Horvath & Greenberg, 1989).
Data collection/gathering techniques: How will you collect/gather the information?
Practitioners will provide their clients with the information sheet about FIT and the present study to obtain informed consent. From the first meeting, the practitioner will administer the ORS and SRS, and each session thereafter, or phase of residential treatment. Practitioners will store their Excel spreadsheet on a password protected hard drive. They will email their spreadsheet to the chief researcher quarterly for outcomes to be tracked overtime, similar to the Goldberg et al. (2016) study.
Impact of and response to participant withdrawal
There will be no change in service delivery if a client decides to withdrawal from the study and their practitioner can delete their outcome and alliance data from the spreadsheet at any time. Practitioners will provide this information to their clients upon seeking informed consent. If a practitioner elects to withdrawal from the study, the chief researcher will delete their spreadsheet from the password protected hard drive storing the practitioner outcomes.
Description of the data analysis process/es
Each practitioner’s outcome data will be used to calculate their overall effect size using Glass’s delta (). This data will be used to explore the variance in the practitioners’ outcomes. An effect size will be calculated quarterly to determine if practitioners’ outcomes change with time.
Bivariate correlation (Pearson’s r) will be used to determine if the many factors collected in the Excel spreadsheet correlate to the outcomes of these outdoor and adventure therapy practitioners’ outcomes. For example, alliance data from the SRS will be used to determine if, like in other studies, the client’s perception of the therapeutic alliance correlates with outcome in the outdoor and adventure therapies.
Results, Outcomes and Future Plans
Data from this pilot study will be published in peer-reviewed journals.
Plans for return of results of research to participants
The chief researcher will send quarterly reports to the participating practitioners containing an update about the progress of the research project and preliminary findings.
Plans for dissemination and publication of project outcomes
Data from this pilot study will be published in peer-reviewed journals.
Other potential uses of the data at the end of the project
If this pilot study reveals potential for ongoing research, the project could develop into a randomised clinical trial focused on what steps, such as ongoing training or modes of supervision, help practitioners improve their outcomes.
References
Bettmann, J. E., Gillis, H. L., Speelman, E. A., Parry, K. J., & Case, J. M. (2016). A meta-analysis of wilderness therapy outcomes for private pay clients. Journal of Child and Family Studies, 25(9), 2659–2673.
Bertolino, B., Bergmann, S. & Miller, S. D. (2012). What works in therapy: A primer. In B. Bertolino & S. D. Miller (Eds.). ICCE Manuals on Feedback-Informed Treatment (FIT) (Manual 1). International Center for Clinical Excellence.
Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16(3), 252–260.
Bowen, D. J., & Neill, J. T. (2013). A meta-analysis of adventure therapy outcomes and moderators. The Open Psychology Journal, 6(1), 28–53.
Chow, D. L., Miller, S. D., Seidel, J. A., Kane, R. T., Thornton, J. A., & Andrews, W. P. (2015). The role of deliberate practice in the development of highly effective psychotherapists. Psychotherapy, 52(3), 337–345.
Duncan, B. L., Miller, S. D., Sparks, J. A., Claud, D. A., Reynolds, L. R., Brown, J., & Johnson, L. D. (2003). The Session Rating Scale: Preliminary psychometric properties of a “working” alliance measure. Journal of Brief Therapy, 3(1), 3–12.
Dobud, W. W., Cavanaugh, D. L., & Harper, N. J. (2020). Adventure Therapy and Routine Outcome Monitoring of Treatment: The Time Is Now. Journal of Experiential Education, 43(3), 262–276.
Dobud, W. W., & Harper, N. J. (2018). Of Dodo birds and common factors: A scoping review of direct comparison trials in adventure therapy. Complementary Therapies in Clinical Practice, 31, 16–24.
Garcia, J. A., & Weisz, J. R. (2002). When youth mental health care stops: Therapeutic relationship problems and other reasons for ending youth outpatient treatment. Journal of Consulting and Clinical Psychology, 70(2), 439–443.
Goldberg, S. B., Rousmaniere, T., Miller, S. D., Whipple, J., Nielsen, S. L., Hoyt, W. T., & Wampold, B. E. (2016). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63(1), 1–11
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002). The psychotherapy dose‐response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9(3), 329–343.
Harper, N. J. & Dobud, N. J. (Eds.). (2020). Outdoor therapies: An introduction to practice, possibilities and critical perspectives. Routledge.
Harper, N. J. & Doherty, N. J. (2020). An introduction to outdoor therapies. In N. J. Harper & W. W. Dobud (Eds.) Outdoor therapies: An introduction to practice, possibilities, and critical perspectives (pp. 3–16). Routledge.
Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 64, 223–233.
Lambert, M. J., Hansen, N. B., Umphress, V., Lunnen, K., Okiishi, J., Burlingame, et al. (1996). Administration and scoring manual for the OQ-45.2. Stevenson, MD: American Professional Credentialing Services.
Lee, I., Choi, H., Bang, K. S., Kim, S., Song, M., & Lee, B. (2017). Effects of forest therapy on depressive symptoms among adults: A systematic review. International Journal of Environmental Research and Public Health, 14(3), 1–18.
Lutz, W., Leon, S. C., Martinovich, Z., Lyons, J. S., & Stiles, W. B. (2007). Therapist effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counseling Psychology, 54(1), 32–39.
Miller, S. D., Duncan, B. L., Brown, J., Sparks, J. A., & Claud, D. A. (2003). The outcome rating scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. Journal of Brief Therapy, 2(2), 91–100.
Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449–457.
Norton, C. L., Tucker, A., Russell, K. C., Bettmann, J. E., Gass, M. A., Gillis, H. L., & Behrens, E. (2014). Adventure therapy with youth. Journal of Experiential Education, 37(1), 46–59.
Seidel, J. & Miller, S. D., (2012). Documenting change: A primer on measurement. In B. Bertolino & S. D. Miller (Eds.). ICCE Manuals on Feedback-Informed Treatment (FIT) (Manual 4). International Center for Clinical Excellence.
Schuckard, E., Miller, S. D., Hubble, M. A. (2017). Feedback-informed treatment: Historical and empirical foundations. In D. S. Prescott, S. D. Miller, & C. L. Maeschalck (Eds.). Feedback-informed treatment in clinical practice: Reaching for excellence (pp. 13–36). International Center for Clinical Excellence
