Essay title - Client Satisfaction and Perception of Alliance in Online Asynchronous Counselling
Introduction
It has previously been reported that adolescents, the highest risk group for suicide and mental health difficulties, do not seek formal help, but turn to informal sources of help such as their peers or the internet when faced with emotional difficulties (Rickwood & Braithwaite, 1994; Richards, Richardson & Pluincéid, 2006; Ryan, Culver, Richardson, O’Connor, McCormick & Pluincéid, 2007).
Tailored adolescent counselling services, which provide an anonymous service, should be expanded and developed, enabling them to bring mental health support to those at risk of difficulty and emotional hardship rather than forcing them to make that first, most difficult step. Further, this mental health support should be normalised by society so as individuals are no longer seen as weak for seeking out support, rather they are seen as strong for being able to admit when they need help.
Statistically, the Republic of Ireland is burdened with the 5th highest suicide rate in the EU, a rate double that of our nearest neighbours, the United Kingdom (WHO, 2005; as cited in Day & Donaghy, 2007). Fortunately, this is a burden which can be tackled head on and the rate of suicide can be greatly reduced, provided adequate and tailored services are put in place in all areas of life on this island. 46% of males report some form of suicidal ideation (Lynch, et al., 2007; as cited in Connolly, 2007), but 39% of males in the 18-24 age bracket cannot name any suicide or depression support service. In Ireland alone, more than 900 cases of suicide are reported each year (Day & Donaghy, 2007) - an average of more than 2 deaths every day, with estimates of unreported suicide at 1.3-1.6 times this figure. In excess of these completed suicides, each year approximately 16,000 cases of Deliberate Self Harm (DSH) are reported to the Irish health sector. Estimates put the actual figure of DSH occurrence in Ireland at 4-5 times this (Day & Donaghy, 2007).
80% of suicidal individuals give an indication of their planned behaviour before carrying out a suicide attempt. If this indication were taken seriously and listened to, the resulting attempt would not occur in a majority of cases. It is unfair to always be reporting to one’s friend when feeling like there is no other way out and therefore a service should be in place 24/7 where individuals are able to access support and feel cared for, respected and valued. Even during normal office hours, it is easier to turn to an online service for support remotely rather than physically going to a clinic or adult professional (C. Vernon, personal communication, November 21st 2007). The importance of these services being accessible 24 hours a day, 7 days a week cannot be overemphasised. According to the National Report on Deliberate Self Harm, 46% of presentations of DSH to hospitals are made between the hours of 8pm and 4am. These hours are also those during which individuals are at the highest risk of suicide and so it is important that there be a service available to them that they feel comfortable and safe accessing. As previously stated, adolescents and young adults much prefer using informal sources of advice, information and counsel such as the internet or their peers. During the hours of highest risk, it can be inconvenient or unfair to get in touch with a close friend or confidant. In fact, 40% of individuals who engage in self harming behaviour report lacking a confidant, so this option is not even available to them. A feeling of guilt on the part of the depressed individual can also easily exacerbate symptoms and emotions – and so anonymous confidential support should be provided in another means – through the medium of the internet.
The Internet and Mental Health
The use of the internet in mental health support and education is a relatively new idea, with the technology to facilitate such online interactions only becoming readily available in a user-friendly manner in the past decade (Ritterband, 2003). In the literature, a number of concerns with this kind of intervention have, time and again, been highlighted (Rochlen, Zack & Speyer, 2004; Cook & Doyle, 2002; Barak, 1999; Leibert, Archer Jr., Munson & York, 2006). These include: lack of nonverbal cues, accountability and security, and excessive anonymity.
The lack of nonverbal cues in online counselling is a major sticking point of most developments in this field, especially since availability of videoconferencing facilities is still far from widespread (Cook & Doyle, 2002). It is suggested that the lack of these non-verbal cues may impede the client-therapist alliance, from both perspectives, as the individuals involved don’t feel as strong a bond as might be experienced in face-to-face counselling. This can result in a feeling of being patronised or judged by the counsellor (Haberstroh, Duffey, Evans, Gee & Trepal, 2007). It can also limit self-expression and trust, further reducing satisfaction & alliance within the therapeutic relationship.
In a field where individuals disclose their deepest concerns and cognitions to relative strangers, it is imperative that checks & balances are in place to ensure that those engaging in counselling are properly qualified to do so (Stricker, 1996; as cited in Barak, 1999). Further, the privacy of clients in the mental health field is of paramount importance. While face-to-face counselling tends to be quite secure, online therapy, which involves text-based communication (whether synchronous or asynchronous), can be subject to tapping & interception. This is an issue which must not only be tackled head on to ensure the proper and safe storing of sensitive information (Rochlen, et al., 2004), but also to ensure that clients are made feel as at ease as possible from the beginning of their therapeutic relationship and not have to worry about unapproved third parties being aware of the communications engaged in between client & counsellor (Barak, 1999).
Though a certain level of anonymity is clearly required for the online relationship to flourish (Leibert, et al., 2006), an excess of anonymity can result in a dangerous situation developing wherein the counsellor has no real-world details about their clients in case of crisis. Childress (1998, as cited in Rochlen et al., 2004) however points out that the majority of websites require a user profile be set up (including basic contact information) before any formal interaction with the website and its services can occur.
There are, however, a number of very real benefits relating to the use of the internet for this purpose. Rochlen et al. (2004) outline how useful it is that information is readily available to counsellors operating through the internet. Though they are unable to physically hand resources to their clients, counsellors can direct their clients to the limitless resources available online. This also expands the resources they can provide their clients with, as they are not limited to the leaflets or books they have on their shelves (Grohol, 2000; as cited in Rochlen et al., 2004).
Griffiths, Lindenmeyer, Powell, Lowe & Thorogood (2006) suggest that through the medium of the internet, social stigma may be removed. This is due to the fact that users access the internet from the privacy of their own computers in their own homes, and therefore are protected from the judgements, whether real or perceived, of others.
Physical invisibility and the resulting anonymity for users also result in an amplification of the dis-inhibition effect which leads to increased honesty and personal disclosure. Joinson (1998, as cited in Rochlen et al., 2004) even suggests this level of dis-inhibition to be unique to the internet. As previously mentioned, users often are required to create a profile for themselves, providing a pseudonym for online interaction, and this may increase participation and therapeutic effectiveness; Kummervold, Gammon, Bergvik, Johnsen, Hasvold & Rosenvinge (2002) found that 64% of respondents found it very important that their real name was not required to participate in a mental health discussion board.
Another benefit of using the internet for mental health support is the previously non-existent Zone of Reflection (Rochlen et al., 2004). This is a phenomenon which can occur in asynchronous support due to the time delay in correspondence. This delay gives both parties the chance to reflect on the wording of their responses, providing the opportunity not to “stutter, and…pause”, which can be off-putting in everyday speech but in online communication is much more accepted, but rather construct a thoughtful response (Haberstroh, et al., 2007).
One of the other phenomena unique to text based counselling, whether online or in a face-to-face setting has been outlined on a number of occasions, and is named after the individual who originally uncovered its effectiveness - The Pennebaker effect (1997; as cited in Rochlen et al., 2004). This effect is the positive impact that simply writing about traumatic or concerning events can have.
Though the extent to which therapies are suited for online use has yet to be fully established (Johnsen, Rosenvinge, & Gammon, 2002), Laszlo, Esterman & Zabko (1999; as cited in Johnsen et al., 2002) suggest that cognitive behavioural therapies are very suitable for use in a text-based medium such as the internet. Mitchell & Gordon (2007) have shown how accepting of CCBT (Computerised Cognitive Behavioural Therapy) students are, especially after being shown a demonstration of or partaking in CCBT. This suggests that computerised therapies would be ideal for student populations and could reach a much broader population not only through the anonymous publishing of correspondence (Barak, 1999) but also through the fact that individuals, particularly students, are prone to using informal resources such as the internet for help rather than any other resource.
Two clients in the Leibert et al. (2006) study outlined a deficit in online counselling. They highlighted the inability of the counsellor to intervene in a time of crisis – an important issue in online counselling. Due to the fact that this intervention had previously been contraindicated for those prone to suicidal ideation (Rochlen et al., 2004), this finding was alluded to but not elaborated on. In fact, the archiving of user details in case of emergency as reported by Barak (1999) in an effort to reduce the inability of an online counsellor to intervene in a crisis is becoming more common a practice as the use of the internet for counselling increases.
Working Alliance
Working Alliance, as a concept, has existed from as far back as the beginning of psychotherapy. Over time, various researchers have seen working alliance as describing various aspects of the client-therapist relationship. These have encompassed the understanding of the client by the therapist, the identification of the client with the therapist and the motivation of the client to work within the treatment situation (Tichenor & Hill, 1989). Working alliance is currently broadly seen as being composed of three aspects – agreement on goals, assignment of tasks and development of bonds (Bordin, 1979).
Walther & Burgoon (1992; as cited in Cook & Doyle, 2002) found that relationships formed through computer-based interaction developed in almost identical fashion to relationships formed in face-to-face settings. This would seem to suggest that relationships develop in the same way, regardless of the medium, a finding further supported by Parks & Roberts’ study into Multi-User Dimensions (1998; as cited in Cook & Doyle, 2002) which found that over 90% of their sample had formed ongoing relationships. These relationships, when compared with offline relationships, were shown to have no difference in breadth or depth achieved. However, personal and therapeutic relationships can differ in many ways, ranging from methods of interaction to interdependency to topics of intellectual discussion (Levenkron, 1998). As a result of these differences, the development of therapeutic relationships should be investigated, especially in terms of differences when developed online.
Though finding self-help material online requires no interaction with others, online counselling is, by its nature, different. It involves contact between counsellor and client in all cases, and even between several clients in group therapies on occasion. The effectiveness of these therapies is, according to Bordin (1979) a function, at least in part, of the strength of the working alliance between the parties involved in the therapy. Due to online therapy being a relatively new endeavour in the field of psychological support, the outcomes and processes relating to the relationships formed in this area are of crucial importance. A broad variety of process measures have been developed in the past (McLeod, 2003).
However, for the present study, it was considered pertinent to focus on evaluating processes which might be impacted by the nature of online counselling. In online therapies, it is important that Bordin’s aspects of the relationship are given special consideration. The fact that the internet gives rise to intense and in depth relationships being formed quickly & easily (Cook & Doyle, 2002) could make up for any relationship impairment caused by the lack of physical interaction – a concern raised by various authors in this field (Cooke & Doyle, 2002; Haberstroh, et al., 2007; Rochlen et al., 2004).
Horvath & Greenburg (1986, 1989; as cited in Hatcher & Gillaspy, 2006) developed the Working Alliance Inventory in an effort to assess the three facets of Bordin’s construct. Over the course of the last 20 years, their inventory has been used in a variety of therapeutic settings to gauge effectiveness as a result of the working alliance. Tracey & Kokotovic (1989) proposed a short form of the inventory, 12 items long (rather than the original 36). Their WAI-S was compiled by taking the 4 items with the largest factor loadings from the original 3 scales only. Confirmatory factor analysis showed that WAI and WAI-S ratings illustrated a two-level structure – General Working Alliance factor & three other factors – one specific to each of the original subscales.
Cook & Doyle (2002) further outline that the construct validity of the WAI is provided with added support by the high intercorrelations with other alliance instruments. Tichenor & Hill (1989) compared 6 working alliance tools and found that scores on the WAI, when completed by an observer of a therapeutic session, correlated highly with three other working alliance instruments completed by other observers.
Busseri & Tyler (2003) once again gauged the interchangeability of the two inventories. They gathered data after the fourth session and also at a further later stage. It was found that the WAI and WAI-S were interchangeable, with highly similar scores being achieved on each inventory. This resulted in the authors questioning the advantage of using the full questionnaire over the quicker, similarly effective inventory, especially in settings where the inventory is completed on a number of occasions over the course of the therapeutic relationship.
However, Eaton, Abeles & Gutfreund (1988) found the level of alliance to be established within the first three therapeutic interactions, regardless of the eventual length of therapy, so perhaps it would be wise to question the repeated administration of the WAI or WAI-S when the alliance is already reasonably concretely established. Given this evident establishment, the first three sessions are crucial to forming a strong working alliance.
A revised short form of the WAI (WAI-SR) was recently validated (Hatcher & Gillaspy, 2006). This version was found to have greater differentiation between the goal & task scales than the WAI. It is suggested that the new task scale is concerned more with the client’s sense of proposed treatment showing promise towards reaching the client’s goals. It is theorised that higher scores on the revised form would result in greater client commitment due to greater promise of results. So, if online counsellors are able to engage their clients in a way which will give greater scores on this scale, it is more likely that the clients will engage in repeated correspondence, furthering the development of a strong working alliance between counsellor & client.
As outlined by Barak (1999), many online therapeutic relationships involve a single encounter between therapist & client through anonymous portals. It is imperative, then, for alliance to be formed almost instantaneously. The fact that the internet gives rise to intense and in depth relationships being formed quickly & easily (Cook & Doyle, 2002) could play a pivotal role in increasing levels of alliance within an online therapeutic setting.
Client Satisfaction Inventory
One of the most important outcomes of therapeutic relationships is the client’s satisfaction levels with therapy upon its completion. It is an appealing means of measuring service efficacy and quality due to it being easily understood by clients and also to it being measured by self-report (McMurtry & Hudson, 2000). Some of the seminal works in service evaluation and research (Bloom, Fischer & Orme, 1999 and Blythe, Tripodi & Briar, 1994; as cited in McMurtry & Hudson, 2000), however, only allude to client satisfaction as an outcome measure and means of easily assessing services and do not go into any great detail about harnessing the full potential of a gauge of client satisfaction.
Over time, it has been found that researchers come up with their own means and methods of gauging client satisfaction (Larsen, Attkinson, Hargreaves & Nguyen, 1979) due to the perceived easiness of measuring client satisfaction. However, recently a number of standardised questionnaires have become prominent, tailored for specific areas of service provision (such as in psychiatric, HIV, long-term nursing and community mental health care). These scales measured specific aspects of client satisfaction including, in some cases, parental satisfaction with services provided for their children (Clare & Pistrang, 1995; as cited in Russell, 2004), which was important for the specific service being provided. Although there was a general, standardised scale for measuring client satisfaction with a service, the Client Satisfaction Questionnaire (Larsen, et al., 1979; cited in McMurtry & Hudson, 2000) was unidimensional and due to its four point likert scale, was both unable to record neutrality and less sensitive than a five or seven point scale.
The short form of the Client Satisfaction Inventory (McMurtry & Hudson, 2000) was developed and validated concurrently with the longer, standard Client Satisfaction Inventory. It was found to have high internal consistency. It was also illustrated to not be impacted by the mood of the client at the time of completion – a feature which is important in gauging satisfaction over an extended period. Further, due to the CSI-SF comprising only 9 items, it can be completed by clients swiftly and easily having completed a therapeutic session. This also makes it an ideal measure for gauging satisfaction with online counselling – very little extra time is taken up, and due to the fact that no qualitative disclosure occurs, clients could be much more comfortable disclosing the relevant data online.
Reynolds, Stiles & Grohol (2006) used the Session Evaluation Questionnaire as a means of gauging session impact, which can be likened to client satisfaction, though on a much more fluid basis, as it reflects attitudes after each individual session. They found that the online participants rated feeling either equally or more satisfied than their face-to-face counterparts with their therapeutic interactions.
In a study using the CSI-SF, Leibert et al. (2006) showed that although clients using online treatment do find the service satisfactory, those engaging in face-to-face counselling are satisfied to a greater level. This could be of some concern to both counsellors in the field of online therapy and also face-to-face counsellors who are contemplating referring clients to online services whether as an alternative or supplement to their treatment. It must be noted that the participants in this study were above the normal age range of university students and only one third of them had completed some 3rd level education – it is possible that a younger more technology aware sample would have found the service more satisfying.
However, the levels of satisfaction which are experienced by online clients are repeatedly reported as being high (Christensen et al., 2002, Kids Help Line, 2002; as cited in Brown, 2003), suggesting that though the levels may not be akin to those associated with face-to-face counselling, they are still sufficiently high to allow the field to continue and expand – possibly eventually reaching a level of satisfaction closer to that associated with face-to-face counselling. This could happen through clients becoming more used to the newer mode of counselling or the technology developing sufficiently to enable those engaging with online counselling to forget that they are not in a therapist’s office.
Helpful Aspects of Therapy
Though therapy primarily focuses on negative events and impacts on individuals’ lives in an effort to minimise the impact associated with such events and to prevent them from re-occurring, recently much more focus has been directed towards identifying and creating good moments during therapy (Mahrer & Nadler, 1986; as cited in Timulak & Lietaer, 2001). The impact of these moments, whether they be the emergence of previously warded-off material; the expression of strong feelings, a state or wellbeing or a different personality state; or even learning about the therapeutic process has been researched (Martin & Stelmaczonek, 1988).
It was found that such positive or helpful moments could be recalled up to six months after completion of therapy, which indicates the extent of the impact these events had on the clients. It is possible that these events were easier to recall due to the short term therapy undertaken – there were less events overall to recall. Due to most online counselling taking place in email correspondence of three or fewer interactions, the positive events which do occur would have the same salience and consequently the same positive impact as the positive events recalled in the Martin & Stelmaczonek study.
These positive moments are also abundant in counselling sessions, with Timulak & Lietaer (2001) identifying, with the help of the clients involved in the study, 38 positive moments in 19 counselling sessions, approximately one every 25 minutes. The main positive feeling experienced by the clients in this study was feeling empowered. This feeling was involved in 50% of the positive moments and was experienced by all clients in the study.
Though the presence of the researcher and recording equipment may have impacted on the quality of the therapy sessions studied (Timulak & Lietaer, 2001), in an online setting, especially when research occurs upon completion of the therapeutic relationship, this confound would not impact upon the therapy, and even more helpful aspects of therapy could come to the surface.
One client in the Timulak & Lietaer (2001) study could see “in [the counsellor’s] eyes” that the counsellor understood them. As most online counselling occurs asynchronously and, when synchronous, without live imaging (Barak, 1999), this sort of positive impact would not yet be transferable to online counselling. However, it is not unimaginable that over the next decade the technology could become available for such high-quality videoconference interactions to occur (Rochlen, et al., 2004), enabling visual cues to give rise to more helpful aspects of therapy which are currently non-existent in online counselling.
Timulak (2007) outlined nine core categories of helpful events, eight of which could easily transfer and be found in an online counselling setting. The one which could not transfer, personal contact, has as previously mentioned, been cited repeatedly as one of the primary pitfalls in online counselling.
The remaining eight categories are all measurable by the Helpful Aspects of Therapy form (Llewellyn, 1985; as cited in Llewellyn, Elliott, Shapiro, Firth & Hardy, 1988) due to its open-ended and succinct nature. This quality also makes it ideal for use in an online setting, where time is seen to be of great importance – it is seen as a quick and easy form, and the open ended questions allow for cutting & pasting of relevant sections from the email exchange engaged in by client and counsellor. Also, having the exchanges saved and accessible on the computer for when the individual client needs to go back to them in the future could be a helpful aspect of online therapy.
Conclusion
The internet is more and more commonly associated with instant communication, instant service provision, instant gratification of human wants and needs (Paulson, 2007). For this reason, online research, in order to ensure being engaged with by participants, necessitates the use of simple and short measures.
Due to young adults’ predisposition to mental ill-health and apparent aversion to accessing traditional support services, it is felt that the setting up and maintenance of non-traditional mental health services are of paramount importance. Auditing of these services must occur on a regular basis in order to ensure that they are functioning to reasonable standards and are providing an appropriate and tailored service.
The current study aims to gauge satisfaction with, alliance within and helpful aspects of online counselling. The three instruments being used in order to achieve this have, as outlined above, previously been validated. It is hypothesised that, even in single email exchanges with a counsellor, high levels of alliance will be reported. It is further hypothesised that the online support being provided by the Trinity College Dublin Student Counselling Service will result in clients reporting high levels of satisfaction with their counselling experience. Finally, in congruence with Timulak & Lietaer (2001), it is hypothesised that clients will report a significant number of helpful events having occurred during their interactions with a counsellor online.
References
Barak, A. (1999). Psychological applications on the Internet: A discipline on the threshold of a new millennium. Applied & Preventive Psychology, 8, 231-245.
Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the Working Alliance. Psychotherapy: Theory, Research and Practice, 16(3), 252-260.
Brown, J. (2003, October). Online counselling – The OTEN experience and implications for the future. Presentation at 16th ODLAA Biennial Forum Conference, Canberra, Australia.
Busseri, M. A., & Tyler, J. D. (2003) Interchangeability of the Working Alliance Inventory and Working Alliance Inventory, Short Form. Psychological Assessment, 15(2), 193-197.
Connolly, J. (2007, August). Suicide in Ireland. Presentation at the biennial meeting of the International Association of Suicide Prevention, Killarney, Ireland.
Cook, J.E., & Doyle, C. (2002). Working alliance in online therapy as compared to face-to-face therapy: Preliminary results. Cyberpsychology & Behavior, 5(2), 95-105.
Day, J, & Donaghy, C. (2007, August). Suicide prevention on the island of Ireland. Presentation at the biennial meeting of the International Association of Suicide Prevention, Killarney, Ireland.
Eaton, T. T., Abeles, N., & Gutfreund, M. J. (1988). Therapeutic alliance and outcome: Impact of treatment length and pretreatment symptomatology. Psychotherapy, 25, 536-542.
Griffiths, F., Lindenmeyer, A., Powell, J., Lowe, P., & Thorogood, M. (2006). Why are health care interventions delivered over the internet? A systematic review of the published literature. Journal of Medical Internet Research, 8(2), e10.
Haberstroh, S., Duffey, T., Evans, M., Gee R., & Trepal, H. (2007). The experience of online counselling. Journal of Mental Health Counselling, 29 (3), 269-282.
Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a revised short version of the Working Alliance Inventory. Psychotherapy Research. 16(1), 12-25.
Johnsen, J. A., Rosenvinge, J. H., & Gammon, D. (2002). Online group interaction and mental health: an analysis of three online discussion forums. Scandanavian Journal of Psychology, 43(5), 445-449.
Kummervold, P. E., Gammon, D., Bergvik, S., Johnsen, J-A. K., Hasvold, T., & Rosenvinge, J. H. (2002). Social support in a wired world: Use of online mental health forums in Norway. Nordic Journal of Psychiatry, 56(1), 59-65.
Larsen, D., Attkinson, C., Hargreaves, W., & Nguyen, T. (1979). Assessment of client patient satisfaction: Development of a general scale. Evaluation and Program Planning, 2, 197–207.
Leibert, T., Archer Jr., J., Munson, J., & York, G. (2006). An exploratory study of client perceptions of internet counselling and the therapeutic alliance. Journal of Mental Health Counselling, 28(1), 69-83.
Levenkron, S. (1998). Cutting: Understanding and Overcoming Self-Mutilation. London: Norton.
Llewellyn, S. P., Elliott, R., Shapiro, D. A., Hardy, G., & Firth-Cozens, J. (1988). Client perceptions of significant events in prescriptive and exploratory periods of individual therapy. British Journal of Clinical Psychology, 27, 105-114.
Martin J., & Stelmaczonek, K. (1988). Participants’ identification and recall of important events in counselling. Journal of Counselling Psychology, 35(4), 385-390.
McLeod, J. (2003) Doing Counselling Research. London: Sage.
McMurtry, S. L., & Hudson, W. W. (2000). The Client Satisfaction Inventory: Results of an initial validation study. Research on Social Work Practice, 10(5), 644-663.
Mitchell, N, & Gordon, P. K. (2007). Attitudes towards computerized CBT for depression amongst a student population. Behavioural and Cognitive Psychotherapy, 35, 421-430.
Paulson, C. M. (2007). Does the Internet Fuel Our Need for Instant Gratification? Retrieved November 22nd 2007 from http://www.associatedcontent.com/article/161013/does_the_internet_fuel_our_need_for.html
Reynolds, D. J., Stiles, W. B., & Grohol, J. M. (2006). An investigation of session impact and alliance in internet based psychotherapy: Preliminary results. Counselling & Psychotherapy Research, 6(3), 164-168.
Russell, F. (2004). Partnership with parents of disabled children in research? Journal of Research in Special Educational Needs, 4(2), 74-81.
Richards, D., Richardson T., & Pluincéid, E. (2006). Student mental health support and education via peer moderated online discussion boards: A Pilot Study. Unpublished paper, Trinity College Dublin.
Rickwood, D. J., & Braithwaite, V. A. (1994). Social-psychological factors affecting help-seeking for emotional problems. Social Science & Medicine, 39(4), 563-572.
Ritterband, L, M., Gonder-Frederick, L, A., Cox, D, J., Clifton, A, D., West, R, W. & Borowitz, S, M. (2003). Internet Interventions: In Review, In Use, and Into the Future. Professional Psychology: Research and Practice, 34(5), 527-534.
Rochlen, A. B., Zack, J. S., & Speyer, C. (2004). Online therapy: Review of relevant definitions, debates, and current empirical support. Journal of Clinical Psychology, 60(3), 269-283.
Ryan, M., Culver, D., Pluincéid, E., O’Connor, A., McCormick M., & Richardson, T. (2007). Student Mental Health: Awareness, Coping and Use of Services. Unpublished undergraduate group project, Trinity College Dublin.
Tichenor, V., & Hill, C. E. (1989). A comparison of six measures of working alliance. Psychotherapy, 26, 195-199.
Timulak, L. (2007). Identifying core categories of client-identified impact of helpful events in psychotherapy: A qualitative meta-analysis. Psychotherapy Research, 17(3), 305-314.
Timulak, L., & Lietaer, G. (2001). Moments of empowerment: A qualitative analysis of positively experienced episodes in brief person-centred counselling. Counselling & Psychotherapy Research, 1(1), 62-73.
Tracey, T. J., & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. Psychological Assessment, 1:207-210.
