Psychology Final Assignment Week 7
nikkieramsey
Draft_MarthaRamsey1.docx
Online versus In-Person Counseling: A Review of Existing Studies
Martha Ramsey Saint Leo University
Capsthesis/Capstone Project II: PSY 695 Instructor Lara Ault
April 23, 2024
13
Abstract
Online psychotherapy and in-person counseling differ with respect to their effectiveness. The current study was guided by a set of research questions which were considered in its review approach. The search for the articles was restricted to the following search terms and phrases, online therapy mental health outcomes, telepsychiatry effectiveness, internet-based interventions for mental health, and the advantages and disadvantages of online counseling. The search was confined to English language articles published in the year 2019 to the year 2024. The chosen time frame coincided with the period of the COVID19 pandemic. This also marked the period when the use of online psychotherapy became common place due to lockdowns and restricted movements. The subjects in the articles were those that had participated in online psychotherapy. A total of 15 peer-reviewed sources were selected for the review after consideration of various criteria. The selected articles not only met the criteria, but also satisfied the requirements for research subjects, method, variables, and research outcomes. The findings of the study established that in-person counseling was favored for effectiveness while online psychotherapy was favored for convenience and affordability.
Online versus In-Person Counseling: A Review of Existing Studies
Introduction
With the advancement in internet and digital technology, access to critical services such as healthcare and support services has become easier (Greenwood et al., 2022). The proliferation of internet technology and online communication has enabled accessibility to several health services. Mental health services remain among the healthcare services that can be accessed through online interaction with experts (Khan et al., 2021). However, the question as to whether online counseling is more effective compared to in-person counseling widely remained answered. Whereas online counseling can equally impact patients in reducing psychological distresses, its effectiveness in terms of creating personal connections with the therapists, increasing level of satisfaction after use, being executed devoid of distractions remain areas of concern (Békés et al., 2021). Online counseling therapy has always been associated with ease of access and affordability thereby making it considered by patients who give preferences to these factors (Ierardi et al., 2022). Barriers such as location of counseling services, stigma, and time commitment make some patients prefer online or digital psychotherapy to in-person counseling (Renn et al., 2019).
The current study focused on the effectiveness of internet therapy as compared to traditional in-person counseling in the prevention and management of mental health challenges. The study took a narrative review approach using a systematic search strategy that considered articles published in the year 2019 up to the year 2024. The selected time frame factored in the issue of mental health challenges during the COVID19-pandemic because online psychotherapy was significantly used during the pandemic period due to lockdowns and constrained movements of people. The researcher intended to use many articles for this purpose.
However, this was narrowed down to a manageable size by relying on the research questions, research problem, and research purpose. There are parameters that guided the inclusion and exclusion criteria for the articles that were used in the study. The use of English language in the writing and publication of an article was a mandatory requirement during the evaluation. The study prioritized peer-reviewed journal articles in the review. Conference abstract, non-peer reviewed articles, and grey literature were not considered for the evaluation.
Problem Statement: The use of online counseling and therapy versus in-person counseling in terms of effectiveness widely remains unexplored with respect to review of the existing studies.
Purpose of the Review: The current narrative review provides an in-depth analysis of current literature on the effectiveness of internet-based therapy compared to in-person counseling and therapy. The review provides information that can help patients and mental healthcare professionals make informed decisions when using these approaches to counseling and therapy. The selected articles for the review contained subjects who had used both online psychotherapy and in-person counseling. These selected articles provided information that was used in the comparison of the types of counseling in terms of their effectiveness.
Research Questions
1. How effective is online psychotherapy in terms of client satisfaction and personal interaction as compared to traditional in-person therapy?
2. What are the advantages and disadvantages of online therapy?
3. How is online therapy is perceived either negatively or positively by the young and old demographics?
Method
The current paper utilized a narrative review with a systematic search strategy. No measurement tools or devices were used as a guideline in obtaining the information for the research. The search for studies that were included in the review was guided by the topic and research questions. The researcher considered several constraints that helped to narrow down the materials that were selected for the review. English language criterion for journalism was considered a critical component in the evaluation. English language use was a key consideration because studies and papers published in non-English languages were not only challenging to comprehend and synthesize but could also comprise the rigor of the review. Several websites that included PubMed, PsycINFO, and Google Scholar were explored for the purpose of extracting the right articles. The search terms included “online psychotherapy and mental health outcomes”, “virtual counseling or virtual psychotherapy”, “telepsychiatry effectiveness”, “internet-based interventions for mental health”, “advantages and disadvantages of online counseling”
Eligibility Criteria
Peer-reviewed articles published between year 2019 to the year 2024 were prioritized to maintain the rigor of the review. The selected time frame limited the review because the search targeted peer-reviewed articles which were published during the COVID19-pandemic.
Additionally, online psychotherapy was frequently used during the pandemic more than any other time in the history of the United States of America. The search process was confined to search terms that included online psychotherapy mental health outcomes, telepsychiatry effectiveness, internet-based interventions for mental health, and advantages and disadvantages of online counselling. When the search phrase online psychotherapy and mental health outcomes was searched, 3500 materials were found from the Saint Leo University library. When the search term was paraphrased to focus on peer-reviewed articles between the year 2019 to the year 2024, the search yielded 89 articles. When the terms virtual counseling or virtual psychotherapy were
used, the search yielded 389 materials from Saint University Library, 28 materials from PubMed, 6 materials from PsycINFO, and 89 materials from Google Scholar. When the search term was paraphrased to include peer-reviewed articles published between the year 2019 to the year 2024, only 12 articles were found in the University Library, 34 articles were found in Google Scholar, 3 articles were retrieved from PubMed, and 2 articles were found in PsycINFO.
The search term telepsychiatry effectiveness yielded 600 articles from the Saint Leo University Library and 891 articles from Google Scholar, PubMed, and PsycINFO. This particular search also yielded a total of 69 books from all these libraries. When the search was narrowed to peer-reviewed articles from the year 2019 to the year 2024 a total of 19 articles were retrieved from these libraries. The search term internet-based interventions for mental health yielded a total of 96 materials from Saint Leo University, 697 materials from Google Scholar, PsycINFO, and PubMed. When the search was narrowed by paraphrasing the term to include peer-reviewed articles published in the 2019 to the year 2024, a total 27 peer-reviewed articles were found from these libraries. When search term advantages and disadvantages of online counseling was used, a total of 123 articles were retrieved from Saint Leo University Library, 368 articles were found in Google Scholar, PsycINFO, and PubMed. When search was narrowed to peer-reviewed articles published in the year 2019 to the year 2024, a total of 29 articles were retrieved from these libraries. The narrowed search based on peer-reviewed articles published between the year 2019 to the year 2024 yielded a total of 134 articles for all the search terms used.
Table for inclusion and exclusion criteria
|
Inclusion criteria |
Exclusion criteria |
|
· Peer-reviewed articles published between the years 2019 to 2024 · Articles focusing on online psychotherapy, telepsychiatry effectiveness, internet-based interventions for mental health, and advantages and disadvantages of online counseling · Articles discussing the experiences, effects, and perceptions of online therapy · Studies involving counselors, patients, or individuals with mental health challenges as subjects · Articles considering the impact of the COVID-19 pandemic on online therapy |
· Articles not published in peer-reviewed journals · Articles not related to the specified topics · Articles not addressing the specified themes · Studies not involving any of the specified subjects · Articles not addressing the influence of the pandemic
|
Quality Evaluation
that did not factor in the issue of COVID19-pandemic were not considered for the evaluation. This was a critical factor for evaluation because online psychotherapy was frequent during the pandemic. A total of 78 articles out of the 134 articles did not meet this quality consideration. The next factor considered was the subjects used in the study. The researcher considered 56 articles that used either counselors, patients, or individuals who experienced mental health challenges. 18 of the remaining 56 articles did not meet this requirement. Next the research considered the outcomes of the studies which were either categorized in terms of experiences, effects, and perceptions. 15 of the remaining 38 articles met this requirement. The researcher was therefore left with 15 articles which were used in the review and evaluated in the attached box score.
The 15 selected articles applied research methodologies that were not narrative reviews. For instance, Ierardi et at. (2022) assessed psychological problems that were experienced by university students in a symptom checklist. A questionnaire was used to record their experiences including life satisfaction after online counseling. Greenwood et al. (2022) relied on randomized controlled trials that compared telehealth counseling to face-to-face counseling. Renn et al. (2019) used online surveys to document patients’ choices on online versus in-person counseling. Sockalingam et al. (2022) used telephone-based cognitive therapy to extract data, while Last et al. (2024) used semi-structured interviews to collect data. Ncheka et al. (2024) equally used semi-structured interviews to collect data.
Records excluded (n = 18). Reasons: Not current (Within the last 5 years), Not peer-reviewed
Studies included in the review (n = 15)
Full-text articles excluded, with reasons (n = 5) Reason: Not directly relevant to the chosen topic
Full-text articles assessed for eligibility (n = 38)
Records screened (n = 56)
Records after duplicates and non-English publication removed (n = 78)
Additional records identified through other sources (n = 0)
Identification
Eligibility
Included
Screening
Records identified through database searching (n = 134)
Review of Literature
This section of the paper performs a review of the selected studies in accordance with the study questions. The section has been broken into the below segments which directly conform with the research questions. The review was narrowed down to a total of 15-peer reviewed articles that met the selection and evaluation criteria.
Online Therapy
Numerous studies have looked at the subject of online therapy. However, only those studies that have compared online therapy to in-person counseling were considered for this study. The studies considered in this research were those that could give the researcher information in doing this comparison as well as answering other research questions. Online therapy was mentioned by Greenwood et al. (2022) who in their study compared telehealth versus face-to- face psychotherapy. Despite the fact that telehealth covers a wide range of health services that can be offered through the internet, Greenwood et al. (2022) not only mentioned psychotherapy as one of the telehealth services, but also focused their study on the comparison of psychotherapy versus face-to-face counseling. The article was therefore found to be relevant and applicable to the study. Greenwood conveys online psychotherapy as a new area that can help patients seek mental health services. The researchers also attributed the increased use of this method of psychotherapy to the COVID19 pandemic which made it difficult for both patients and healthcare providers to interact face-to-face. Online therapy was also mentioned by Khan et al. (2021) in their study that looked at the experiences of the counselors in online psychotherapy.
The study highlighted a number of factors that that affect counselors’ experience. Some of the factors are therapeutic alliance, convenience, online counseling skills, ethical and legal concerns, reaching out to diverse client base, client suitability for online psychotherapy, professional and personal goals, and client satisfaction.
Whereas Khan et al. (2021) mentioned psychotherapy by focusing on counselor experiences when providing this type of psychotherapy, Zeren et al. (2022) mentioned online psychotherapy but focused on client experiences of online counselling. Zeren et al. (2022) took cognizance of the fact that there are few studies that have investigated the experiences of the clients in this type of counseling. Their study focused on two key themes which were associated with client experiences. The themes were therapeutic alliance and patient experiences. Flexibility of online counseling, limitations of online counseling, and improvement in the symptoms were associated with patient satisfaction. On the other hand, therapeutic alliance was associated with themes such as the bond between the counselor and the client, tasks, and goals. Zeren et al. (2022) relied on semi-structured interviews to extract information from 6 client-participants.
Online therapy was also mentioned by Westerhof et al. (2019) who in their study looked at how online therapy could be used to overcome depressive symptoms when it was led by the counselor versus when it was supported by the peers. The results of the study established that counselor-led online psychotherapy was more effective than peer supported counseling in terms of reducing depressive symptoms.
Effectiveness of Online Therapy
Studies on online psychotherapy established that it was effective in helping patients overcome depressive symptoms. For instance, Ierardi et al. (2022) established that online psychotherapy was effective in helping patients reduce distresses such as anxiety, depression, and obsessive and compulsive disorders. The study took a comparative approach and showed that there were some differences in when online psychotherapy was compared to in-person counseling in terms of effectiveness. Whereas online psychotherapy showed effectiveness in managing psychological distresses in some subscales, in-person counselling was effective in managing psychological distresses in all the subscales that were considered for the study. Sander et al. (2021) conducted a study that looked into the effectiveness of online psychotherapy. The study established that online psychotherapy was effective in helping patients overcome mental health disorders. However, the method could not be adapted to special needs patients with severe mental illnesses.
Békés et al. (2021) conducted a study that investigated the effectiveness of online psychotherapy. Even through the study also took a comparative approach, it established that overtime online psychotherapy lost its effectiveness. Their study also raised concerns about connectedness between the counselor and the patient. Schuster et al. (2020) looked at the pros and cons of online psychotherapy and those of a blended model. The findings of the study showed that a blended model was more effective than online psychotherapy. In-person counseling was more effective compared to online psychotherapy in terms of creating personal interactions between the counselor and the client. Sockalingam et al. (2022) in their study that looked at the use of telephone-based psychotherapy for people with mental health distresses established that the method was effective in treating psychological disorders in patients with eating disorders. Xie et al. (2023) in their study established that there was short-term improvement associated with remote dignity therapy. Patients who underwent the treatment experienced reduction in dignity loss, gained more hope and meaning of life, and experienced reductions in depression and anxiety.
Accessibility and reach of online therapy
The use of online psychotherapy in the treatment of online psychological disorders is given very high priority because of the convenience. According to Renn et al. (2019), patients preferred online psychotherapy for depression and anxiety due to ease, privacy, and location. Easy location was a need for internet psychotherapy. The study also indicated that rural Americans with internet access can benefit from online psychotherapy due to its ease of access. Online cognitive-based psychotherapy for low mood was recommended to low-income students by Ncheka et al. (2024) because it was easy and affordable. The study which relied on student population also revealed that students or young people who were technologically savvy preferred online psychotherapy to in- person counseling. Accessibility in terms of demographics was also confirmed in the studies conducted by Ierardi et al. (2022) who showed that young individuals were more likely to subscribe to the use of online psychotherapy in the treatment of depressive disorders as compared to older adults.
Therapeutic Alliance and User Experience Online Therapy
Therapeutic alliance and user experience is another area that should be considered when determining the use and effectiveness of online psychotherapy. Khan et al. (2021) in their study found that therapeutic alliance is a key factor in determining user experience in online psychotherapy. Even though the study lauded online psychotherapy in helping clients deal with symptoms of psychological disorders, the study revealed that a strong therapeutic alliance or personal interaction between a counselor and a client was necessary in improving client experience. Therapeutic alliance and user experience was also mentioned by Zeren et al. (2022) in their study that investigated client’s experiences of online counseling. The study also considered patient satisfaction. The results of the study showed that tasks, bond, and goals were the themes associated with therapeutic alliance while improvement in symptoms and flexibility were associated with client satisfaction (Zeren et al., 2022). According to Zeren et al. (2022), clients who used online psychotherapy, reported positive experiences in relation to online counseling.
Box Score Table of studies and outcomes
Discussion
The review established that the experience of counselors was a key determinant in the success of online counseling. This was evident in the studies conducted by Khan et al. (2021). Their study also showed that therapeutic alliance and patient satisfaction were associated with a reduction in depressive symptoms, thus improving experiences of the users. Similar findings were found out by Zeren et al. (2022). The review also showed that even though online counseling was likely to be preferred with respect to convenience, cost effectiveness, and accessibility, in-person or face-to-face counseling was favored by majority of the subjects in the selected articles. The review explored the planning practices for public counseling therapists (Khan et al., 2021).
In an effectiveness-oriented study, Ierardi et al. (2022) established that online psychotherapy would be of benefit to a patient in cutting down on the effects that come with the symptoms of depression, including depression and anxiety. Through its comparative approach, the study established that face-to-face counseling would help clients to overcome all subscales of depressive disorders. Békés et al. (2021) noted that online psychotherapy was relatively effective in helping patients cope with depressive symptoms, although the effectiveness of this model diminished over time because of the failure to establish the interpersonal relationship between clients and their counsellors.
On a study by Schuster et al. (2020) that looked at the advantages and disadvantages of online psychotherapy and blended models, it maintained that the blended models were more effective than internet-based psychotherapy. In their study, Renn et al. (2019) noted that generally, clients use online psychotherapy based on reasons to do with accessibility, commitment to time, privacy, affordability, and location of services. Ncheka et al. (2024) confirmed the same regarding affordability and convenience, hence listing them as the main issues that make students from low-income settings use online psychotherapy. Along the same line, both Ncheka et al. (2024) and Ierardi et al. (2022) discuss the issue of demographics. The two studies revealed why young individuals were more likely to use online psychotherapy instead of in-person counseling. Unlike older adults, young individuals are technologically savvy and prefer to use online-psychotherapy due to cost considerations (Ierardi et al., 2022).
Limitation
Limitations constrain the generalizability of the findings. The review focused solely on peer-reviewed articles published from 2019 to 2024 and might have disregarded relevant literature outside this timeframe. This limitation could result in a biased representation of the effectiveness of online psychotherapy and in-person counseling (Mohseni et al., 2022). Exposure to English-only articles could have introduced possible bias in the analysis and left out useful knowledge from non-English sources. Favorable research may be overrepresented compared to neutral or negative ones. Access to certain articles may also have been restricted, leading to potential gaps in the review's coverage. Additionally, focusing only on articles published during COVID-19 would not capture effectiveness in long-term trends or changes between online therapy and traditional offline counselling in selected circumstances. Future research must elaborate the scope of the research concerning the period and the language used to draw a precise picture of the covered issue. Recognizing and acknowledging these limitations is essential to effectively interpreting the findings and the growth and improvement of the research on e-therapy's effect on mental health.
Implications
The results show that mental health treatment options are complex and require person-centred approaches. Online therapy is affordable and accessible, but it lacks the personal connection and effectiveness of face-to-face counselling, especially for some demographic groups. Mental health practitioners should recognize these and offer treatment based on the patient's preferences and needs. The study goes ahead to maintain that quality in the therapeutic alliance is determinative of the outcome of mental health. The relationship quality between a counselor and the client is that which will determine the effectiveness of the treatment and experience of the user. No matter the mode of delivery, mental health practitioners need to major in establishing strong therapeutic alliances. Furthermore, this finding supports that demographic factors, such as age and technological savviness, are determinants of preference for counseling modality. The youth may, therefore, prefer e-counselling due to prior convenience and familiarity associated with this mode from prior personal encounters. Understanding such differences is critical equitable care access (Alavi et al., 2023).
Future research
Future research should include more articles from different language and temporal contexts to overcome the constraints found. The inclusion would improve understanding of how online psychotherapy and in-person counseling work across people and contexts (Kotera et al., 2021). Future research should also examine telehealth technological trends and security, privacy, and ethics advancements. Managing these concerns allows mental health providers to provide safe and effective online therapy while retaining patient confidentiality and confidence. Longitudinal studies are needed to compare online psychotherapy to in-person counseling for enduring efficacy. Researchers can assess therapy efficacy and durability by observing results over time.
Conclusion
The current review had the key aim of answering the research question as to whether online or internet-based psychotherapy was more effective in handling mental health problems compared to in-person counseling. The reviewed materials showed that most of the subjects in the articles considered for review were likely to favor in-person counseling in terms of effectiveness. However, online counseling was favored for reasons such as convenience and affordability. The study had several limitations and therefore cannot not be generalized to all settings. One of the limitations was that the articles used in the review were published between year 2019 to the year 2024. The practical implications are that counseling experts should consider the conditions that may make a patient choose one counseling method in favor of the other. Areas for future research are provided by limitations. Future studies should focus on diverse articles from different library settings. Future studies should investigate issues such as security in online psychotherapy, and patient privacy protection. In as much as online psychotherapy comes with the benefits of convenience and cost savings, future studies should focus on ethical issues associated with online psychotherapy.
References
Alavi, N., Moghimi, E., Stephenson, C., Gutierrez, G., Jagayat, J., Kumar, A., Shao, Y., Miller, S., Yee, C. S., Stefatos, A., Gholamzadehmir, M., Abbaspour, Z., Shirazi, A., Gizzarelli, T., Khan, F., Patel, C., Patel, A., Yang, M., & Omrani, M. (2023). Comparison of online and in-person cognitive behavioral therapy in individuals diagnosed with major depressive disorder: a non-randomized controlled trial. Frontiers in Psychiatry, 14. https://doi.org/10.3389/fpsyt.2023.1113956
Békés, V., Doorn, K., Luo, X., Prout, T. A., & Hoffman, L. (2021). Psychotherapists’ challenges with online therapy during COVID-19: concerns about connectedness predict therapists’ negative view of online therapy and its perceived efficacy over time. Frontiers in Psychology, 12(12). https://doi.org/10.3389/fpsyg.2021.705699
Greenwood, H., Krzyzaniak, N., Peiris, R., Clark, J., Scott, A. M., Cardona, M., Griffith, R., & Glasziou, P. (2022). Telehealth versus face-to-face psychotherapy for less common mental health conditions: systematic review and meta-analysis of randomized controlled trials. JMIR Mental Health, 9(3). https://doi.org/10.2196/31780
Ierardi, E., Bottini, M., & Riva Crugnola, C. (2022). Effectiveness of an online versus face-to-face psychodynamic counselling intervention for university students before and during the COVID-19 period. BMC Psychology, 10(1). https://doi.org/10.1186/s40359-022-00742-7
Khan, S., Shapka, J. D., & Domene, J. F. (2021). Counsellors’ experiences of online therapy. British Journal of Guidance & Counselling, 50(1), 1–23. https://doi.org/10.1080/03069885.2021.1885009
Kotera, Y., Kaluzeviciute, G., Lloyd, C., Edwards, A.-M., & Ozaki, A. (2021). Qualitative investigation into therapists’ experiences of online therapy: implications for working clients. International Journal of Environmental Research and Public Health, 18(19), 10295. https://doi.org/10.3390/ijerph181910295
Last, B. S., Mirhashem, R., & Yang, Y. (2024). From plan to practice: A qualitative study of public mental health therapists’ session-planning practices. Psychological Services. https://doi.org/10.1037/ser0000840
Mohseni, M., Ameri, H., & Arab-Zozani, M. (2022). Potential limitations in systematic review studies assessing the effect of the main intervention for treatment/therapy of COVID-19 patients: An overview. Frontiers in Medicine, 9. https://doi.org/10.3389/fmed.2022.966632
Ncheka, J. M., Menon, J. A., Davies, E. B., Paul, R., Mwaba, S. O. C., Mudenda, J., Wharrad, H., Tak, H., & Glazebrook, C. (2024). Implementing internet-based cognitive behavioural therapy (moodgym) for African students with symptoms of low mood during the COVID-19 pandemic: a qualitative feasibilty study. BMC Psychiatry, 24(1). https://doi.org/10.1186/s12888-024-05542-4
Renn, B. N., Hoeft, T. J., Lee, H. S., Bauer, A. M., & Areán, P. A. (2019). Preference for in-person psychotherapy versus digital psychotherapy options for depression: survey of adults in the U.S. Npj Digital Medicine, 2(1). https://doi.org/10.1038/s41746-019-0077-1
Sander, J., Bolinski, F., Diekmann, S., Gaebel, W., Günther, K., Hauth, I., Heinz, A., Kleiboer, A., Riper, H., Trost, N., Vlijter, O., Zielasek, J., & Gerlinger, G. (2021). Online therapy: an added value for inpatient routine care? Perspectives from mental health care professionals. European Archives of Psychiatry and Clinical Neuroscience, 272(1), 107–118. https://doi.org/10.1007/s00406-021-01251-1
Schuster, R., Topooco, N., Keller, A., Radvogin, E., & Laireiter, A.-R. (2020). Advantages and disadvantages of online and blended therapy: Replication and extension of findings on psychotherapists’ appraisals. Internet Interventions, 21, 100326. https://doi.org/10.1016/j.invent.2020.100326
Sockalingam, S., Leung, S. E., Ma, C., Hawa, R., Wnuk, S., Dash, S., Jackson, T., & Cassin, S. E. (2022). The impact of telephone-based cognitive behavioral therapy on mental health distress and disordered eating among bariatric surgery patients during COVID-19: preliminary results from a multisite randomized controlled trial. Obesity Surgery. https://doi.org/10.1007/s11695-022-05981-6
Westerhof, G. J., Lamers, S. M. A., Postel, M. G., & Bohlmeijer, E. T. (2019). Online therapy for depressive symptoms: an evaluation of counselor-led and peer-supported life review therapy. The Gerontologist, 59(1), 135–146. https://doi.org/10.1093/geront/gnx140
Xie, M., Wang, C., Li, Z., Xu, W., Wang, Y., Wu, Y., & Hu, R. (2023). Effects of remote dignity therapy on mental health among patients with hematologic neoplasms and their significant others: A randomized controlled trial. International Journal of Nursing Studies, 104668–104668. https://doi.org/10.1016/j.ijnurstu.2023.104668
Zeren, G., Erus, S. M., Amanvermez, Y., Genç, A. B., & Duy, B. (2022). Client’s experiences of online counseling: satisfaction and therapeutic alliance. Çukurova Üniversitesi Eğitim Fakültesi Dergisi, 51(1), 634–658. https://doi.org/10.14812/cuefd.843542
