Psychometric Validation of Sheffield Profile for Assessment and Referral to Care (SPARC) in Korean Cancer Patients
Article information
Abstract
Purpose
Identifying the palliative care needs of patients with advanced cancer is important for maintaining quality of life and timely transition to palliative care. We aimed to validate the Korean Sheffield Profile for Assessment and Referral for Care (K-SPARC) in such patients and establish its psychometric properties, including reliability, validity, and responsiveness to change.
Materials and Methods
We used the forward-back translated version of SPARC, which was verified through a pilot study, to assess the palliative care needs of patients with advanced cancer. Reliability was evaluated by internal consistency using Cronbach's alpha coefficients and test-retest reliability. Criterion validity was analyzed against other questionnaires, including the Korean versions of the Functional Assessment of Cancer Therapy-General (FACT-G Korean) and Korean versions of the Edmonton Symptom Assessment System (K-ESAS). Factor analysis was used to assess construct validity.
Results
Two hundred fifty-nine patients were included from 2019 to 2022. Forty-nine percent of all patients were women, and the median age was 63 years. Cronbach’s alpha coefficient (range, 0.642 to 0.903) and test-retest reliability (range, 0.574 to 0.749) indicated acceptable reliability. The correlation coefficients between K-SPARC and FACT-G Korean suggested significant criterion validity. The correlation coefficients for the physical, social, emotional, and functional domains were 0.701, 0.249, 0.718, and 0.511, respectively (p < 0.001, all). Factor analysis demonstrated satisfactory construct validity of the tool.
Conclusion
This study demonstrated the utility of K-SPARC as an evaluation tool for providing palliative care to patients with advanced cancer through psychometric validation; the tool had good internal consistency, reliability, and acceptable validity.
Introduction
Palliative care plays a crucial role in improving the quality of life of patients with serious illnesses and their families by providing relief from pain, symptoms, and emotional distress [1]. The American Society of Clinical Oncology recommends that patients begin palliative care as soon as they are diagnosed with cancer, regardless of cancer type or stage [2]. Early referral to palliative care is of importance, as it has been associated with better symptom management, reduced hospitalizations, and enhanced patient and caregiver satisfaction [2,3]. However, effective palliative care assessment and referral rely on the identification of patients who may benefit from said care, which remains challenging in many healthcare settings [4].
A significant medical necessity exists for a tool to enable a holistic approach, aligning with the main objectives of palliative care. Even though comprehensive tools for appraising palliative care needs have been developed, there is yet to be a recognized gold standard among the various assessment tools. Some existing assessment tools do not address the multifaceted aspects of palliative care needs, often focusing solely on one or two dimensions. In some cases, the validation and consistency of the tools have not been substantiated. Also, it is requisite to ensure that the developed tools maintain consistency in evaluating the supportive care requirements by individual patients, considering cross-cultural differences.
Accordingly, the Sheffield Profile for Assessment and Referral to Care (SPARC) has been developed [5]. Created at the University of Sheffield, SPARC was designed to facilitate the systematic identification of palliative care needs in patients with advanced illnesses. The tool encompasses a range of domains, including physical, psychological, social, and spiritual aspects of care, and has been validated in multiple populations and settings [6,7]. SPARC has been successfully implemented in various healthcare institutions, helping ensure that patients receive appropriate and timely palliative care [5].
A pilot study was conducted to adapt and translate the SPARC tool for use in the Korean population [7]. This study aimed to assess the feasibility and potential utility of the Korean version of SPARC (K-SPARC) in identifying palliative care needs among cancer patients in Korea. The findings of the pilot study demonstrated promising results, with K-SPARC showing good internal consistency and reliability. Moreover, the tool was well received by healthcare professionals and patients, highlighting its potential for widespread adoption in the Korean healthcare context [7].
The present study aimed to validate K-SPARC, building upon the findings of a previous pilot study. We conducted a comprehensive validation process to establish the psychometric properties of K-SPARC, including its reliability, validity, and responsiveness to change.
Materials and Methods
1. Study design
This study was designed to determine the validity of K-SPARC by conducting semi-structured debriefing interviews based on the results of a previous pilot study. We evaluated the test-retest reliability, validity, responsiveness, and prognostic value of K-SPARC in Korean patients with cancer.
2. Patients
The study population included adult patients aged ≥ 20 years who were diagnosed with advanced cancer and were able to understand written Korean. Patients with symptomatic brain metastases, uncontrolled mental illnesses, or cognitive problems were excluded. Upon enrollment, patient data, including age, sex, Eastern Cooperative Oncology Group performance status, medical diagnosis, cancer stage, ongoing cancer treatment, disease duration, admission status, and educational background, were obtained from the patient’s medical records. On the same day, patients completed initial surveys consisting of the Korean version of the Edmonton Symptom Assessment Scale (K-ESAS), the Korean version of the Functional Assessment of Cancer Therapy-General (FACT-G Korean), and a religious questionnaire. A retest of the K-SPARC was conducted 4-10 days later. Follow-up surveys, including K-SPARC, K-ESAS, FACT-G Korean, and the Korean version of patients’ global impression of change were administered 2-3 weeks after the initial survey.
3. Instruments used in the study
1) K-SPARC
SPARC is a self-assessment questionnaire designed to identify patients’ supportive and palliative care needs. It comprises 45 questions distributed across eight distinct subscales: communication and information issues (1 question), physical symptoms (21 questions), psychological issues (9 questions), religious and spiritual issues (2 questions), independence and activity (3 questions), family and social issues (4 questions), treatment issues (2 questions), and personal issues (3 questions). The questionnaire also included items related to symptoms, needs, and distress. In this study, we validated only 45 SPARC questions. Of the 45 questions, four were yes/no questions, while the remaining items utilized a four-point Likert scale with options such as “not at all,” “a little bit,” “quite a bit,” or “very much.” In the Polish version of SPARC, Cronbach’s alpha coefficients for physical symptoms, psychological issues, religious and spiritual issues, independence and activity, and family and social issues were 0.68, 0.86, 0.65, 0.77, 0.80, and 0.62, respectively [6]. As mentioned above, the authors previously translated K-SPARC and conducted a pilot study for it [7].
2) Other instruments
In this study, K-ESAS and FACT-G Korean were used as comparison tools to assess the feasibility of the survey. K-ESAS utilizes numerical scales from 0 to 10 to evaluate the average intensity of symptoms experienced in the past 24 hours, including pain, fatigue, nausea, depression, anxiety, drowsiness, loss of appetite, decreased well-being, dyspnea, and sleep disturbance, with 10 representing the highest intensity. K-ESAS has been validated in previous studies [8]. FACT-G, originally developed and validated by Victorson et al. [9] in the United States, comprises 27 Likert-type items divided into physical, emotional, social/family, and functional wellbeing. This scale is commonly used to assess the health-related quality of life in patients with cancer. FACT-G has also been translated into Korean and validated [10].
4. Statistical analysis
Descriptive analyses were performed for all patients. The validation process involved evaluating the internal consistency, criterion validity, and construct validity. Cronbach’s α value was evaluated with reliability analysis to verify the consistency of measurement of six domains from 41 items in the K-SPARC questionnaire. A Cronbach’s alpha coefficient closer to 1.0 indicates greater consistency of the response on the scale and is generally considered good if the coefficient is ≥ 0.60 [11,12]. Reliability was appraised through an analysis of test-retest reliability, and consistency was confirmed. Test-retest reliability was calculated using the intraclass correlation coefficient (ICC) [13]. Researchers evaluated the criterion validity of K-SPARC by performing a correlation analysis with other established tools such as the physical and social well-being domains of FACT-G Korean or K-ESAS. The FACT-G Korean questionnaire assesses items equivalent to four of the six domains of K-SPARC: physical, social, emotional, and functional. We applied a confirmatory factor analysis to examine the construct validity of the K-SPARC questionnaire, aiming to estimate how well the 41 questions represented the six domains. Construct validity was examined using the model’s goodness-of-fit, combined reliability, average variance extracted, and correlation coefficients between factors. Several model fit indices and criteria were used to examine the goodness-of-fit of the model: goodnessof-fit index, adjusted goodness-of-fit index, standardized root mean squared error, Tucker-Lewis index, comparative fit index, and root mean square error of approximation. After evaluating the model fit, we calculated the composite reliability for convergent validity and average variance extracted (AVE) for discriminant validity. The measurement difference between the initial and follow-up was calculated for K-ESAS, FACT-G Korean, and K-SPARC and used to evaluate responsiveness to change. All data were analyzed using the Statistical Package for the Social Sciences software ver. 25.0 (IBM SPSS Statistics) and AMOS (SPSS Inc.). A p-value < 0.05 was deemed statistically significant.
Results
1. Patient characteristics
A total of 298 patients who completed the baseline survey were included in the study between April 2019 and February 2022. Of the 298 patients, 39 did not respond to the retest, and 259 completed the survey. Of 259 patients included in this study, 49% were women. The median age of participants was 63 years, with the age distribution indicating that the 25th percentile of the patients’ ages was 53, whereas the 75th percentile was 70. Breast cancer is the most common primary cancer, followed by lung, gastrointestinal, and head and neck cancers. Of all patients, 81.3% had stage IV cancer, and 87.2% received chemotherapy. The primary purpose of treatment was palliative care (88.4%). The treatment location was mostly an outpatient (59.3%). Education levels were diverse, with 46.5% of participants having completed high school and 27.4% having a college or university education. Approximately half the participants were religious (50.6%). The baseline patient characteristics are presented in Table 1.
Demographic characteristics of the patients
2. Internal consistency and test-retest reliability
Cronbach’s alpha was used to assess the reliability of K-SPARC. Table 2 presents the Cronbach’s alpha values for K-SPARC and the Polish versions. K-SPARC has demonstrated high reliability in most areas, with Cronbach’s alpha values ranging from 0.642 for religious and spiritual issues to 0.903 for psychological issues. The Polish version also showed good reliability with alpha values ranging from 0.62 for treatment issues to 0.86 for psychological issues. Table 3 presents the findings of the correlation analysis for test-retest consistency evaluation using Pearson’s correlation analysis and ICC. Pearson’s correlation coefficients ranged from 0.578 for religious and spiritual issues to 0.746 for independence and activity in the total population, demonstrating consistency of the measurements over time. ICC values further supported this, ranging from 0.729 to 0.857, indicating high reliability and stability across the different K-SPARC domains.
Internal consistency of each domain in K-SPARC using Cronbach’s α coefficients
Test-retest reliability assessment of K-SPARC
3. Criterion validity
Table 4 outlines the statistically significant Spearman correlation coefficients between the corresponding domains of K-SPARC and FACT-G Korean. Specifically, it aligns the domains of physical symptoms, family and social issues, psychological symptoms, independence, and activity with the FACT-G Korean domains of physical, social/family, emotional, and functional well-being. The correlation coefficients for the physical, social, emotional, and functional domains were recorded at 0.701, 0.249, 0.718, and 0.511, respectively, each with p-values < 0.001. Table 4 presents a correlation analysis between K-SPARC and K-ESAS within their respective physical and psychological domains. Spearman’s correlation coefficients were significant for the physical (0.717 [p < 0.001]) and psychological domains (0.625 [p < 0.001]). The analysis specifically linked the K-SPARC physical symptoms to K-ESAS items, including pain, fatigue, nausea, drowsiness, loss of appetite, dyspnea, and sleep disturbance. Similarly, the psychological symptoms of K-SPARC were correlated with the depression and anxiety items of K-ESAS, highlighting the tool’s criterion validity across various patient groups.
Correlation analysis of K-SPARC with FACT-G Korean and K-ESAS
4. Construct validity
To assess construct validity, we evaluated both convergent and discriminant validity. The model fit indices are acceptable level (S1 Table). Convergent validity was evidenced by standardized loading values predominantly falling between 0.4 and 0.9, each statistically significant with p-values < 0.001. The details of the factor analysis are shown in S2 Table. Furthermore, the CR for each latent construct was > 0.7, with the AVE surpassing 0.5, for all but the physical symptoms dimension, indicating a robust level of validity (Table 5). In the discriminant validity analysis, although physical symptoms and psychological issues demonstrated correlations with domains such as religious and spiritual issues, independence and activity, family and social issues, and treatment issues, the strength of these associations did not undermine discriminant validity. This is confirmed by the fact that the correlations between these factors were lower than the square roots of the average variance extracted, ensuring discriminant validity across the latent constructs.
Confirmatory factor analysis of K-SPARC
5. Responsiveness
The differences in K-SPARC were significantly correlated with differences in FACT-G in the Korean population. The Wilcoxon signed-rank test indicated a moderate correlation in the physical domain; however, the social and functional domains showed very weak correlations (Table 6).
Responsiveness of K-SPARC, as measured by the correlation between changes in K-SPARC and FACT-G Korean
Discussion
SPARC is a comprehensive screening tool used to evaluate the physical, psychological, spiritual, and social needs of patients. To our knowledge, the present study is the main validation study of K-SPARC in a clinical context, particularly in patients with advanced cancer. Based on this pilot study, our study demonstrated satisfactory psychometric characteristics (validity and reliability) of the K-SPARC in 219 patients with advanced cancer.
The integration of early palliative care, which provides active treatment and palliative care in the early stages of cancer diagnosis, is notable for prolonging patient survival and improving the quality of life of both patients and caregivers [14,15]. Amid the evolution of therapeutic agents, the disease trajectory and prognosis of cancer are changing with the advent of new treatments. Substantial unmet needs that patients experience, including physical, psychological, emotional, and spiritual needs, have become more diverse [16]. Accordingly, it is necessary for palliative care to explore the perceived needs of patients with advanced cancer to provide holistic care. Several tools, including SPARC, have been developed to evaluate various domains of palliative care needs in patients with advanced cancer, primarily to carefully assess support needs. For instance, the Needs Assessment of Advanced Cancer Patients (NA-ACP) and Problems and Needs in Palliative Care (PNPC) are multidimensional screening tools designed to assess the specific unmet needs of advanced cancer patients. Both tools have shown satisfactory results through rigorous psychometric testing, including content validity, construct validity, and internal consistency across a range of domains [17,18]. The various tools used to identify comprehensive unmet supportive care needs in patients with advanced cancer differ in terms of the number of items, recall time, mode of administration, target population, and scoring system [19]. Previous studies have been conducted; however, a unanimous conclusion has yet to be reached regarding the standards for needs assessment tools. Physicians should adopt these tools according to the clinical situation. Considering the practicality of the K-SPARC demonstrated in our study as a well-defined screening tool for holistic care, the K-SPARC could be considered valuable for monitoring and capturing the unmet needs of patients across the cancer journey in real practice in Korea.
The results of the internal consistency and test-retest analyses of the K-SPARC were favorable. Cronbach’s alpha coefficient was > 0.7, except for religious and spiritual issues. Similarly, the test-retest coefficients were found to be approximately 0.7, barring treatment, religious, and spiritual issues. Accordingly, the reliability of K-SPARC was inferred to be acceptable and comparable to the results of previous validation studies for SPARC [20,21]. Nevertheless, similar to the pilot study, a relatively low Cronbach’s alpha was observed for religious and spiritual issues [7]. To achieve an insightful interpretation of this result, it is important to consider cultural backgrounds. First, approximately half (n=130, 50.6%) of the 257 patients whose religious data were available had religious affiliations. Supplementary analysis indicated that the importance attributed to religion in daily life was significantly higher in patients with religious beliefs than in those without (1.82 vs. 6.65, p < 0.001). However, there was no significant correlation between the domains of religious and spiritual issues in K-SPARC and the score of religious importance in the religious questionnaire, regardless of religious affiliation (S3 Table). Previous research has shown that religion affects patients’ perceptions of palliative care, and palliative care patients with religious affiliations show greater awareness of issues related to religious and spiritual needs [22,23]. The proportion of religious individuals among Koreans is relatively low compared to the global demographic [24]. This result aligns with that of our study population, in which approximately half of the patients were irreligious. Thus, the patients may have had limited awareness and motivation to address their religious and spiritual needs. Second, patients in many Asian countries, including Korea, tend to be family-centered and passive in decision-making and sometimes defer important decisions to family and experts [25,26]. There is also a cultural reluctance to openly discuss or directly address the topic of death rooted in the concept of filial piety. Finally, as discussed in the pilot study, the concept and terms of spirituality were unfamiliar to patients, which posed a challenge for them in articulating their unmet spiritual needs [7]. Nevertheless, over the decades, the decision-making styles of patients with advanced cancer in Asia have gradually shifted from passive and indirect to more active and engaged [27]. Accordingly, the evaluation of spiritual items in SPARC requires additional tracking in future studies as a vital element of palliative care for patients with advanced cancer.
We used FACT-G Korean and K-ESAS to confirm the criterion validity of K-SPARC. A significant correlation was identified between K-SPARC and both the physical and psychological items of K-ESAS. All FACT-G Korean domains displayed statistically significant positive correlations, with the physical and emotional domains exhibiting a particularly high positive correlation of > 0.7. The findings of our study indicate a positive, albeit weak, correlation between the social domains of FACT-G Korean and K-SPARC. This finding differs from the results of a pilot study in which a negative correlation was observed. Summarizing the results of the two studies, our findings clarified in a more significant number of patients that the low correlation of the social domain between FACT-G Korean and K-SPARC may be the result of cultural differences and the main focus of items in the social domain of each questionnaire. Furthermore, the study employed three tools with different timeframes for symptom-related responses: items experienced within the past 24 hours for K-ESAS, the past week for FACT-G Korean, and the preceding month for K-SPARC. Given the significant correlation between K-SPARC, K-ESAS, and FACT-G in Koreans in our study, K-SPARC is considered a valuable tool for assessing the holistic symptom burden of patients at various time points, ranging from a few days to months, and for identifying their palliative care requirements.
Six domains were extracted from the factor analysis, and the validity of the tool structure was confirmed. When analyzing the correlation coefficients between factors, two domains—physical symptoms and psychological issues—had relatively high correlation coefficients (S4 Table). Specifically, two items contributed to a high correlation coefficient: “Feeling restless and agitated?” and “Feeling that everything is an effort?” The question “Feeling restless and agitated?” belongs to the physical-symptom domain of the original SPARC structure. This item was designed to evaluate psychological aspects and showed a relatively high modification index in the psychological, religious, and spiritual domains. Considering the possibility of impeding discriminant validity, the item composition was changed to the psychological domain for analysis. Moreover, the question “Feeling that everything is an effort?” is open to ambiguous interpretation in both English and Korean. A relatively high correlation coefficient may be due to the fact that participants could interpret “effort” ambiguously in terms of both physical and psychological burdens. In a previous pilot study, no additional evaluation was conducted on the question “Feeling that everything is an effort?” owing to the absence of specific challenges or complaints of difficulties in responding to the questions. Thus, inferring the meaning of patients’ answers is currently difficult. Although it would have been possible to draw back discriminant validity, it was not significant enough to hinder the analysis. Considering the possibility of an interpretation that is different from intention, additional research may require qualitative research related to the differences in responses to questionnaires in a cultural context [28,29].
One of the most obvious limitations of this study was the small sample size. Our study excluded patients with active brain metastasis and leptomeningeal seeding, as well as those with clinical conditions that could potentially affect cognitive function. Therefore, our study does not represent all patients with advanced cancer in actual clinical settings, and our results should be cautiously interpreted in terms of generalizability. Meanwhile, relatively low Cronbach’s alpha was observed in religious and spiritual issues, presumed to be attributed to the unfamiliar and abstract concept of spirituality for Koreans. Further study will be needed to verify spiritual needs using other existing assessment tools in the context of Korea. However, to the best of our knowledge, our study is the largest validation study of SPARC for advanced cancer patients and a meaningful full psychometric validation of SPARC considering the cultural background.
In conclusion, our study demonstrated that K-SPARC has good internal consistency, reliability, and acceptable validity. Based on our results, SPARC is a useful tool for evaluating and screening the unmet needs of patients with advanced cancer by integrating early palliative care and preparing for the transition to end-of-life care. K-SPARC could be considered a proven tool in actual clinical settings for patients with advanced cancer when taking a holistic approach to decide whether specialized palliative care should be given.
Electronic Supplementary Material
Supplementary materials are available at Cancer Research and Treatment website (https://www.e-crt.org).
Notes
Ethical Statement
The study protocol was approved by the institutional review board of each participating hospital (Chungnam National University Sejong Hospital [approval number: 2020-09-036], Kangdong Sacred Heart Hospital [approval number: 2018-08-014-032], Gyeongsang National University Hospital [approval number: 2019-08-019], Kyung Hee University Medical Center [approval number: 2019-04-044], National Medical Center [approval number: H-1904-101-012], Seoul National University Bundang Hospital [approval number: B-1810-497-306], Catholic University of Korea Seoul St. Mary`S Hospital [approval number: KC19QEDI0218], Severance Hospital [approval number: 4-2019-1028], Ulsan University Hospital [approval number: 2019-07-059], Ewha Womans University Seoul Hospital [approval number: 2019-09-012], and Hanyang University Seoul Hospital [approval number: 2019-05-008]). All patients provided written informed consent with the approval of the institutional review boards.
Author Contributions
Conceived and designed the analysis: Kim HJ, Jung EH, Kwon JH, Kim YJ, Koh SJ, Lee MA, Kang JH, Rha SY, Nam EM, Baek SK, Lee HY, Song HH, Won YW, Lee H.
Collected the data: Kwon JH, Kim YJ, Koh SJ, Lee MA, Kang JH, Rha SY, Nam EM, Baek SK, Lee HY, Song HH, Won YW.
Contributed data or analysis tools: Kim HJ, Jung EH, Kwon JH, Lee H.
Performed the analysis: Kwon JH, Lee H.
Wrote the paper: Kim HJ, Jung EH, Kwon JH.
Conflict of Interest
Conflict of interest relevant to this article was not reported.
Funding
The research fund of Chungnam National University supported this work. The funders had no role in the study design, data collection, analysis, interpretation, or manuscript writing.
Acknowledgments
The authors express gratitude to the clinical investigators, patients, and their families for their participation and support. We also thank the Korean Society of Medical Oncology for awarding the Palliative Care Research Award in 2018.