Construct validity and reliability of the perceived stress scale for nursing students in South Africa

Background Increased levels of stress in nursing students are negatively related to caring behaviours and also result in poorer job proficiency and nurses who are more inclined to leave the profession. The Perceived Stress Scale (PSS), developed by Sheu and colleagues, is one of the most cited instruments for measuring stress and sources of stress amongst nursing students in international studies. However, it has not been widely validated for this purpose. Objectives This research aimed to test the construct validity and reliability of the PSS for South African nursing students. Method A cross-sectional survey was conducted at a Central South African University, and 471 of the 685 registered nursing students (68.8% response rate) participated in the study. Questionnaires were distributed and collected during classes. Confirmatory factor analysis was performed to test the hypothesised six-factor latent structure and determine the construct validity of the PSS. The internal consistency of the PSS was measured using Cronbach’s alpha. Results The model fit was a good fit and supported the six-factor latent structure as stress from (1) taking care of patients, (2) teachers and nursing staff, (3) assignments and workload, (4) peers and daily life, (5) lack of professional knowledge and skills and (6) clinical environment. Overall the PSS had a Cronbach’s alpha of 0.93. Conclusion The results confirm the construct validity and the internal consistency of the PSS for South African nursing students.

According to Lazarus and Folkman (1984): [P]sychological stress is a particular relationship between the person and the environment that is appraised by the person as taxing or exceeding his or her resources and endangering his or her wellbeing. (p. 19) Whilst stress amongst nurses has been well studied, there are few instruments to measure stress amongst nursing students specifically (Watson et al. 2013:160). Systematic reviews of studies conducted from 2000 to 2015 (Labrague et al. 2017:471-480) and 2010(McCarthy et al. 2018 on stress in nursing students revealed that the majority of the studies utilised the Perceived Stress Scale (PSS) developed by Sheu et al. (1997:341-351). The PSS was developed to determine the type of stress perceived and levels of stress experienced in clinical settings by nursing students at Taiwanese Universities. Whilst the PSS has been widely used internationally (i.e. Europe and Asia), previous studies have not fully examined the psychometric properties of the scale (Algaralleh, Altwalbeh & Alzayyat 2019:778). Furthermore, there are limited African studies that have utilised this instrument, and neither tested the factor structure of the scale (Labrague et al. 2018:402-408;Ugwoke et al. 2018:13212-13218). In particular, Langtree et al. (2018:91) reported developing a new questionnaire to identify the causes of stress in first-year South African nursing students, as there was not an appropriate research instrument for their study. With this in mind, the paper aims to test, for the first time, the hypothesised six-factor latent structure and establish the construct validity and internal consistency of the PSS for South African nursing students.

Research methods and design
Design, setting and sample A cross-sectional survey, which collects data at one point in time, was undertaken at a purposively selected South African University. Purposive sampling relies on the researcher's knowledge and judgment of the context. In this regard, the first author was familiar with the purposively selected university and was able to obtain permission for the study and access to respondents with relative ease (Babbie 2016:106, 187). At the time of the study, there were 685 registered nursing students at the university. A total of 471 questionnaires were completed with a 68.8% response rate, of which 27 had extensive missing data and were discarded, leaving a total of 444 questionnaires.

Data collection
Data collection took place early in October 2018. In collaboration with the class facilitator, fieldworkers distributed envelopes containing information leaflets, consent forms and questionnaires to the students at the start of their class. Interested students were allowed time during class to complete the questionnaire. In order to maintain confidentiality, signed consent forms and completed questionnaires were placed in a sealed envelope and returned to the class facilitator. The research team separated the consent forms from the questionnaires so that no information could be linked to a particular respondent.

Measures
The questionnaire comprised demographic and background questions as well as the PSS. The demographic and background questions focussed on issues such as gender, age, marital status, dependents, home language, place of residence, source of payment of university fees and year of study (Engelbrecht & Wilke 2021:141). The PSS (Sheu et al. 1997:341-351) includes 29 items measured on a five-point Likert Scale ranging from 0 (Never) to 4 (Very often). Factor analyses in previous samples have shown a six-factor model for the PSS, consisting of stress from (1) taking care of patients, (2) teachers and nursing staff, (3) assignments and workload, (4) peers and daily life, (5) lack of professional knowledge and skills and (6) clinical environment. Table 1 provides an indication of which items were hypothesised to measure the six factors identified in earlier research (Sheu et al. 1997:341-351;Sheu, Lin & Hwang 2002:165-175).  Higher scores indicate higher levels of stress. Usually, both total scores and individual subscale scores are measured. The cut-off points for levels of perceived stress are as follows: high 2.67-4.00, moderate 1.34-2.66 and low 0.0-1.33 (Alsaqri 2017:3;Labrague 2013:426).
A content validity index of 0.94 was also reported. In addition, 50.7% of the total variance was accounted for by the six factors, which confirmed the construct validity of the scale (Sheu et al. 2002:168).

Data analysis
Statistical Package for the Social Sciences (SPSS) (IBM SPSS version 25) and R program version 3.6.0 (R Core Team 2016) and version 0.5-23, of the lavaan package (Rosseel 2012:1-36) were used for data analysis. Categorical variables were described using frequencies and percentages, and means (M) and standard deviations (s.d.) were calculated for continuous variables. Composite scores were calculated for the subscales.
The internal consistency of the scale and subscales was tested using Cronbach's alpha.
Confirmatory factor analysis was used to test whether the data supports a hypothesised model and is suitable when the theoretical constructs are well understood. This study tested the six-factor model proposed by Sheu et al. (1997Sheu et al. ( :341-351, 2002. Confirmatory factor analysis requirements of multicollinearity, residual values and multivariate outliers and normality were examined. A weighted-least squares estimator with robust estimation of means and variances (WLSMV) was used; the comparative fit index (CFI) and the Tucker-Lewis Index (TLI) confirmed if the model fitted the data; the root mean square error of approximation (RMSEA) measured whether the model represented the data patterns. The performance of the model was tested by examining differences between the expected and actual correlation matrix (Kigozi 2020:3).

Ethical considerations
The Ethics Committee at the Faculty of Economic and Management Sciences, University of the Free State (UFS-HSD2017/1097), provided ethical clearance for the study. Voluntary, informed consent was obtained from all respondents.

Biographic characteristics
Four out of five respondents, 88.5%, were female and their average age was 28.4 years (s.d.: 9.94). Slightly less than half of the respondents were married or in a long-term relationship (46.5%), and 38.7% had children who were financially dependent on them (see Table 2 for biographic information).

Internal consistency
A Cronbach's alpha value of 0.7 suggests that a scale has adequate internal consistency (Taber 2018(Taber :1278. Overall, the PSS had a Cronbach's alpha of 0.93. As can be seen in Table 3, all subscales had a Cronbach's alpha of at least 0.7.

Construct validity
Descriptive statistics for all observed variables are provided in Table 4.
Exploratory data analysis revealed that the assumptions of multi-collinearity, residual values and multivariate outliers were met. However, there were substantial deviations from normality for some of the variables, which was not surprising given the ordinal nature of the data. Diagonally weighted least squares estimation was used to account for the violation of the normality assumption. The latent factors were standardised, allowing free estimation of all factor loadings. The model fit was good, with a TLI of 0.937, a CFI of 0.943, and an RMSEA of 0.075 90% confidence interval (CI) (0.071, 0.08). As expected, the indicators all showed  http://www.curationis.org.za Open Access significant positive factor loadings, with standardised coefficients ranging from 0.513 to 0.966 (see Table 5).
There were also significant positive correlations amongst all six latent factors, indicating that participants who showed high perceived stress in one dimension were also likely to show high perceived stress in the other dimensions as well (see Table 6).
Taken together, these results are consistent with previous studies showing a six-factor latent structure for the PSS (Sheu et al. 1997(Sheu et al. :341-351, 2002, consisting of stress from (1) taking care of patients, (2) teachers and nursing staff, (3) assignments and workload, (4) peers and daily life, (5) lack of professional knowledge and skills and (6) clinical environment, and the results confirm the construct validity of the PSS in this South African sample of 444 nursing students.

Discussion
Whilst numerous international studies have focused on stress in nursing students, there are few validated instruments to measure stress (Watson et al. 2013:160), with the PSS by Sheu et al. (1997:341-351) being the most cited scale (Labrague et al. 2017:471-480;McCarthy et al. 2018:197-209   it appears as if the PSS has not been used in South Africa to determine perceived stress amongst nursing students. This is supported by the work of Langtree et al., who in 2018 noted the lack of an appropriate instrument to measure the causes of stress amongst first-year nursing students in South Africa and therefore developed their own scale. Given the international popularity of the PSS and the dearth of research on the psychometric properties of the scale, this was the first study to measure the construct validity and internal consistency of the PSS for South African nursing students. The results confirmed the internal consistency and construct validity of the PSS for South African nursing students. More specifically, the six-factor model of the PSS identified in earlier research (Sheu et al. 1997(Sheu et al. :341-351, 2002 was confirmed. There were significant positive correlations amongst all six latent factors -(1) taking care of patients, (2) teachers and nursing staff, (3) assignments and workload, (4) peers and daily life, (5) lack of professional knowledge and skills and (6) clinical environment -indicating that participants who showed high perceived stress in one dimension were also likely to show high perceived stress in the other dimensions as well.
The overall mean score for the PSS was 1.48 (s.d. 0.61), which is indicative of moderate levels of stress. Similar findings were reported in studies conducted in Saudi Arabia (Ahmed & Mohammed 2019:120-121;Shdaifat, Jamama & Al Amer 2018:38) and Jordan (Khater et al. 2014:200-201). Higher scores, although still within the 'moderate' category, were reported in Saudia Arabia (Aedh et al. 2015:35;Alsaqri 2017:3), Turkey (Karaca et al. 2017:36), Philippines (Labrague 2013:427-428;Labrague et al. 2018:405), Greece (Labrague et al. 2018:405) and Nigeria (Labrague et al. 2018:405). Stress from assignments and workload was the main source of stress in the current study, whilst stress from lack of professional knowledge was the least reported. A possible explanation for this is that the survey was undertaken in the month prior to the commencement of examinations, when the students were finalising classes and assignments and not engaging in clinical work.
Increased levels of stress result in burnout which is correlated with poorer psychological and physical health and may influence the work of individuals in helping professions and eventually also the well-being of their patients (Enns et al. 2018:230). There is evidence that increased levels of stress and burnout in nursing students are negatively related to caring behaviours (Li et al. 2020: 5-6); they also result in poorer job proficiency and nurses who are more inclined to leave the profession (Rudman & Gustavsson 2012:998;Rudman, Gustavsson & Hultell 2014:620). Given that nursing is already a highly stressful occupation (Dlamini & Visser 2017:1068Khamisa et al. 2015:660; Van der Colff & Rothmann 2014:381) and the importance of retaining nurses in the profession (Van Rensburg 2014), stress should be addressed at the level of the nursing student. Therefore, the importance of valid and reliable scales to measure stress levels and sources of stress amongst nursing students should be acknowledged, so that meaningful and useful inferences can be made when recommending strategies to address stress in nursing students. The current study established that the PSS is a useful tool for nursing educators and researchers to help identify the main sources of stress amongst nursing students. As the nursing curriculum is both academically and clinically demanding, it is important for institutions providing nursing education to introduce preventative measures, such as teaching coping strategies and providing mentorship programmes, so that students are taught how to deal with stress at the outset of their education.
As with all research, this study has limitations. The study was undertaken at a single university amongst a convenience sample of nursing students, although slightly more than a third of the group was reached. As such, caution is required when generalising the findings to other university settings. A self-administered questionnaire was used, which implies that there may be some level of response bias. It is recommended that future research should replicate the present study and that further confirmatory factor analysis

Conclusion
This study confirms the six-factor structure and internal consistency of Sheu et al.'s (1997:341-351) PSS, suggesting that it is a valid and reliable instrument to test the perceived stress from (1) taking care of patients, (2) teachers and nursing staff, (3) assignments and workload, (4) peers and daily life, (5) lack of professional knowledge and skills and (6) clinical environment for South African nursing students.