The construct validity of the Perceived Stress Scale

https://doi.org/10.1016/j.jpsychores.2016.03.009Get rights and content

Highlights

  • We studied the construct validity of the Perceived stress scale with modern test theory; Rasch model and Mokken analysis.

  • Our large population-based study indicated that the current version of the PSS has scalability problems.

  • An analysis revealed better fit for a two-dimensional than a unidimensional model.

Abstract

Objective

Stress impacts the quality of life and is associated with increased risk of mental and physical disorders. The Perceived Stress Scale (PSS) is widely used for measuring psychological distress. Although the instrument was originally defined as a single construct, several studies based on classical test theory suggest that a two-dimensional structure is more dominant. We aimed to explore the construct validity and dimensionality of the PSS-10 using modern test theory to determine if the scale is predominantly for a one- or a two-dimensional model.

Methods

The study population consisted of 32,374 citizens who completed the PSS-10 as part of the Danish National Health Survey in 2010. We investigated the construct validity of the PSS-10 by CFA. We examined the scalability by investigating the fit of the data distribution in a unidimensional Rasch model and performing modification of response categories, persons and items. The scale dimensionality was additionally assessed by Mokken and Rasch analysis.

Results

The PSS-10 did not fit the Rasch model. Item four indicated the largest misfit, and items four and seven displayed disordered thresholds. Unidimensionality could not be established although the data showed improved fit to the Rasch model for the two dimensions respectively with the positive and negative items. Mokken analysis revealed fit to the unidimensional model, but disordered thresholds were shown for item four.

Conclusion

Our large population-based study indicated scalability problems in the current version of the PSS-10. The conducted analysis overall revealed better statistical fit for a two-dimensional than a unidimensional model.

Introduction

Stress considerably impacts the quality of life worldwide [1] and is associated with a range of adverse health outcomes, including increased risk of cardiovascular events [2], [3], metabolic syndromes [4], [5], [6] and mortality [7], [8], [9]. Stress can also lead to mental illness which is a burden for the individual but may also cause serious productivity losses with societal implications [10]. Although a stress condition carries a substantial burden, it is merely considered a ‘risk factor’. For instance, no diagnosis code for stress exists in the 10th version of the International Classification of Disease (ICD-10) and in the 5th version of the Diagnostic and Statistical Manual of Mental Disorders (DSM V) [11], [12].

The Perceived Stress Scale (PSS) is a widely used instrument for measuring stress [13]. The PSS evaluates the degree to which an individual has perceived life as unpredictable, uncontrollable and overloading during the previous month. The PSS also assesses the degree to which external demands seem to exceed the individual's perceived ability to cope [13], [14].

The original 14-item scale (PSS-14) was developed in 1983 by S. Cohen et al. [13], [15], [16], but this first version was later revised and reduced into 10-item and 4-item versions [14]. The PSS-10 was originally defined as a single construct because the ‘distinction between the two different dimensions in terms of the positively and negatively scored items, was considered irrelevant’ [16]. But exploratory factor analysis (EFA) later indicated that a two-dimensional structure was more dominant in the PSS-10 [17]. A confirmatory factor analysis (CFA) by Andreou et al. confirmed that the one-dimensional model did not provide acceptable fit, while the two-dimensional model tented to show a better fit both in the PSS-10 and PSS-14 [18]. A principal component analysis (PCA) supported the existence of two dimensions: one dimension related to perceived stress (measured by six negatively worded items), while another related to coping ability and stress resilience counter-stress (measured by four positively worded items) [17]. A Turkish study by Örücü and Demir found gender differences in a translated version of the PSS-10 [19], whereas a study by Barbosa-Leiker et al. indicated that stress and counter-stress were measured equivalently in men and women by the PSS-10 [20]. Furthermore, a study by Gitchel et al. found that women reported higher levels of perceived stress overall and on the positively worded items, but not on the negatively worded items. The study suggested that gender-related item directionality on the PSS-10 might be the primary biasing factor [21].

The psychometric properties of the different versions of the PSS have been extensively studied in many countries by classical test theory (CTT). Several studies conducted in the general population in a variety of countries have found that Cronbach's α for the total scale ranges between 0.75 and 0.91 [13], [18], [22], [23]. The criterion validity was evaluated by Mitchell et al., who found that the PSS was significantly negatively correlated with the mental component of the Short-Form Health Survey (SF-36) from the Medical Outcome Study (MOS); (p < 0.05) and (r =  0.70) [24].

Overall the PSS-10 seems to have some unsolved issues as several of the assessment methods (including modern test theories) have indicated problems [17], [18].

The present study adopts a broader perspective by further investigating the construct validity and the dimensionality of the PSS-10 by CFA and further by applying modern test theory within the framework of a parametric Rasch analysis and a non-parametric Mokken scale analysis. In addition, we aim to investigate the fit of the Rasch model to PSS-10 data collected from the Danish National Health Survey (DNHS) for a unidimensional model and the two suggested dimensions of the scale.

Section snippets

Study population

The PSS-10 formed part of a battery of self-report questionnaires on physical and mental health in the DNHS in 2010. The DNHS was based on six random subsamples; one from each of the five Danish regions (mutually exclusive) and a national sample. In this study, we used the population-based sample of 52,400 persons from the Central Denmark Region [25].

All randomly selected individuals received an introductory letter, which briefly described the purpose of the voluntary survey and invited the

Study population

In the total sample of (N = 34,168); 5451 (15.9%) were aged 16–29 years, 4689 (13.7%) were aged 30–39 years, 6448 (18.9%) were aged 40–49 years, 6509 (19.1%) were aged 50–59 years, 6356 (18.6%) were aged 60–70 years and 4715 (13.8%) were aged > 70 years.

To achieve a more homogenous population the non-Danish or native language missing (N = 1794) was excluded and then the dataset comprised 32,374 persons with information on gender, age and responses to the PSS-10 (Fig. 1). As 57 values were missing for

Main findings

This large population-based study revealed that the Danish version of the PSS-10 did not fit the Rasch model, and no valid overall scale emerged after extensive modifications of the one-dimensional model. Performed modifications included deleting items with the largest misfit, excluding persons with extreme fit residuals, performing different combinations of subgroups, identifying challenging items and collapsing response categories. The problems primarily centred on the positively worded

Conclusion

The current version of the PSS-10 seems to have scalability problems. Our findings revealed that the PSS-10 did not fit the Rasch model as a unidimensional model, while the Mokken analysis identified a unidimensional model. We obtained a better, but not yet acceptable, fit to the Rasch model by splitting the items into the two dimensions suggested in previous research. Our CFA analysis confirmed the same grouping of items as evidenced in former studies. In conclusion, our results overall

Conflict of interest

The authors report to have no competing interests.

Ethical standards

The study was based on anonymised data and was approved by the Danish Data Protection Agency Journal Number (1-16-02-571-13).

Acknowledgements

This study was supported by an unrestricted grant from the Lundbeck Foundation (grant number: R155-2012-11280) and the Central Denmark Region Foundation for Primary Health Care Research (Praksisforskningsfonden).

We would like to thank the Centre for Public Health and Quality Improvement in the Central Denmark Region for allowing us to work with the data collected for the DNHS study.

References (56)

  • K.L. Robinson et al.

    Psychological distress and premature mortality in the general population: a prospective study

    Ann. Epidemiol.

    (2004 Aug)
  • M. Hamer et al.

    Psychological distress and cancer mortality

    J. Psychosom. Res.

    (2009 Mar)
  • H.A. Whiteford et al.

    Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study

    Lancet

    (2010)
  • Rosengren A, Hawken S, Ôunpuu S, Sliwa K, Zubaid M, Almahmeed WA, et al. Association of psychosocial risk factors with...
  • S.A. Stansfeld et al.

    Psychological distress as a risk factor for coronary heart disease in the Whitehall II study

    Int. J. Epidemiol.

    (2002 Feb)
  • R.J. Anderson et al.

    Anxiety and poor glycemic control: a meta-analytic review of the literature

    Int. J. Psychiatry Med.

    (2002)
  • T. Chandola et al.

    Chronic stress at work and the metabolic syndrome: prospective study

    BMJ

    (2006 Mar 4)
  • N. Horri et al.

    Stressful life events, education, and metabolic syndrome in women: are they related? A study in first-degree relatives of type 2 diabetics

    Metab. Syndr. Relat. Disord.

    (2010 Dec)
  • F.A. Huppert et al.

    Symptoms of psychological distress predict 7-year mortality

    Psychol. Med.

    (1995 Sep)
  • OECD

    Sick on the Job? Myths and Realities about Mental Health and Work, Mental Health and Work

    (2011)
  • World Health Organisation

    The ICD-10 classification of Mental and Behavioural Disorders

  • American Psychiatric Association

    Diagnostic and Statistical Manual of Mental Disorders

    (2013)
  • S. Cohen et al.

    A global measure of perceived stress

    J. Health Soc. Behav.

    (1983 Dec)
  • S. Cohen et al.

    Perceived Stress in a Probability Sample of the United States

    The Social Psychology of Health

    (1988)
  • L.R. Olsen et al.

    Prevalence of major depression and stress indicators in the Danish general population

    Acta Psychiatr. Scand.

    (2004)
  • S. Cohen et al.

    Negative life events, perceived stress, negative affect, and susceptibility to the common cold

    J. Pers. Soc. Psychol.

    (1993)
  • D.M. Golden-Kreutz et al.

    Assessing stress in cancer patients: a second-order factor analysis model for the Perceived Stress Scale

    Assessment

    (2004 Sep)
  • E. Andreou et al.

    Perceived Stress Scale: reliability and validity study in Greece

    Int. J. Environ. Res. Public Health

    (2011 Aug)
  • M.C. Örücü et al.

    Psychometric evaluation of perceived stress scale for Turkish university students

    Stress and Health

    (2009 21 July)
  • C. Barbosa-Leiker et al.

    Measurement invariance of the perceived stress scale and latent mean differences across gender and time

    Stress. Health

    (2013 Aug)
  • W.D. Gitchel et al.

    Gender effect according to item directionality on the perceived stress scale for adults with multiple sclerosis

    Rehabilitation Counseling Bulletin

    (2011 October 01)
  • M.Ç. Örücü et al.

    Psychometric evaluation of perceived stress scale for Turkish university students

    Stress. Health

    (2009)
  • M. Eklund et al.

    Psychometric properties and factor structure of the Swedish version of the Perceived Stress Scale

    Nord. J. Psychiatry

    (2014 Oct)
  • A.M. Mitchell et al.

    Perceived stress in survivors of suicide: psychometric properties of the Perceived Stress Scale

    Res. Nurs. Health

    (2008 Dec)
  • A.I. Christensen et al.

    The Danish National Health Survey 2010. Study design and respondent characteristics

    Scand. J. Public Health

    (2012 Jun)
  • L. Guttmann

    The basis for Scalogram analysis

  • D. Andrich

    An elaboration of Guttman scaling with Rasch models for measurement

  • J.F. Pallant et al.

    An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS)

    Br. J. Clin. Psychol.

    (2007 Mar)
  • Cited by (98)

    View all citing articles on Scopus
    View full text