A clinimetric analysis of the Hopkins Symptom Checklist (SCL-90-R) in general population studies (Denmark, Norway, and Italy)

Original Citation:

A clinimetric analysis of the Hopkins Symptom Checklist (SCL-90-R) in general population studies (Denmark, Norway, and Italy)

Published version:

DOI:10.3109/08039488.2016.1155235 Terms of use:

Open Access

(Article begins on next page)

Anyone can freely access the full text of works made available as "Open Access". Works made available under a Creative Commons license can be used according to the terms and conditions of said license. Use of all other works requires consent of the right holder (author or publisher) if not exempted from copyright protection by the applicable law. Availability:

This is a pre print version of the following article:

For Peer Review Only

A clinimetric analysis of the Hopkins Symptom Checklist (SCL-90-R) in general population studies (Denmark, Norway, and Italy)

Danilo Carrozzino 1), 3) Olav Vassend 2),4) Flemming Bjørndal 3) Claudia Pignolo 5) Per Bech 3) Abstract: 207 words Main text: 2877 words Tables: 4

1

) Department of Psychological, Health, and Territorial Sciences, University “G. d’Annunzio” of Chieti-Pescara, Italy

2)

Department of Psychology, Psychological Institute. University of Oslo, 0317 Oslo, Norway

3)

Psychiatric Research Unit, Psychiatric Centre North Zealand, Copenhagen University Hospital, Hillerød, Denmark

4)

Child and Adolescent Mental Health Services Capital Region of Denmark, Copenhagen University Hospital, Glostrup, Denmark

5)

Department of Psychology, University of Turin, Torin, Italy

Corresponding author:

Per Bech

Psychiatric Research Unit

Psychiatric Centre North Zealand, Copenhagen University Hospital Dyrehavevej 48 DK-3400 Hillerød, Denmark Per.bech@regionh.dk Tel.: +45 38 64 30 95 Fax: +45 38 64 30 99 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

Background:

Although the Symptom Checklist (SCL-90-R) is one of the most widely used self-reported scales covering several psychopathological states, the scalability of the SCL-90-R has been found very problematic. Aims:

We have performed a clinimetric analysis of the SCL-90-R, taking both its factor structure and scalability (total scale score a sufficient statistic) into account.

Methods:

The applicability of the SCL-90-R has been found acceptable in general population studies from Denmark, Norway and Italy. These studies were examined with Principal Component Analysis (PCA) to identify the factor structure. The scalability of the traditional SCL-90-R subscales (somatization, hostility, and

interpersonal sensitivity) as well as the affective subscales (depression and anxiety and ADHD) were tested by Mokken’s item response theory model.

Results:

Across the three general population studies the traditional scaled SCL-90-R factor including 83 items was identified by PCA. The Mokken analysis accepted the scalability of both the general factor and the clinical SCL-90-R subscales under examination.

Conclusion:

The traditional, scaled, general 83 item SCL-90-R scale is a valid measure of general psychopathology. The SCL-90-R subscales of somatization, hostility, and interpersonal sensitivity as well as the affective subscales of depression, anxiety, and ADHD) were all accepted by the Mokken test for scalability, i.e. their total scores are sufficient statistics.

Keywords

Symptom Checklist SCL-90-R; Principal Component Analysis, Mokken analysis, scalability 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

The Hopkins Symptom Checklist (HSCL) was originally developed by Parloff et al (1) as a measure of improvement in the treatment of patients with anxiety or depression. The first version included 41 symptoms (SCL-41) pertaining to items of both emotional and bodily distress, (2). Parloff et al (1) declared that use of the total SCL-41 score as criterion of improvement was only acceptable if there were a net decline of all symptoms in the scale. Within modern psychometrics this requirement is referred to as scalability, i.e. the total score is a sufficient statistic (3).

Over the years the SCL-41 has been enlarged by the inclusion of many anxiety or depression symptoms, and in the latest revision (SCL-90-R), Derogatis (4) recommended using both the total SCL-90-R score and, if needed, the profile of the nine factorial subscales identified by Derogatis and Cleary (5).

In spite of its many items the SCL-90-R is one of the most widely used measures of distress. Compared to other questionnaires such as the Beck Depression Inventory (6) the SCL-90-R contains items that are less invasive, implying that general population studies are easy to perform (7). On the other hand, in such general population studies it has proved very difficult to identify the nine SCL-90-R subscales. Thus, using confirmative factor analysis, Vassend and Skrondal (8) were unable to identify the nine SCL-90-R

conventional factors while Olsen et al (9), using the Rasch (10) item response model, were unable to support the scalability of the total SCL-90-R (i.e. the total scale score was not sufficient). Vassend and Skrondal (8) actually concluded that statistical models are not sufficient for the interpretation of the construct validity of the SCL-90-R and called for clinical interpretations.

The field of clinimetrics has recently seen developments (11) in which clinical validity has a higher priority than statistical methods when testing the construct validity of a questionnaire. Among the few factors in the SCL-90-R found specific by Vassend and Skrondal (8) were, inter alia, somatization (bodily distress) and interpersonal sensitivity or hostility (emotional distress). Vassend and Skrondal (8) found that among the 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

depressive symptoms in the SCL-90-R some referred to emotional distress whereas others referred to depressive illness. In the Appendix we indicate clinically relevant SCL-90-R subscales with reference to mental disorders such as major depression, anxiety disorders, and ADHD (attention deficit disorders). From a clinimetric point of view the obsessive-compulsive factor in the conventional SCL-90-R includes too many unspecific obsessive-compulsive items such as “trouble remembering things”, “trouble concentrating” or “feeling blocked in getting things done” which might be considered as core elements in ADHD.

On this background we have performed a clinimetric analysis of three general population studies: the Norwegian study by Vassend and Skrondal (8), the Danish study by Olsen et al (9), and a new Italian study (12). Clinimetrically, we have analyzed the three most specific distress factors identified by Derogatis and Cleary (5), namely somatization, interpersonal sensitivity, and hostility, as well as the clinically selected depression, anxiety, and ADHD subscales (Appendix), including the short depression scale based on the core symptoms in the Hamilton Depression Scale “HAM-D6” (3) (Appendix). Furthermore we have attempted an analysis of the factor structure of the SCL-90-R focusing on a general distress factor which was the basis of the scale (1), using principal component analysis without and with varimax rotation, analogue to Derogatis and Cleary (5). The scalability of this general factor as well as the seven subscales (Appendix) was tested by use of the non-parametric item response theory established by Mokken (13).

Materials and Methods Sample

We focused the study on three different general populations, as follows:

1. The Danish sample included participants recruited randomly from The Civil Registration System in Denmark. The research protocol, including the SCL-90-R, was mailed to 17 males and 17 females born in each year in the period 1920-1979 (i.e., age ranging from 20 to 79 years). Out of 2040 Danish citizens recruited, only 1153 respondents returned the questionnaire fully completed. The response rate was 58%. Out of 1153 respondents, 16 cases were excluded and not used for analyses. The final sample, consisting of 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

1137 participants, had an almost equal representation of men and women. Data were collected in April 2000. The data set is part of the general community sample study performed by Olsen, Mortensen, and Bech (9).

2. The Norwegian sample was collected using a two-stage stratified cluster sampling design. The strata were type of municipality and geographic area, in order to ensure representativeness in terms of these variables. The Norwegian sample is part of the general population study performed by Vassend and Skrondal (14). In brief, in the Norwegian population study approximately 1200 individuals were contacted and asked to participate in the investigation. The response rate was 90%. The final sample, consisting of 1082 respondents, is considered representative of the non-institutionalised Norwegian population aged 15 years and above.

3. The Italian sample comprised participants drawn randomly from all over of Italy (i.e., North-West of Italy, North-East of Italy, Center of Italy, South of Italy, and Italian Islands). The study was carried out between August 2013 and August 2014. A randomized subsample of 1000 subjects was selected to meet the

stratification criteria. However, 10 participants were excluded because they had not completed the full SCL-90-R. The remaining 990 individuals (i.e., 483 males and 507 females) were used for analyses. The final subsample of 990 participants is considered representative of the non-institutionalised Italian population aged 18 years and above.

Ethics 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

The Danish data were collected by a mailed questionnaire booklet. Participation was totally anonymous, as there was no name or other identification code. This method was accepted by the Danish local Ethical Committee for ensuring anonymity.

All Norwegian data were collected by trained interviewers after written informed consent had been obtained. All respondents completed the self-rating scales anonymously. The study was approved by the Oslo Ethics Committee and the Norwegian Data Inspectorate.

The Italian participants were enrolled following obtainment of written informed consent. The questionnaire was completed anonymously. The study was accepted by the Institutional Review Board of the Department of Psychological, Health, and Territorial Sciences, University “G. d’Annunzio” of Chieti-Pescara, Italy.

Measures

The Symptom Checklist-90-Revised (SCL-90-R) is a patient-reported questionnaire for measurement of psychological distress or the degree of the affective distress. All items are as symptoms formulated negatively. Each item is scored on a 5-point Likert scale ranging from 0 (= not at all), 1 (= a little bit), 2 (= moderately), 3 (= quite a bit), and 4 (= extremely). The Appendix shows the SCL-90-R items in the conventional factors: somatization (k = 12), hostility (k = 6), and interpersonal sensitivity (k = 9).

There are four clinical subscales covering depression, anxiety and ADHD; SCL-90-R items corresponding to the Major Depression Inventory (MDI) and to the Hamilton Depression Scale (HAM-D6) (3); SCL-90-R items corresponding to the co-anxiety or major anxiety scale (ASS8) (15); and SCL-90-R items identified by author FB as measuring ADHD. According to Derogatis and Cleary (5) the scaled SCL-90-R version covering the 9 conventional subscales includes in all 83 items, referred to as the scaled SCL-90-R general factor scale.

The NEO-PI comprises 180 items of which 144 items cover Neuroticism (N), Extraversion (E), and Openness (O). The Neuroticism scale consists of 48 items grouped into facets: anxiety, hostility, depression, self-consciousness, impulsiveness, and vulnerability. The Extraversion scale and Openness scale also consist of 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

48 items, while the Agreeableness and the Conscientiousness scale consist of 18 items each (14,16). The item format of NEO-PI consists of either positively (positive mental health), or negatively (negative mental health) formulated items with five-point Likert answer categories. Within the NEO-PI neuroticism scale we have identified two subscales, each consisting of 14 items which measure the two opposite poles of euthymia and dysthymia. These subscales were used for a Rasch correlation analysis in the Norwegian data set (17).

Psychometric analyses

Principal Component Analysis (PCA)

Without rotations we focused on the first few principal components that represent most of the information captured by the manifest items in the SCL-90-R, which is the background for the Principal Component Analysis (PCA) (18,19). When using unrotated PCA analysis, we focused on the first principal components with an eigenvalue above 1. The correlations of the items with the principal components in the PCA are referred to as loadings, a term borrowed from factor analysis (19). We focused our PCA analysis on the two-factor model, in which the first principal component is a general two-factor, whereas the second principal component is a dual factor with negative versus positive loadings (3). We also performed varimax rotation (19,20). The SAS statistical package (version 9.0.0, 2002) was used.

Mokken Analysis

The non-parametric item response analysis of the data structure was performed by using the Mokken analysis (13). The Mokken analysis was used to evaluate the measurement aspects of the SCL-90-R for scalability; that is, to what extent the total score covers the components or items of the scale sufficiently (3) .The Mokken model (13) is based on the Guttman cumulative rating scale principle (3,21): that scorings on lower prevalence manifest items must be preceded by scorings on high prevalence items. This item response theory model is based on the principle of measurement in which a score on lower prevalence 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

items presupposes a score on higher prevalence items. The scalability is evaluated by use of Loevinger’s coefficient of homogeneity (3) but the rank order of prevalence is evaluated by the mean scores of the manifest items. According to Mokken (13) or van Schuur (22) coefficients of homogeneity from 0.20 to 0.29 belong in a questionable zone as to constituting a cumulative scale, from 0.30 to 0.39 they are just acceptable, while a coefficient of homogeneity of 0.40 or higher clearly indicates scalability. The Mokken analysis was performed using the MSP program (23).

Pearson Correlation analysis

The parametric Pearson correlation analysis was performed correlating the NEO-PI subscales of euthymia and dysthymia with the different SCL-90-R versions. Pearson coefficients were preferred to Spearman coefficients because 95% confidence intervals are an acceptable interpretation when comparing statistical significance between the coefficients. The correlation analyses were conducted by means of the SAS statistical package (version 9.0.0, 2002).

Results

The percentage of females in the Danish study was 53.3 %, in the Norwegian study 51.0 % and in the Italian study 51.2 % (P=0.48). The mean age (sd) in the Danish study was 48.8 years (16.7), in the Norwegian study 39.3 years (14.5), and in the Italian study 48.4 years (17.3) (P<0.01).

Table 1 shows the results from the unrotated principal components analysis based on the 83 items of the SCL-90-R selected by Derogatis and Cleary (5) when identifying the 9 conventional SCL-90-R factors. The ratio between the first and the second principal components was 8 in the Danish and the Norwegian study, and 7 in the Italian study. No difference was seen when comparing males with females (Table 1), indicating that the first principal component is indeed a general factor of distress as also identified by Derogatis and Cleary (5). 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

In all three studies the second and third principal components were bidirectional, i.e. with both positive and negative loadings. However, the pattern of this structure of items was not consistent across the three studies. When only considering items numerically higher (positive loadings) or lower (negative loadings) than 0.25 the second principal component in the Danish study had three hostility items with negative loadings and eleven anxiety items with positive loadings. The third principal component in the Danish study had nine somatization items with negative loadings and four interpersonal sensitivity items with positive loadings. This indicated that the specific distress factors of somatization, interpersonal sensitivity, and hostility were in operation whereas the clinical depression items were not. However, the pattern was not consistent across the three studies. The varimax rotation identified 39 items in each of the three studies with positive loadings of 0.28 or higher, but the pattern of these items was not consistent across the studies.

Table 2 shows the scalability in terms of the Mokken coefficients of homogeneity. The level of these coefficients between 0.30 and 0.39 indicates an only just acceptable scalability. The total score of all the 83 items of the general factor had a coefficient of homogeneity at an only just acceptable level with no difference between males and females. Among the subscales the somatization factor was at a border level of 0.40 concerning the coefficient of homogeneity. As for the other subscales, the Mokken analysis accepted the scalability both for males and females (Table 2).

Table 3 shows the mean (sd) scores at the item level. In all the scales females scored significantly higher than males.

Table 4 shows the Pearson coefficients when correlating the NEO-PI subscales of euthymia and dysthymia, the general 83 item scaled version, and the seven subscales. In this analysis the somatization and hostility subscales had significantly lower coefficients when compared to the other subscales when using the general factor of 83 items as index.

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

The 90 items contained in the SCL-90-R can be considered to constitute a multidimensional construct of distress, covering both bodily and emotional components. In other words, the individual items can be viewed as a data bank of items. The SCL-90-R has over the years become one of the most widely used general measures of self-reported psychopathology and has consequently been recommended for inclusion in the DSM-5 (24).

From a modern clinimetric point of view (3) it seems curious that Derogatis and Cleary (5) excluded seven of the 90 items of the SCL-90-R to ensure univocal factor loadings on the nine primary factors identified by principal component analysis. The remaining 83 items have therefore until now been referenced as the scaled SCL-90-R version (4). However, the “additional” seven items have clinical significance as most of them are included in the DSM-5 major depression symptom universe.

In accordance with the factor analytic studies on the SCL-90-R in general population studies, e.g. (8) or in clinical studies, e.g. Cyr et al (5,25) or Derogatis and Cleary (5) we have demonstrated across the Danish, Norwegian and Italian populations that the 83 item scaled version of the SCL-90-R is a general factor of distress both in males and females.

The identification by principal component analysis of a general factor is not an argument for scalability (3), nor an argument from a measurement point of view that the summed total score is a sufficient measure of distress severity, which actually was the objective of the first SCL-41 version constructed by Parloff et al (1). From a clinimetric point of view we used the Mokken analysis to test the scalability of both the global 83 item version of SCL-90-R, the conventional subscales of somatization, interpersonal sensitivity, and hostility, the clinically related subscales of depression, anxiety, and ADHD. Whereas the general 83 item version obtained coefficients of homogeneity between 0.30 and 0.39, indicating a just acceptable

scalability, the specific subscales mostly obtained coefficients of homogeneity of 0.40 or higher, indicating a very acceptable scalability in both males and females. Our finding that the females scored significantly 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

higher than the males is therefore not a psychometric gender problem, but evidence of the fact that females experience more distress than males on both the general factor of distress and on the specific subscales. This is in accordance with findings when measuring neuroticism on the Eysenck Neuroticism Scale (26). This is a further argument that distress or dysthymia is a measure of neuroticism (Fava & Bech, 2015).

Urbán et al (27), using the bifactor structural model analogue to Vassend and Skrondal (8), have recently demonstrated in a non-clinical community sample that somatization was the most specific factor, i.e. did not correlate with the general SCL-90-R factor. Within our measurement-oriented analysis we have shown that somatization (bodily distress) and hostility (externalizing distress) were identified as specific subscales when using the NEO-PI neuroticism subscales as index of validity (17). The Eysenck neuroticism scale, when developed as a dysthymia scale (28) excluded somatization from the universe of symptoms while hostility was considered an externalizing factor to be found within the dimension of extraversion.

Paap et al (29) have attempted an approach using the Mokken analysis as an exploratory factor analysis when validating the SCL-90-R. This approach was originally used by Maier and Philip (30) when validating the Hamilton Depression Scale in order to reduce the number of items. Paap et al (29) reduced the SCL-90-R to 60 items but had difficulties when comparing males with females. Within the field of clinimetrics, the Mokken analysis should be used as a test for scalability, not as an explorative way to identify factors.

Conclusion

In conclusion, we have supported the original bifactor model put forward by Vassend and Skrondal (8) when comparing their Norwegian SCL-90-R general population study with a Danish and an Italian SCL-90-R general population study. Thus, the total score of the 83 item SCL-90-R general factor is from a

measurement point of view a sufficient statistic to indicate distress severity which was the original

objective for the SCL questionnaire. Our clinimetric approach compared the clinical subscales of depression, 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

anxiety, and ADHD with the conventional SCL-90-R subscales of somatization, hostility, and interpersonal sensitivity. Among these subscales somatization and hostility were the most specific when indicating an SCL-90-R profile. 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

(1) Parloff MB, Kelman HC, Frank JD. Comfort, effectiveness, and self-awareness as criteria of improvement in psychotherapy. Am J Psychiatry 1954;111:343-51.

(2) Bech P. Rating scales for psychopathology, health status and quality of life. A compendium on documentation in accordance with the DSM-III-R and WHO systems. Berlin: Springer; 1993. (3) Bech P. Clinical psychometrics. Oxford: Wiley Blackwell; 2012.

(4) Derogatis LR. Symptom Checklist-90-Revised (SCL-90-R). In: Rush AJ, First MB, Blacker D, editors. Handbook of psychiatric measures. 2nd ed. Washington D.C.: APA; 2008. p. 73-76.

(5) Derogatis LR, Cleary PA. Factorial invariance across gender for the primary symptom dimensions of the SCL-90. Br J Soc Clin Psychol 1977;16:347-56.

(6) Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry 1961;4:561-71.

(7) Naughton MJ, Wiklund I. A critical review of dimension-specific measures of health-related quality of life in cross-cultural research. Qual Life Res 1993;2:397-432.

(8) Vassend O, Skrondal A. The problem of structural indeterminacy in multidimensional symptom report instruments. The case of SCL-90-R. Behav Res Ther 1999;37:685-701.

(9) Olsen LR, Mortensen EL, Bech P. The SCL-90 and SCL-90R versions validated by item response models in a Danish community sample. Acta Psychiatr Scand 2004;110:225-9.

(10) Rasch G. Probalistic models for some intelligence and attainment tests. Expanded edition. Chicago: Chicago University Press; 1980.

(11) Tomba E, Bech P. Clinimetrics and clinical psychometrics: Macro- and microanalysis. Psychother Psychosom 2012;81:333-43.

(12) Pignolo C, Zennaro A, Di Nuovo S, Fulcheri M, Lis A, Mazzeschi C. Validazione italiana. In: Zennaro A, Di Nuovo S, Fulcheri M, Lis A, Mazzeschi C, editors. PAI - Personality Assessment Inventory. Adattamento italiano. Florence, Italy: Hogrefe; 2015. p. 303-335.

(13) Mokken RJ. Theory and practice of scale analysis. Berlin: Mouton; 1971.

(14) Vassend O, Skrondal A. Validation of the NEO Personality Inventory and the five-factor model. Can findings from exploratory and confirmatory factor analysis be reconciled? European Journal of Personality 1997;11:147-66.

(15) Bech P, Bille J, Moller SB, Hellstrom LC, Ostergaard SD. Psychometric validation of the Hopkins Symptom Checklist (SCL-90) subscales for depression, anxiety, and interpersonal sensitivity. J Affect Disord 2014;160 May:98-103. 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

(16) Vassend O, Skrondal A. Factor analytic studies of the neo personality inventory and the five-factor model: The problem of high structural complexity and conceptual indeterminacy. Personality and Individual Differences 1995;19:135-47.

(17) Bech P, Carrozzino D, Austin SF, Moller SB, Vassend O. Measuring euthymia within the Neuroticism scale from the NEO Personality Inventory. A Mokken analysis of the Norwegian general population study for scalability. Submitted 2015.

(18) Hotelling H. Analysis of a Complex of Statistical Variables with Principal Components. Journal of Educational Psychology 1933;24:417-41.

(19) Dunteman GH. Principal components analysis. Newbury Park: SAGE Publications; 1989.

(20) Kaiser HW. The varimax criterion for analytic rotation in factor analysis. Psychometrika 1958;23:187-200.

(21) Michell J. An introduction to the logic of psychological measurement. New York: Psychology Press; 1990.

(22) van Schuur RH. Ordinal item response theory. Mokken scale analysis. London: SAGE Publications; 2011. (23) Molenaar IW, Debels P, Sijtsna K. User's manual MSP, a program for Mokken scale analyses for

polytomous items (version 3.0). Groeningen, The Netherlands: ProGAMMA; 1994.

(24) Paris J. The intelligent clinician's guide to the DSM-5. Oxford: Oxford University Press; 2013. (25) Cyr JJ, McKenna-Foley JM, Peacock E. Factor structure of the SCL-90-R: is there one? J Pers Assess 1985;49:571-8.

(26) Eysenck HJ, Eysenck SBG. Psychoticism as a dimension of personality. London: Hodder and Stoughton; 1976.

(27) Urban R, Kun B, Farkas J, Paksi B, Kokonyei G, Unoka Z, et al. Bifactor structural model of symptom checklists: SCL-90-R and Brief Symptom Inventory (BSI) in a non-clinical community sample. Psychiatry Res 2014;216:146-54.

(28) Eysenck HJ, Eysenck MW. Personality and individual differences. New York: Plenum Press; 1985. (29) Paap MC, Meijer RR, Cohen-Kettenis PT, Richter-Appelt H, de Cuypere G, Kreukels BP, et al. Why the factorial structure of the SCL-90-R is unstable: comparing patient groups with different levels of

psychological distress using Mokken Scale Analysis. Psychiatry Res 2012;200:819-26.

(30) Maier W, Philipp M. Comparative analysis of observer depression scales. Acta Psychiatr Scand 1985;72:239-45. 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

None of the authors has disclosures of interest to declare.

Legends Table 1

Principal Components Analysis of the scaled 83 item SCL-90-R Table 2

Mokken analysis based on the coefficient of homogeneity Table 3

General population mean (sd) scores at the item level Table 4

Pearson correlation analysis of theNorwegian study: The euthymia and dysthymia subscales in NEO-PI versus the SCL-90-R subscales

. 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

Tables

Table 1

Principal Components Analysis of the scaled 83 item SCL-90-R

Principal components

All observations Eigen values

Denmark Norway Italy

First 27.4 27.2 24.3 Second 3.5 3.2 3.5 Third 2.9 2.6 2.3 Fourth 2.2 2.3 2.2 Fifth 2.1 1.8 1.9 Sixth 1.6 1.6 1.8 Principal components Males Eigen values

Denmark Norway Italy

First 25.4 25.5 23.6 Second 3.6 3.3 3.7 Third 3.0 3.0 2.8 Fourth 2.7 2.6 2.4 Fifth 2.2 2.1 2.1 Sixth 2.0 1.9 2.0 Principal components Females Eigen values

Denmark Norway Italy

First 29.0 28.7 24.0 Second 3.7 3.5 3.6 Third 3.0 2.6 2.7 Fourth 2.4 2.3 2.2 Fifth 2.1 1.9 2.0 Sixth 1.7 1.8 1.9 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

Table 2 Mokken analysis based on the coefficient of homogeneity

All observations Coefficient of homogeneity Denmark Norway Italy The scaled

total score (83 items) 0.39 0.37 0.33

Traditional distress factors

• Somatization (k=12) 0.40 0.39 0.38 • Hostility (k=6) 0.45 0.49 0.46 • Interpersonal sensitivity (k=9) 0.50 0.48 0.43 Clinical syndrome • Depression, MDI (k=10) 0.51 0.42 0.42 • Depression, HAM-D6 (k=6) 0.60 0.52 0.47 • Anxiety (ASS8) (k=8) 0.48 0.45 0.39 • ADHD (k=6) 0.45 0.42 0.44 Males Coefficient of homogeneity Denmark Norway Italy The scaled

total score (83 items)

0.36 0.35 0.32

Traditional distress factors

• Somatization (k=12) 0.35 0.37 0.37 • Hostility (k=6) 0.41 0.52 0.46 • Interpersonal sensitivity (k=9) 0.49 0.44 0.44 Clinical syndrome • Depression, MDI (k=10) 0.51 0.40 0.42 • Depression, HAM-D6 (k=6) 0.58 0.49 0.44 • Anxiety (ASS8) (k=8) 0.49 0.40 0.37 • ADHD (k=6) 0.41 0.39 0.48 Females Coefficient of homogeneity Denmark Norway Italy The scaled

Total score (83 items)

0.41 0.39 0.32

Traditional distress factors

• Somatization (k=12) 0.43 0.40 0.36 • Hostility (k=6) 0.50 0.47 0.47 • Interpersonal sensitivity (k=9) 0.51 0.52 0.42 Clinical syndrome • Depression, MDI (k=10) 0.50 0.44 0.41 • Depression, HAM-D6 (k=6) 0.61 0.53 0.47 • Anxiety (ASS8) (k=8) 0.48 0.49 0.39 • ADHD (k=6) 0.49 0.45 0.41 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

Table 3 General population mean (sd) scores at the item level

All observations Mean (sd) Denmark (N=1137) Norway (N= 1081 Italy (N=990) The scaled total score (83 items) 0.44 (0.43) 0.36 (0.40) 0.49 (0.44)

Traditional distress factors

• Somatization (k=12) 0.49 (0.53) 0.41 (0.48) 0.57 (0.55) • Hostility (k=6) 0.34 (0.41) 0.32 (0.45) 0.41 (0.52) • Interpersonal sensitivity (k=9) 0.54 (0.56) 0.44 (0.55) 0.51 (0.56) Clinical syndrome • Depression, MDI (k=10) 0.68 (0.66) 0.50 (0.54) 0.71 (0.60) • Depression, HAM-D6 (k=6) 0.76 (0.77) 0.52 (0.62) 0.79 (0.71) • Anxiety (ASS8) (k=8) 0.41 (0.47) 0.38 (0.49) 0.54 (0.53) • ADHD (k=6) 0.73 (0.65) 0.59 (0.58) 0.65 (0.61) Males Mean (sd) Denmark (N= 531) Norway (N= 530) Italy (N= 483) The scaled total score (83 items) 0.40 (0.40) 0.31 (0.36) 0.40 (0.39) Traditional distress factors

• Somatization (k=12) 0.42 (0.46) 0.35 (0.43) 0.44 (0.48) • Hostility (k=6) 0.31 (0.39) 0.30 (0.45) 0.39 (0.51) • Interpersonal sensitivity (k=9) 0.48 (0.52) 0.39 (0.49) 0.42 (0.50) Clinical syndrome • Depression, MDI (k=10) 0.60 (0.61) 0.43 (0.48) 0.59 (0.55) • Depression, HAM-D6 (k=6) 0.66 (0.72) 0.43 (0.54) 0.65 (0.63) • Anxiety (ASS8) (k=8) 0.37 (0.45) 0.33 (0.43) 0.45 (0.48) • ADHD (k=6) 0.69 (0.61) 0.54 (0.55) 0.57 (0.61) Females Mean (sd) Denmark (N=606) Norway (N= 551) Italy (N=507) The scaled total score (83 items) 0.48 (0.46) 0.40 (0.44) 0.57 (0.46) Traditional distress factors

• Somatization (k=12) 0.56 (0.57) 0.46 (0.51) 0.70 (0.58) • Hostility (k=6) 0.36 (0.42) 0.34 (0.45) 0.43 (0.53) • Interpersonal sensitivity (k=9) 0.60 (0.59) 0.50 (0.60) 0.59 (0.59) Clinical syndrome • Depression, MDI (k=10) 0.75 (0.69) 0.57 (0.58) 0.83 (0.62) • Depression, HAM-D6 (k=6) 0.84 (0.79) 0.61 (0.69) 0.93 (0.75) • Anxiety (ASS8) (k=8) 0.46 (0.49) 0.43 (0.53) 0.63 (0.56) • ADHD (k=6) 0.77 (0.68) 0.63 (0.61) 0.72 (0.61) 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

Table 4 Pearson correlation analysis of the Norwegian study: The euthymia and dysthymia subscales in NEO-PI versus the SCL-90-R subscales

All observations

Pearsons coefficient (95 % confidence interval) Euthymia subscale Dysthymia subscale The scaled total score (k = 83) -0.43 (0.38 – 0.47) 0.60 (0.56 – 0.63) Traditional distress factors

• Somatization (k=12) -0.28 (0.23 – 0.34) 0.41 (0.36 – 0.46) • Hostility (k=6) -0.28 (0.23 – 0.34) 0.36 (0.31 – 0.41) • Interpersonal sensitivity (k=9) -0.47 (0.42 – 0.52) 0.61 (0.57 – 0.64) Clinical syndrome • Depression, MDI (k=10) -0.41 (0.35 – 0.45) 0.57 (0.53 – 0.61) • Depression, HAM-D6 (k=6) -0.39 (0.34 – 0.44) 0.54 (0.50 – 0.58) • Anxiety (ASS8) (k=8) -0.42 (0.37 – 0.47) 0.58 (0.54 – 0.62) • ADHD (k=6) -0.35 (0.29 – 0.40) 0.49 (0.45 – 0.54) Males

Pearsons coefficient (95 % confidence interval) Euthymia subscale Dysthymia subscale The scaled total score (k = 83) -0.40 (0.32 – 0.47) 0.59 (0.53 – 0.64) Traditional distress factors

• Somatization (k=12) -0.25 (0.17 – 0.33) 0.41 (0.34 – 0.48) • Hostility (k=6) -0.23 (0.15 – 0.31) 0.33 (0.26 – 0.41) • Interpersonal sensitivity (k=9) -0.44 (0.36 – 0.50) 0.58 (0.52 – 0.63) Clinical syndrome • Depression, MDI (k=10) -0.36 (0.28 – 0.43) 0.57 (0.50 – 0.62) • Depression, HAM-D6 (k=6) -0.37 (0.29 – 0.44) 0.54 (0.48 – 0.60) • Anxiety (ASS8) (k=8) -0.39 (0.31 – 0.46) 0.55 (0.49 – 0.61) • ADHD (k=6) -0.31 (0.24 – 0.39) 0.51 (0.45 – 0.57) Females

Pearsons coefficient (95 % confidence interval) Euthymia subscale Dysthymia subscale The scaled total score (k = 83) -0.43 (0.36 – 0.50) 0.59 (0.53 – 0.64) Traditional distress factors

• Somatization (k=12) -0.29 (0.21 – 0.36) 0.39 (0.32 – 0.46) • Hostility (k=6) -0.32 (0.24 – 0.39) 0.38 (0.30 – 0.45) • Interpersonal sensitivity (k=9) -0.49 (0.42 – 0.55) 0.62 (0.57 – 0.67) Clinical syndrome • Depression, MDI (k=10) -0.42 (0.35 – 0.49) 0.57 (0.51 – 0.62) • Depression, HAM-D6 (k=6) -0.39 (0.32 – 0.46) 0.53 (0.46 – 0.58) • Anxiety (ASS8) (k=8) -0.43 (0.36 – 0.49) 0.59 (0.53 – 0.64) • ADHD (k=6) -0.36 (0.29 – 0.43) 0.47 (0.40 – 0.53) 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

Somatization items (k=12)

1 Headaches

4 Faintness or dizziness 12 Pains in heart or chest 27 Pains in lower back 40 Nausea or upset stomach 42 Soreness of muscles

48 Trouble getting your breath 49 Hot or cold spells

52 Numbness or tingling in part of the body 53 Lump in your throat

56 Feeling weak in parts of your body 58 Heavy feeling in your arms or legs

Hostility items (k=6)

11 Feeling easily annoyed or irritated

24 Temper outbursts that you could not control 63 Having urges to beat, injure or harm someone 67 Having urges to break or smash things

74 Getting into frequent arguments 81 Shouting or throwing things

Interpersonal sensitivity (k=9)

6 Feeling critical of others.

21 Feeling shy or uneasy with the opposite sex 34 Your feelings being easily hurt

36 Feeling others do not understand you or are unsympathetic 37 Feeling that people are unfriendly or dislike you

41 Feeling inferior to others

61 Feeling uneasy when people are watching or talking about you 69 Feeling very self-conscious with others

73 Feeling uncomfortable about eating or drinking in public

Major depression (k=10)

30 Feeling sad

32 Feeling no interest in things

14 Feeling low in energy or slowed down 41 Feeling inferior to others

26 Blaming yourself for things 15 Thoughts of ending your life 55 Trouble concentrating

78 Feeling so restless you couldn’t sit still /or 71 Feeling everything is an effort (highest score) 66 Sleep that is restless or disturbed

19 Poor appetite /or 60 Overweight (highest score) 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

For Peer Review Only

Depression, HAM-D6 (k=6)

30 Feeling sad

26 Blaming yourself for things 31 Worrying too much for things 71 Feeling everything is an effort

14 Feeling low in energy or slowed down 32 Feeling no interest in things

Anxiety (ASS8) (k=8)

2 Nervousness or shakiness inside 31 Worrying too much about things

50 Having to avoid certain things, places, or activities because they frighten you 23 Suddenly scared for no reason

72 Spells of terror or panic

45 Having to check and double-check what you do

65 Having to repeat the same actions such as touching, counting, washing 73 Feeling uncomfortable about eating or drinking in public

ADHD (k=6)

55 Trouble concentrating 9 Trouble remembering things

28 Feeling blocked in getting things done 78 Feeling so restless you couldn’t sit still 11 Feeling easily annoyed or irritated

24 Temper outbursts that you could not control 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57