MEASURING UNIVERSITY STUDENTS’ APPROACHES TO LEARNING STATISTICS: A CROSS-CULTURAL AND MULTILINGUAL VERSION OF THE ASSIST
Francesca Chiesi1, Caterina Primi1, Ayse Aysin Bilgin2, Maria Virginia Lopez3, and Maria del Carmen Fabrizio3 1
NEUROFARBA – Section of Psychology, University of Florence, Italy 2
Department of Statistics, Macquarie University, Australia 3
Faculty of Agriculture, University of Buenos Aires, Argentina francesca.chiesi@unifi.it
University students often encounter difficulties in statistics courses that hinder their progress in the attainment of their degree. In identifying variables that may constitute barriers faced by students, it is important to investigate the approach that students adopt in learning statistics. Focusing on issue of measurement, the present paper aimed to develop a brief version of the Approaches and Study Skills Inventory for Students (ASSIST), one of the well-known measures of a student’s approach to learning. The final goal was to obtain a cross-cultural and multilingual version of the ASSIST to investigate learning approaches in multinational research. Results indicated that the abbreviated Spanish, Italian and English versions of the ASSIST showed good psychometric properties and the three-factor structure of the original version (Deep, Surface and Strategic approaches) was confirmed.
INTRODUCTION
It is crucial that during their tertiary education, students are enabled to interpret and critically evaluate statistical information, understand data-related arguments, build intuition about data, and make reasoned judgments and decisions. In order to accomplish this goal and promote statistical literacy in students, all over the world many higher education degree programs have included statistics courses in their curriculum to better prepare their graduates for the work environment, and enable them to deploy evidence based practices in their work. Unfortunately, statistics is commonly viewed as a difficult and unpleasant topic and students often perceive statistics courses as a burden, sometimes fail to pass the exams, and these failing grades may cause students to abandon their academic and professional aspirations. Thus, research focused on improving statistical education may be able to identify variables that attenuate or accentuate statistical learning and determine the nature of barriers faced by students.
Learning approaches, one of the most widely used frameworks for understanding how students go about learning in their higher education (Tight, 2003), may offer one such perspective on this problem, but currently have not yet been investigated referring specifically to the study of statistics. Within this framework, learning approaches are defined as deep, surface, and strategic (Tait, Entwistle, & McCune, 1998). A deep approach to learning is characterized by a personal commitment to learning and an interest in the subject. Students adopting this approach aim to understand the topics, they engage critically with the arguments put forward, relate them to their prior knowledge, and evaluate whether conclusions are justified by the evidence (Biggs, 2003; Ramsden, 2003). Consequently, deep learning is more likely to result in better retention, integration, and application of knowledge and lead to higher quality learning outcomes (e.g., Ramsden, 2003). In contrast, a surface approach is characterized by a lack of personal engagement in the learning process. As such, students focus on rote-learning, they tend to study the topics in an unrelated manner, and they are constrained by the specific task. This approach often leads to misunderstanding of important concepts and, as a consequence, to poor learning outcomes (Ramsden, 2003; Watkins, 2000). Finally, students who adopt a strategic approach are primarily focused on achieving the highest possible grades. Their interest in content is driven by assessment demands and they use whatever learning strategy will maximize their chances of academic success (Watkins, 2000). These students have a competitive and vocational motivation and have been described as cue seekers, i.e., their main goal is to pursue hints regarding the content of assessment from their teachers (Duff, 2004). In sum, students learn by relying on understanding, on rote memorizing, or on finding strategies that fit the specific course requirements.
Concerning measurement, while the first research on students’ approaches to learning was mainly qualitative (Marton & Saljo, 1976a; 1976b), subsequent researchers developed inventories for use with large samples. Among them, the Approaches and Study Skills Inventory for Students (ASSIST; Tait, Entwistle, & McCune, 1998) is one of the most widely used instrument to measure students’ learning approaches in higher education. The validity and reliability of the ASSIST has been confirmed in several studies in different countries and within different disciplines (Buckley, Pitt, Norton, & Owens, 2010; Byrne, Flood, & Willis, 2004; Diseth, 2001; Entwistle, Tait, & McCune, 2000; Kreber, 2003; Reid, Duvall, & Evans, 2005). Nonetheless, some studies failed to confirm the factor structure that was anticipated by the theory (e.g., Evans, Kirby & Fabrigar, 2003). In particular, whereas the two constructs of deep and surface approaches appear to be generally supported, there is still some doubt about the strategic approach (Cuthbert, 2005).
Starting from this premise, we aimed to develop an instrument derived from the ASSIST to measures university students’ learning approaches to statistics, and to prove that it is equally reliable and valid in versions with different languages. Indeed, our main aim was to conduct a comprehensive study in three different countries - Argentina, Italy, and Australia - to gain a better understanding of the influence of students’ learning approaches on their achievement in statistics. In detail, the aim of the present study was two-fold. First, we aimed to develop an abbreviated version of the ASSIST in order to better define the three-factor structure of the scale, excluding items referring to variables that are not directly related to learning approaches (e.g. anxiety or general interest).Second, we aimed to test the invariance of the English, Spanish, and Italian versions of this revised scale.
METHODS Participants
Data were collected at four locations. The first cohort came from the University of Florence in Italy where students were enrolled in introductory statistics courses. There were 409 students in this cohort with mean age of 20.8 years (SD=3.80), and most of the participants were women (82%). This percentage reflects the gender distribution of the population of psychology students in Italy. The second cohort also came from the University of Florence in Italy (403 participants where 79% were female, mean age was 20.5 years (SD=3.26). The third cohort consisted of 543 university students enrolled in undergraduate introductory statistics courses at the University of Buenos Aires (38% female students, mean age of 20.9 years, SD=3.51). Final cohort consisted of 292 university students enrolled in undergraduate introductory statistics courses at the Macquarie University (50% female students, mean age of 21.8 years, SD=4.62). All students participated on a voluntary basis after they were given information about the general aim of the investigation (i.e. collecting information to improve students’ achievements in statistics).
The course of interest in Italy was compulsory for first year students. The course ran for ten weeks, and consisted of a four hour lecture and two hour tutorial (with students working in groups) per week (for a total of 60 hours per semester). Classes were based around the discussion of theoretical issues, followed by practical examples and pen and paper exercises. Also in Argentina the course was compulsory. It ran for 16 weeks and consisted of a two hour lecture, a three hour tutorial per week, and two practicals a term (for a total of 80 hours per semester). In Australia, although the course was not compulsory for all students, many courses in the University have this unit as a prerequisite for further study. Students were required to attend a two hour lecture, one hour tutorial and one hour practical each week (for a total of 50 hours per semester). In all three countries, tutorials consisted of guided pen and paper problem solving tasks including manual calculations, and practicals involving the use of statistics packages.
Concerning assessment, in Italy it consisted of a group report (an ungraded assignment designed to provide students with formative feedback), a written final examination composed of problem solving exercises and open-ended questions, and an oral examination. In Argentina, the performance of students was assessed through continuous evaluation (with assignments that are submitted in every class) and two midterm tests (four or five problem-solving exercises). Students who obtained intermediate performances (between 40% and 70% success) were required to sit a final examination (if below 40% they failed, if above 70% they passed without the final exam),
which consisted of multiple choice questions. In Australia assessments for the unit included online quizzes, three group-based assignments, a class test (multiple choice questions) run under exam conditions organized during tutorials just before the mid semester break, and a final examination, including exercises and open-ended questions.
Measure and Procedure
The ASSIST contains 52 statements, and respondents indicate the degree of their agreement with each statement using a five-point Likert scale where 1=disagree and 5=agree. The statements are combined into 13 subscales of four statements each, which are then further grouped into the three main scales Deep (Seeking meaning, Relating ideas, Use of evidence, Interest in ideas), Strategic (Organized Studying, Time Management, Alertness to Assessment Demands, Achieving, Monitoring Effectiveness), and Surface (Lack Of Purpose, Unrelated Memorizing, Syllabus-Boundless, Fear of Failure). The ASSIST has been translated into Italian and Spanish using a forward-translation method: for each version, two non-professional translators worked independently and then they compared their translations with the purpose of assessing the equivalence. Then, a small group of Italian-speaking and Spanish-speaking people read the first obtained versions, they revised it, and then the final Spanish and Italian forms were obtained.
In all countries, we surveyed students towards the end of their study period so that they were exposed to almost all the concepts to be covered in the semester, had been assessed in some aspects of their learning, and would have been given feedback on their assessment tasks.
RESULTS
In line with the original version of the ASSIST (Tait et al., 1998), analyses were conducted on the thirteen ASSIST subscale scores: Seeking Meaning (SM), Relating Ideas (RI), Use of Evidence (UE), Interest in Ideas (II), Organized Studying (OS), Time Management (TM), Alertness to Assessment Demands (AAD), Achieving (A), Monitoring Effectiveness (ME), Lack of Purpose (LP), Unrelated Memorizing (UM), Syllabus-Boundless (SB), Fear of Failure (FF).
Content analysis at the item level has revealed that two subscales (FF and II) addressed general issues, anxiety and general interest in studying, that are related to learning approaches but refer to different constructs. Thus, from a theoretical standpoint we decided to leave out these subscales. Then, an exploratory factor analysis was conducted with SPSS 17.0 applying Principal Axis Factor Estimation. The Bartlett’s test (χ²=1169.30, df=36, p<.001) attested that the data were adequate for the analysis. The number of factors to be extracted was determined by eigenvalues above 1.0 which indicated three latent factors. We applied an Oblimin Rotation and factor loadings attested that all the subscales saturated highly in the expected factor with the exception of AAD and ME (both factor loadings were <.40). These two subscales were removed from the analysis, and then we obtained a three-factor structure explaining52.8% of the variance with all the subscales loading highly on the expected factor (Table 1). The Strategic factor correlated positively with the Deep factor (.41), the Surface factor correlated negatively with the Strategic factor (-.53) and the Deep factor (-.21). Concerning reliability, the internal consistency, measured using Cronbach's Alpha coefficients, attested an adequate internal consistency for each scale (Table 1).
Table 1. Factor loadings and percentage of accounted variance for the three-factor solution. Deep Surface Strategic
Seeking Meaning (SM) .66
Relating Ideas (RI), .73
Use of Evidence (UE) .54
Lack of Purpose (LP) .72
Unrelated Memorizing (UM) .71
Syllabus-Boundness (SB) .64
Organized Studying (OS) .75
Time Management (TM) .88
Achieving (A) .66
Cronbach’s Alpha .70 .73 .83
Cross-validation of the factorial structure
The above described brief version of the ASSIST was administered to a different sample of university students. This sample (n=403) was split in two sub-samples (202 cases for the calibration sample and 201 cases for the validation sample) to cross-validate the factorial structure on independent samples from the same population. This procedure was chosen to ensure that the constructs had not changed in the translation and to reduce the risk that the model might be driven by the characteristics of a particular sample (Cudek & Browne, 1983). The three-factor model was tested separately on data from the calibration and validation samples, and it showed a good fit for both samples. In the calibration sample (χ2
(21)=48.01, p<.01, χ2/df=2.0, CFI=.95, TLI=.93, RMSEA=.07) factor loadings were all significant (p<.001) and ranged from .58 to .89, the Strategic factor correlated positively with the Deep factor (.44, p<.001), and the Surface factor correlated negatively with the Strategic factor (-.64, p<.001) and the Deep factors (-.36, p<.01). In the validation sample (χ2(21)=46.62, p<.01, χ2
/df=1.9.5, CFI=.96, TLI=.94, RMSEA=.07) factor loadings were all significant (p<.001) and ranged from .58 to .81, the Strategic factor correlated positively with the Deep factor (.46, p<.001), and the Surface factor correlated negatively with the Strategic factor (-.71, p<.001) and the Deep factors (-.26, p<.01).
Then, a multi-group CFA was performed. Testing for factorial equivalence involves a series of hierarchically ordered steps that begin with the establishment of a baseline model with no equality constraints (Baseline). The fit of the Baseline model serves to evaluate if the same pattern of fixed and free parameters holds across samples. Increasingly stringent hypotheses of equivalence are tested in subsequent models by imposing equality constraints on different sets of parameters. The tenability of hypotheses of equivalence is determined by comparing the difference in fit between nested models. In the present study, the Baseline model (unconstrained model) was compared with three different models: Model A (invariance of measurement weights), Model B (Model A plus invariance of structural covariances), Model C (Model B plus measurement residuals). Differences in χ2
values between models were not significant when comparing the unconstrained model with the increasingly more constrained ones. According to the principle of parsimony, the Model C was chosen. Fit comparisons of models are presented in Table 2.
Table 2. Goodness-of-Fit statistics for cross-validation assuming the unconstrained model (Baseline) is correct.
Model χ2 df χ2 /df ∆χ2 ∆df p(∆χ2) CFI TLI RMSEA
Baseline 94.63 48 1.97 - - - .96 .94 .049
Model A 106.93 54 1.98 12.29 6 .06 .95 .94 .049
Model B 112.94 60 1.88 18.30 12 .10 .95 .94 .047
Model C 119.63 69 1.73 24.99 21 .25 .95 .95 .043
Factorial invariance across countries
Parameter invariance is of particular concern when using a translated version of a survey instrument, and it allows the use of the translated instrument in cross-cultural research (e.g., Baumgartner & Steenkamp, 1998). Testing the factorial equivalence among the Spanish, English, and Italian versions involves a series of hierarchically ordered steps (as detailed above) that begin with the establishment of a baseline model for each version separately. Thus, the first goal of this analysis was to confirm this factorial structure of the Italian version of the ASSIST for the English and Spanish versions. The three-factor structure was tested on data from the Argentinean sample. The model showed a good fit: χ2(24)=96.73, p<.001, χ2
/df=4.03, CFI=.94, TLI=.91, RMSEA=.075. For the measurement model, each of the subscales loaded strongly and significantly on their hypothesised factor (factor loadings ranged from .54 to .76). For the structural model, a positive correlation was found between Deep and Strategic (.37), Surface correlated negatively with Deep (-.25) and Strategic (-.41). For the Australian sample the model showed a good fit: χ2
(24)=76.41, p<.001, χ2/df=3.2, CFI=.95, TLI=.92, RMSEA=.07. For the measurement model, each of the subscales loaded strongly and significantly on their hypothesised factor (factor loadings ranged
from .65 to .84). For the structural model, a positive correlation was found between Deep and Strategic (.59), Surface correlated negatively with Deep (-.13) and Strategic (-.24).
Since confirmation of the same factor model is not a sufficient condition to establish the equivalence of the Spanish, English, and Italian versions of ASSIST, we tested the invariance of the factor model’s parameters across the three samples. The overall and comparative fit statistics of invariance models are presented in the Table 3. Whereas the chi-square tests supported the measurement (Model A) and the structural invariance (Model B) hypotheses, the test was significant for Model C indicating that there was not equality of measurement errors across countries.
Table 3. Goodness-of-Fit statistics for test of invariance across countries assuming the unconstrained model (Baseline) to be correct.
Model χ2 df χ2
/df ∆χ2 ∆df p(∆χ2) CFI TLI RMSEA
Baseline 411.98 93 4.43 - - - .91 .90 .053
Model A 416.86 99 4.21 4.88 6 .56 .91 .90 .051
Model B 427.05 105 4.06 15.07 12 .24 .91 .90 .050 Model C 500.85 114 4.39 88.87 21 <.001 .90 .89 .052 CONCLUSION
The aim of the current study was two-fold.
First, we aimed to develop an abbreviated version of the ASSIST in order to better define the three factor structure referring to the Deep, Surface, and Strategic approaches. Once items that covered aspects of a student’s motivation not directly related to learning approaches were eliminated, the factor analyses provided evidence of the underlying three-factor structure of this abbreviated version of the ASSIST. In detail, the Deep approach scale includes items covering the intention to understand, that is the effort put in to seeking meaning, relating ideas and using evidence. The surface approach scale covers aspects related to unreflective studying, that is memorising without understanding and without relating the topics among them, poor involvement, general lack of interest and the intention to do very little beyond what is actually required to pass. The strategic approach scale is more concerned with organising study including time management and directing effort.
Second, we aimed to test the invariance of the English, Spanish, and Italian versions of this revised scale. The multigroup confirmatory factor analysis provided evidence that the three dimensions established for the Italian version were confirmed in the Spanish and English versions. Indeed, invariance testing attested a substantial equivalence between the three versions with the exception of the equality of measurement errors. However, reviewing the inconsistencies in the literature regarding the equality of the residuals, Vandenberg and Lance (2000) suggested that group differences in residual variances are indicative of differences in measurement reliability (i.e. random noise) rather than evidence of bias. Thus, the invariance at the structure level (number and relationships among factors) and at the measurement level (relationships between latent variables and their indicators) support the hypothesis that the ASSIST may be used across countries in the three different languages.
Nonetheless, the present study needs further investigation regarding the validity of the abbreviated version of the ASSIST. Thus, the construct validity of the scale should be explored relating the new scores obtained for the Deep, Surface, and Strategic approaches to measures of constructs that are expected to be related to learning approaches, and to measures of constructs that should have no relationship with them.
In conclusion, the abbreviated version of the ASSIST obtained in this study represents an instrument to measure learning approaches equally reliably and validly in the different language versions. Thus, it is a useful tool for a multinational study aiming to develop better understanding of the influence of students’ learning approaches on their achievement in statistics. We know that students in the different countries not just differ in their language, but they might also differ with degree programs enrolled in, with different degree of compulsion to attend the statistics classes and
different requirements for passing their courses class, this heterogeneity might help in investigating the relevance of each specific learning approach once the invariance at the measurement level is attested.
REFERENCES
Biggs, J. (2003). Teaching for quality learning at University (2nd edition). Buckingham: SRHE and Open University Press.
Buckley, C.A., Pitt, E., Norton, B. & Owens, T. (2010). Students approaches to study conceptions of learning and judgments about the value of networked technologies. Active Learning in Higher Education, 11, 55-65.
Byrne, M., Flood, B., & Willis, P. (2004). Validation of the approaches and study skills inventory for students (ASSIST) using accounting students in the United States and Ireland: A research note. Accounting Education: An International Journal, 13, 449-459.
Cuthbert, P.F. (2005). The student learning process: Learning styles or Learning approaches? Teaching in Higher Education, 10, 235-249.
Diseth, A. (2001). Validation of a Norwegian version of the approaches and study skills inventory for students (ASSIST): An application of structural equation modelling. Scandinavian Journal of Educational Research, 45, 381-394.
Duff, A. (2004). Understanding academic performance and progression of first year accounting and business economics undergraduates: The role of approaches to learning and prior academic achievement. Accounting Education: An International Journal, 13, 409-430.
Entwistle, N., Tait, H., & McCune, V. (2000). Patterns of response to an approaches to studying inventory across contrasting groups and contexts. European Journal of Psychology of Education, 15, 33-48.
Evans, C. J., Kirby, J. R., & Fabrigar, L. R. (2003). Approaches to learning, need for cognition, and strategic flexibility among university students. British Journal of Educational Psychology, 73, 507-528.
Kreber, C. (2003). The relationship between students’ course perception and their approaches to studying undergraduate science courses: A Canadian experience. Higher Education Research & Development, 22, 57-75.
Marton, F. & Saljo, R. (1976a). On qualitative differences in learning, I: Outcomes and process. British Journal of Educational Psychology, 46, 4-11.
Marton, F. & Saljo, R. (1976b). On qualitative differences in learning, II: Outcome as a function of the learner's conception of the task. British Journal of Educational Psychology, 46, 115-27. Ramsden, P. (2003). Learning to teach in higher education (2nd edition). London: Routledge
Falmer.
Tight, M. (2003). Researching higher education. Maidenhead: McGraw Hill.
Reid, W., Duvall, E., & Evans, P. (2005). Can we influence medical students’ approaches to learning? Medical Teacher, 27, 401-407.
Tait, H., Entwistle, N., & McCune, V. (1998). ASSIST: A re-conceptualisation of the approaches to studying inventory. In C. Rust (Ed.) Improving student learning: Improving students as learners (pp. 262- 271). Oxford: The Oxford Centre for Staff and Learning Development. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the MI literature: Suggestions,
practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.
Watkins, D. (2000). Learning and teaching: A cross-cultural perspective. School Leadership and Management, 20, 161-173.
