21 July 2021 Original Citation:
Timing of Gestures: Gestures Anticipating or Simultaneous With Speech as Indexes of Text Comprehension in Children and Adults
Published version: DOI:10.1111/cogs.12381 Terms of use:
Open Access
(Article begins on next page)
Anyone can freely access the full text of works made available as "Open Access". Works made available under a Creative Commons license can be used according to the terms and conditions of said license. Use of all other works requires consent of the right holder (author or publisher) if not exempted from copyright protection by the applicable law. Availability:
This is the author's manuscript
1
This is an author version of the contribution published on:
Questa è la versione dell’autore dell’opera:
[Ianì, Cutica, & Bucciarelli, Cognitive Science, 2016, pagg.1-18 doi:
10.1111/cogs.12381]
The definitive version is available at:
La versione definitiva è disponibile alla URL:
2
Timing of gestures:
Gestures anticipating or simultaneous with speech as indexes of text comprehension in children and adults
Francesco Ianìa, Ilaria Cuticab, & Monica Bucciarellia
a
Department of Psychology and Center for Cognitive Science, University of Turin Via Po, 14 –10123 Turin, Italy
b
Department of Health Sciences, University of Milan Via A. Di Rudinì, 8 – 20140, Milan, Italy
Corresponding author: Francesco Ianì University of Turin Department of Psychology Via Po, 14 10123 Turin, Italy e-mail: francesco.iani@unito.it tel: +39.011.6703038; fax:+39.011.8146231
3
Abstract
The deep comprehension of a text is tantamount to the construction of an articulated mental model of that text. The number of correct recollections is an index of a learner’s mental model of a text. We assume that another index of comprehension is the timing of the gestures produced during text recall; gestures are simultaneous with speech when the learner has built an articulated mental model of the text, whereas they anticipate the speech when the learner has built a less articulated mental model. The results of four experiments confirm the predictions deriving from our assumptions for both children and adults. Provided that the recollections are correct, the timing of gestures can differ and can be considered a further measure of the quality of the mental model, beyond the number of correct recollections.
4
1. Introduction
Deep comprehension and learning from text involve the construction of a mental model of the information provided in the text (Cutica & Bucciarelli, 2008; Graesser, Singer & Trabasso, 1994; McNamara, Miller & Bransford, 1991). How to assess the learner's internal mental models or, in other words, her level of understanding, remains an open question in the literature on learning (Bucciarelli & Cutica, 2012). In this study we propose that the timing of gesture production at recall might be a good candidate as an index of text comprehension.
Enactment is a fruitful way of improving learning from text; enactment of a text results in better recall of the information in that text by both children (Cutica, Ianì & Bucciarelli, 2014) and adults (Cutica & Bucciarelli, 2013). More specifically, gesturing while reading a text favors the construction of an articulated mental model of the text, as measured through more recollections in the form of paraphrases and more text-based inferences. However, while the beneficial role of gestures during the learning phase has been extensively studied (see, e.g., Feyereisen, 2009;
Hornstein & Mulligan, 2004), fewer investigations have taken into account the role and the features of gestures during the recall phase. We predict that, in learning from text, the temporal relationship between gestures produced at recall and the correct recollections they accompany might reflect different levels of text comprehension. We distinguish between gestures produced in advance of speech and gestures produced simultaneously with speech. Gestures produced in advance of speech should reflect an effort to organize the discourse in case the speaker has not built a sufficiently articulated mental model of the concepts to be expressed. Indeed, we know that gestures help the organization and production of discourse in both children and adults, both at a conceptual and lexical level (see, e.g., Cutica & Bucciarelli, 2011; Kita & Davies, 2009). In particular, if we consider the temporal relationship between gestures and speech in terms of early and advanced stages of learning a task, we know that at an early stage learners are often unable to express new
5
concepts in words, whereas their gestures may reflect an implicit knowledge of the concept; in other words, the gestural expression of a given concept might be produced slightly in advance with respect to the verbal expression when speakers have difficulty in expressing new concepts (e.g., Church & Goldin-Meadow, 1986). If we consider the temporal relationship between gestures and speech in terms of speech production, we know that the gestural expression of a given concept is produced slightly in advance with respect to the verbal expression when the speaker has difficulty in retrieving lexical items (e.g., Krauss, Chen & Gotfexnum, 2000). Conversely, we assume that gestures may be produced simultaneously with speech when the speaker has built a well articulated mental model of the concepts to be expressed. Indeed, we know that when speech flows fluently, people tend to produce gestures that are closely synchronized with the words to which they refer; in particular, gestures and speech are usually produced within milliseconds of one another (e.g., McNeill, 1992).
In line with previous studies in the literature, we argue that gestures and speech may have a common origin (McNeill, 1985; Morrel-Samuels & Krauss, 1992). However, rather than purporting that gestures derive from images (see McNeill, 1985), we assume that they derive from mental models, mental representations more abstract than images in that a model represents a set of states of affairs rather than just a single state of affairs. In a mental model “the difference in representation between declarative knowledge and procedural knowledge disappears. The two types of knowledge continue to be different and are perfectly distinguishable, but they can now be represented in one single general modality, the mental model” (Bara, 1995, p.116). Hence, mental models can activate abstract propositional representations that may be reflected in speech, but they can also activate motoric representations that may be reflected in gestures. Our assumption is consistent with the proposal that gestures derive from a system of motoric representations of concepts, many of which also come to be reflected in speech (Morrell-Samuels & Krauss, 1995). A series of studies have
6
revealed that children’s and adults’ verbal memory reports benefit from spontaneous gestures, as well as instructed gestures (see, for a review, Stevanoni & Salmon, 2005). A possible explanation is that they help verbal reporting by breaking down some forms of knowledge, such as perceptions, into a form that can be expressed in words (see, e.g., Alibali, Kita & Young, 2000). Another possible explanation is that gestures help speech retrieval and production (see, e.g., Rauscher, Krauss & Chen, 1996). In particular, Frick-Horbury and Guttentag (1998) argued that gestures may serve as retrieval cues for items whose verbal labels are not immediately accessible (see also Krauss, Chen & Chawla, 1996). When the speaker is unable to access the verbal target the gesture “may prime the lexical search by serving as an additional representation of the searched-for word, thereby cueing lexical access” (ibidem, p. 44); the activation of the motor component would prime the verbal component and therefore aid verbal production. Frick-Horbury and Guttentag (1998) examined the effects of restricting gestures on lexical retrieval and free recall to investigate whether performance on lexical retrieval tasks was better under unrestricted rather than restricted hand movement conditions. The participants in their study whose hand movements were restricted retrieved fewer words in response to word definitions, compared to the participants who were free to produce hand gestures. Alas, these studies did not analyze the temporality of gestures; within our proposed framework we would expect anticipatory gestures to assist both verbal reporting and retrieval of information more than simultaneous gestures. Morrel-Samuels and Krauss (1992) addressed gesture and speech asynchrony in a study on lexical retrieval as a function of word familiarity. They purported that “The words that constitute an utterance are a reflection of the speaker’s attempt to convey lexically the abstract concepts contained in the underlying proposition. For many reasons…the individual may be unable (a) to access a lexical entry that precisely
expresses the concept represented in the proposition or (b) to construct such an expression out of multiple lexical elements…” (ibidem, p. 616). A prediction descending from these assumptions is
7
that the more accessible the lexical affiliate, the closer its onset to that of the accompanying gesture. Morrel-Samuel and Krauss invited the participants in their experiment to provide narrative
descriptions of photographs and found that unfamiliar words were accompanied by greater asynchrony between gesture initiation and speech onset. We assume that individuals who have constructed an articulated mental model of a text are more likely to accompany correct recollections with simultaneous gestures, compared to individuals who have constructed a less articulated model, and vice-versa that individuals who have constructed a less articulated model are more likely to accompany correct recollections with anticipatory gestures. It is plausible that having a less
articulated model implies being unable to access a lexical entry that precisely expresses the concept represented in the proposition or to construct such an expression out of multiple lexical elements.
Our predictions can be applied only to correct recollections considered as an index of mental model quality (literal recollections, paraphrases and discourse-based inferences) and not to
erroneous ones: we have no predictions for erroneous recollections because they represent a heterogeneous group, originating from different causes, and not necessarily related to the good or bad quality of the mental model - for instance, if the learner misunderstands some piece of
information, the information that was misunderstood may even be included in a well articulated mental model, thus supporting a wrong recollection.
In sum, we propose that the timing of gestures may be an index to assess the learner’s internal mental model. Given the relevance of permanent learning, we aimed to verify the validity of such an index in both children’s and adults’ learning from text. Experiments 1 and 2 tested the prediction that children’s and adults’ possibility to gesture in the phase of learning a text, compared with the impossibility to gesture, results in a better mental model of the content to be learnt, with the consequence of a greater gesture-speech synchronicity at recall. Two further experiments were conducted to establish a direct link between gesture-speech synchronicity and mental model’s
8
quality, and to test the predictions that in good learners respect to poorer learners gesture speech at recall feature a greater synchronicity (in children: Experiment 3; in adults: Experiment 4).
2. Timing of gestures produced during recall of a learnt text
To test our predictions, we conducted an analysis of the gestures produced at recall by the children in a study by Cutica et al. (2014) (henceforth Experiment 1) and the adult participants in a study by Cutica and Bucciarelli (2013) (henceforth Experiment 2). Both the children and the adults were invited to study two scientific texts in two conditions; in the Gesture condition they were invited to enact, in the learning phase, the concepts in the text, whereas in the No Gesture condition they were invited to keep their hands still. The original results revealed that the participants were more accurate at recall in the Gesture condition than in the No Gesture condition; our new analyses aimed to ascertain whether, at recall, more simultaneous gestures were produced in the Gesture condition than in the No Gesture condition, and whether more anticipatory gestures were produced in the No Gesture than in the Gesture condition. A brief description of the two experiments is provided below, along with our new data analyses.
Participants
Experiment 1 Twenty-four fifth-grade students (12 females and 12 males; mean age 10;09
years) took part in the experiment voluntarily after their parents had given their informed consent. Each child encountered two scientific texts, one in the Gesture condition and one in the No Gesture condition.
Experiment 2 Forty university students (27 females, 13 males, mean age: 22 years) attending a
course in General Psychology at Turin University took part in the experiment on a voluntary basis. Each participant encountered two scientific texts, one in the Gesture condition and one in the No Gesture condition.
9
Material and Procedures
Experiment 1
One of the two scientific texts was about the circulatory system, and the other was about the pulling force (see the excerpts in Appendix A.1); neither of them addressed subjects that the
children had already studied. Each text consisted of 204 words. We counterbalanced the occurrence of the two texts in the Gesture and in the No Gesture condition across the participants. Furthermore, half of the participants dealt first with the Gesture condition and half with the No Gesture condition. The children participated in the experiment individually and in a quiet room, where the only other person present was the experimenter. The entire session was video-recorded.. In the Gesture condition the children were instructed as follows: “Please read this text out loud twice and study it. While you are reading it, use your hands to help yourself. Try to represent what you are reading with your hands. To make sure that the children had understood the instructions, before reading the text and by way of example they were asked to use their hands to represent an apple on the table and a flying kite. In the No Gesture condition the children were instructed as follows: ‘Please read this text out loud twice and study it, keeping your hands still and on the table’. In each condition, at the end of the second reading, children were asked to recall as much information as they could through the following instruction: “Now please tell me everything you can remember about the text you have just read, giving me as many information as possible”.
In order to code the children’s recollections, each of the scientific texts was divided into 18 semantic units, corresponding to the main concepts that the children could recall. Each semantic unit was coded by two independent judges according to the following coding schema (Cutica et al, 2014):
Literal recollection: a semantic unit recollected in its literality; Paraphrase: a semantic unit recollected as a paraphrase;
10
Discourse-based inference: a recollection in which the participant gave explicit information
that was already implicit in the semantic unit.
Elaborative inference: a semantic unit recollected with the addition of plausible details. Error: a recollection the meaning of which was inconsistent with the semantic unit.
Thus, literal recollections and paraphrases, along with discourse-based inferences, were counted as correct recollections, whereas elaborative inferences and errors were considered as wrong recollections.
The new analysis of the data involved three pairs of judges. Two independent judges blind to the aim of the experiment analyzed the video-recordings of the experimental sessions and coded the presence/absence of gestures at recall in correspondence with correct and wrong recollections both in the Gesture and the No Gesture condition. They were instructed as follows: “Identify, for each participant, any hand movement with a clear start and a clear end and produced in correspondence with a recollection. Code any movement of this kind as ‘gesture’. Do not code movements such as scratching or pushing hair off the face as gestures”.
Then, two different independent judges blind to the aim of the experiment coded the gestures produced in correspondence with the correct recollections as simultaneous or anticipatory. In particular, as most of the recollections were accompanied by a single gesture, the judges coded the first gesture produced in correspondence with the recollection. The judges were given the following instructions: “Consider whether the gesture was initiated with the start of a word in the recollection or slightly before the word but not with a preceding word. The participants might produce gestures with different timing with respect to the word:
• a gesture for which the preparatory phase (i.e., the moment in which a movement away from the resting position begins in order to prepare the gesture) starts before the word is an anticipatory
11
• a gesture for which the preparatory phase starts in correspondence with the word is a
simultaneous gesture.”
This coding subsequently led to an analysis of the percentages of correct recollections accompanied by anticipatory and simultaneous gestures.
Moreover, to detect whether any gestures conveyed information contrasting with the information expressed in the speech (mismatching gestures), two different independent judges examined each gesture-word pair: they detected no gesture-speech mismatches.
Experiment 2
The material consisted of two scientific texts, one concerning airplane flight and the other concerning the nature of sound (see the excerpts in Appendix B.1). Each text consisted of 370 words. The experimental procedures and instructions were identical to those of Experiment 1, with the exception that to code the results each text was divided into 52 semantic units. Two judges coded each concept recalled by the participants according to the same coding schema used in Experiment 1 (Cutica & Bucciarelli, 2013).
As for Experiment 1, the three pairs of judges involved in the new analysis coded the presence/absence of gestures in correspondence with the recollections, the timing of the gestures, and the presence of speech mismatches. As regards the latter, they detected no gesture-speech mismatches.
Results
The results of Experiments 1 and 2 confirmed the original predictions that both children and adults would construct a more articulated mental model of the text in the Gesture condition, compared to the No Gesture condition, as revealed by the greater number of paraphrases and
discourse-based inferences. The results of our new analyses of the gestures produced at recall by the participants in the two experiments will now be presented.
12
Experiment 1
The judges reached a significant level of agreement on their first judgments for the two conditions overall as regards the presence/absence of gestures in correspondence with the recollections (they agreed on 96% of the items: Cohen’s K for the independent encodings of
presence/absence of gestures could not be calculated neither here nor in Experiments 2, 3 and 4, as it does not involve a ‘0’ score, namely absence of gestures) and for the coding of gestures as simultaneous or anticipatory with respect to correct recollections (Cohen’s K=.91, p<.001). In both cases, for the final score the judges discussed each item on which they disagreed until reaching full agreement. As a general result, the children produced a mean number of 7.5 gestures at recall (sd= 5.9); a mean of 4.9 in the Gesture condition (sd=4.0) and a mean of 2.6 in the No Gesture condition (sd=3.4). Examples of gestures coded as anticipatory and simultaneous are given in Appendix A.2.
Table 1 illustrates the percentages of correct recollections as a function of the timing of the gestures produced.As predicted, in the Gesture condition children produced more correct
recollections simultaneously accompanied by gestures than in the No Gesture condition (Wilcoxon test: z=3.0, p<.004, Cliff’s δ =.50) and more correct recollections accompanied by anticipatory gestures in the No Gesture condition than in the Gesture condition (Wilcoxon test: z=2.1, p<.04,
Cliff’s δ =.17).
Insert Table 1 about here
Experiment 2
The judges’ level of agreement on the first judgments was significant for both the two conditions overall as regards the presence/absence of gestures in correspondence with the recollections (they agreed on 97% of the items) and the coding of gestures as simultaneous or anticipatory with respect to correct recollections (Cohen’s K=.80, p<.001); in disagreement, they discussed each itemuntil reaching full agreement. As a general result, in both conditions the
13
participants produced a mean number of 3.9 gestures at recall (sd=2.9); a mean of 2.8 (sd=2.2)in the Gesture condition and a mean of 1.0 (sd=1.4) in the No Gesture condition. Examples of gestures coded as anticipatory and simultaneous are given in Appendix B.2.
Table 2 illustrates the percentages of correct recollections as a function of the gestures’ timing. As predicted, in the Gesture condition individuals produced more correct recollections simultaneously accompanied by gestures than in the No Gesture condition (Wilcoxon test: z=2.59,
p=.01, Cliff’s δ=.39). However, contrary to our expectations, individuals produced correct
recollections accompanied by anticipatory gestures to the same extent in the two conditions (Wilcoxon test: z=1.74, p=.08, Cliff’s δ=.24).
Insert Table 2 about here
Discussion of the results of Experiments 1 and 2
The results of the experiments confirmed our predictions with only one exception: adults in Experiment 2 produced few anticipatory gestures in both the Gesture and the No Gesture
conditions, resulting in a floor effect. One possible explanation is that adults, but not children, became aware of our hypothesis and were more likely to organize their discourse production, as reflected by a greater gesture-speech synchrony. If this explanation is plausible, when gestures are not experimentally manipulated and the participants are free to gesture or not during the learning phase, we would expect adults to produce anticipatory gestures at recall too.
The results of the experiments revealed that when children and adults were told to gesture while reading a text, this had two consequences. First, they produced more correct recollections; second, they tended to produce more simultaneous gestures when recalling the information.Both these outcomes suggest that simultaneous gestures reveal a good quality mental model whereas anticipatory gestures reveal a poor quality mental model. However, one may argue that these effects could be ascribable to gesture manipulation during the learning phase, for instance to the fact that
14
gesturing during encoding might make later gesturing about the same material more fluent, independent of any relationship with the quality of the mental model of the text. We devised two further experiments, one with children and one with adults, to rule out this possibility.
2. Timing of gestures produced during recall of a learnt text: Good learners versus poor learners
In the two new experiments, the children and the adult participants were invited to study and recall two scientific texts, one at a time, without any manipulation during the learning phase,
therefore under more natural and ecological conditions compared to those in the previous
experiments. As for Experiments 1 and 2, they were asked to read each text out loud twice; at the end of the second reading they were asked to recall as much information as they could.
In order to quantify the differences in the quality of the mental models both among children and adults we identified in each group of participants those whose performance in terms of correct recollections was significantly below the mean, around the mean, and significantly above the mean. We predicted that the production of anticipatory gestures with correct recollections would decrease from poor learners, to average learners to good learners, whereas the production of simultaneous gestures with correct recollections would increase from poor learners, to average learners to good learners. If our predictions are confirmed, we can discard the possibility that the increased
production of simultaneous gestures in the gesture condition of Experiments 1 and 2 was due to the invitation to gesture during encoding, and not to the construction of a better mental model. Also, a further aim of the experiment on adults was to detect any differences in their accuracy in a more natural context and verify whether poor learners were more likely than good learners to accompany correct recollections with anticipatory gestures. If our prediction is confirmed we can discard the possibility that the adults in Experiment 2 produced very few anticipatory gestures because these
15
diminish with age. Participants
Experiment 3 Thirty-six fifth-grade students (14 females and 22 males; mean age 10;4 years)
took part in the experiment voluntarily after their parents had given their informed consent. Each child encountered two scientific texts, one in the Gesture condition and one in the No Gesture condition.
Experiment 4 Thirty university students (24 females, 6 males, mean age: 23;5 years)
attending a course in General Psychology at Turin University took part in the experiment on a voluntary basis. Each participant encountered two scientific texts, one in the Gesture condition and one in the No Gesture condition.
Material and Procedures
Experiment 3
The two scientific texts differed from those of Experiment 1 because the children taking part in Experiment 3 had already studied the circulatory system and pulling force at the time of the experiment. One of the two scientific texts was about levers (218 words), and the other was about acid, basic and neutral substances (211 words; see the excerpts of the texts in Appendix C.1); neither of them addressed subjects that the children had already studied. We counterbalanced the occurrence of the two texts in the experimental session across the participants. The children participated in the experiment individually and in a quiet room, where the only other person present was the
experimenter. The entire session was video-recorded. The children were asked to read each text out loud twice and, at the end of the second reading, they were invited to recall as much information as they could. In order to code the children’s recollections, each of the scientific texts was divided into 20 semantic units, corresponding to the main concepts that the children could recall. Each semantic unit was coded by two independent judges according to the same coding schema of Experiments 1
16
and 2.Two different independent judges blind to the aim of the experiment coded the
presence/absence of gestures in correspondence with the recollections and two more independent judges coded the children’s gestures at recall as anticipatory or simultaneous with respect to their correct recollections. The children taking part in Experiment 3 produced a greater number of
gestures in correspondence with each single recollection than those in Experiment 1: a mean of 15.7 (sd=13.8) versus a mean of 7.5 gestures (sd=5.9) and the same result held for the adult participants in Experiments 4 and 2: a mean of 78.6 (sd=40.9) versus a mean of 3.9 (sd=2.9) gestures,
respectively. Therefore, we decided to code the timing of the gestures by taking into account all the gestures produced in correspondence with the correct recollections. The instructions to the judges were the same as for Experiments 1 and 2. This coding subsequently led to the analysis of the percentages of anticipatory and simultaneous gestures accompanying correct recollections.
Moreover, to detect whether any gestures might convey information in contrast with the information expressed in speech (mismatching gestures), two different independent judges examined each gesture-word pair: they detected no gesture-speech mismatches.
Experiment 4
The adult participants encountered the same two scientific texts encountered by the adult participants in Experiment 2: the airplane flight and the nature of sound. The participants were asked to read the text out loud twice and, at the end of the second reading, they were invited to recall as much information as they could.
For Experiment 4, the four pairs of judges involved in Experiment 3 coded each concept recalled by the participants according to the same coding schema used in Experiment 3, the presence/absence of gestures in correspondence with the recollections, the timing of the gestures, and the presence of speech mismatches. As regards the latter, they detected no gesture-speech mismatches.
17
Results
Experiment 3
The judges reached a significant level of agreement on their first judgments for the coding of children’s recollections (Cohen’s K=.829, p < .001), for the coding of the gestures produced in correspondence with the recollections (they agreed on 97% of the items), and for the coding of the timing of the gestures for both conditions (Cohen’s K ranging from .69 to .75, all p < .01). In all cases, for the final score they discussed each item on which they disagreed until reaching full agreement. Examples of gestures coded as anticipatory and simultaneous are given in Appendix C.2.
In order to identify children whose accuracy was below the mean, at the mean level, and above the mean, we adopted the following procedure. Of the 36 children we excluded 4 children from the analyses, because they either did not gesture or gestured only while not speaking. For the remaining 32 children we calculated the mean of correct recollections and the standard deviation. The 32 children produced a mean of 13.47 correct recollections with a standard deviation of 5.42. We selected those children whose performance was one standard deviation below and above the mean, henceforth poor learners and good learners, respectively. This criterion of selection led us to focus our analysis on 7 good learners and 5 poor learners. In order to identify average learners we considered the 6 children nearest to the mean performance (the number was established using the mean of the number of children in the two extreme groups). Then, a first analysis confirmed that the participants in these three groups significantly differed in accuracy: correct recollections increased from poor learners (a mean of 5.6, sd=2.1) to average learners (a mean of 13.4, sd=.5) to good learners (a mean of 21.4, sd=2.3), and the trend was statistically significant (Jonckheere trend test: z=4.50, p<.0001).
18
Figure 1 illustrates the percentages of correct recollections as a function of the timing of the gestures produced. Anticipatory gestures accompanying correct recollections decreased from poor learners to average learners to good learners, and simultaneous gestures accompanying correct recollections increased from poor learners to average learners to good learners: both trends were statistically significant (Jonckheere z=1.96, p<.03 in both cases).
Insert Figure 1 about here
Experiment 4
The judges’ level of agreement on thei first judgments was significant for the coding of children’s recollections (Cohen’s K=.939, p < .001), for the coding of the gestures produced in correspondence with the recollections (they agreed on 100% of the items), and for the coding of the timing for both conditions (Cohen’s K ranging from .76 to .91, all p < .001). In all cases, for the final score they discussed each item on which they disagreed until reaching full agreement.
As regards the crucial statistical analyses, we analyzed the data of all 30 adult participants because all of them gestured while speaking. They produced a mean of 39.10 correct recollections with a standard deviation of 15.29. In order to identify adults whose accuracy was below the mean, at the mean level, and above the mean, we adopted the same procedure as for the children. As a result, we focused our analysis on 4 good learners and 4 poor learners. In order to identify average learners in terms of accuracy, we considered the 4 adults nearest to the mean performance.
According to this criterion there were 6 average learners, because three participants ranked fourth with respect to the mean of correct recollections. A first analysis confirmed that the participants in the three groups significantly differed in accuracy: correct recollections increased from poor learners (a mean of 16.8, sd=4.8) to average learners (a mean of 37.8, sd=2.4) to good learners (a mean of 66.3, sd=8.2), and the trend was statistically significant (Jonckheere trend test: z=3.79, p<.0001).
19
Figure 2 illustrates the percentages of correct recollections as a function of the timing of the gestures produced. Anticipatory gestures accompanying correct recollections decreased from poor learners to average learners to good learners, and simultaneous gestures accompanying correct recollections increased from poor learners to average learners to good learners: both trends were statistically significant (Jonckheere z=2.01, p<.03, in both cases).
Insert Figure 2 about here
3. Discussion and Conclusions
We predicted that participants with a good mental model of the text would have less need to produce anticipatory gestures to help them in organizing their discourse. On the other hand, we assumed that participants with an articulated mental model of the text would be more likely to produce simultaneous gestures, an integral part of the recollection when coming from a good mental model. The results of the four experiments confirmed the predictions deriving from our
assumptions.
Studies in the literature reported that gestures can be predictive of improvement in learning. For example, a mismatch between a child's spoken explanation and gestures for solving a math problem (Church & Goldin-Meadow, 1986) or in learning about balance (Pine, Lufkin & Messer, 2004) indicates learning readiness. Our results, along with those earlier studies, suggest that a learner’s gestures can be informative of the learning process.
Our results are also in line with studies revealing that gestures facilitate the recall of information. For example, Stevanoni and Salmon (2005) instructed children to gesture while recalling an event in which they had taken part and found that their verbal recall was facilitated, compared to that of children who were not instructed to gesture while recalling the event. Our Experiments 1 and 2 manipulated gesture production at encoding; future studies might manipulate
20
gesture at recall, as Stevanoni and Salmon did, to conduct a more in-depth investigation into possible different beneficial effects of gestures at encoding and recall.
Our results suggest that the timing of gestures may be a valid index to assess a learner’s internal mental model. Our investigation, like those focused on indexes of a learner’s transitional state, can provide means for enabling teachers to calibrate their interventions on the person's level of comprehension (see Goldin-Meadow, Alibali & Church, 1993).
Limitations and future directions
As a first limit, our studies lack the granularity to allow us to investigate the temporality of gestures as a function of their typology: iconic and deictic gestures may, for example, play different roles in revealing to the observer the quality of the mental model entertained by the learner.
A second limit of our studies is that they treated timing as a dichotomous variable
(anticipatory versus simultaneous gestures); future studies might use computerized coding rather than human coding and they might allow us to discover that gesture timing is a continuous variable (see also Morrel-Samuels & Krauss, 1992): we predicted that the poorer a learner’s mental model the more anticipatory will be the gestures produced by the learner with respect to correct
recollections.
Finally, it would be of great interest to study the temporality of gestures with respect to circumstances other than learning, in which the individual still entertains a poor mental model; instances of deceit would be of particular interest. It is possible that simultaneous gestures reflect the sincerity of the speaker whereas anticipatory ones reveal the speaker’s deceitful intention. We are presently investigating this possibility.
21
Acknowledgements
This research was supported by a grant from Italian MIUR to the third author (Prin project 2010-2011, prot. 2010RP5RNM).
22
References
Alibali, M. W., Kita, S., & Young, A. J. (2000). Gesture and the process of speech production: We think, therefore we gesture. Language and cognitive processes, 15(6), 593-613.
Bara, B. G. (1995). Cognitive science: A developmental approach to the simulation of the mind. Psychology Press.
Bucciarelli, M., & Cutica, I. (2012). Mental models in improving learning. In Seel, N.M. (Ed.),
Encyclopedia of the Sciences of Learning, 2012, Part 13, 2211-2212. Springer: NY.
Church, R. B., & Goldin-Meadow, S. (1986). The mismatch between gesture and speech as an index of transitional knowledge. Cognition 23, 43–71.
Cutica, I., & Bucciarelli, M. (2008). The deep versus the shallow: Effects of co‐speech gestures in learning from discourse. Cognitive Science, 32, 921-935.
Cutica, I. & Bucciarelli, M. (2011). “The more you gesture, the less I gesture”: Co-speech gestures as a measure of mental model quality. Journal of Nonverbal Behavior, 35, 173-187.
Cutica, I. & Bucciarelli, M. (2013). Cognitive change in learning from text: Gesturing enhances the construction of the text mental model. Journal of Cognitive Psychology, 25, 201-209.
Cutica, I., Ianì, F., & Bucciarelli, M. (2014). Learning from text benefits from enactment. Memory
& Cognition, 42, 1026-1037.
Feyereisen, P. (2009). Enactment effects and integration processes in younger and older adults’ memory for actions. Memory, 17, 374-385.
Frick-Horbury, D., & Guttentag, R. E. (1998). The effects of restricting hand gesture production on lexical retrieval and free recall. The American journal of psychology, 111(1), 43.
Graesser, A.C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101, 371-395.
23
Hornstein, S.L., & Mulligan, N.W. (2004). Memory for actions: Enactment and source memory.
Psychonomic Bulletin & Review, 11, 367-372.
Goldin-Meadow, S., Alibali, M.W., & Church, R. B. (1993). Transitions in concept acquisition: Using the hand to read the mind. Psychological Review, 100, 279–297.
Kita, S., & Davies, T. S. (2009). Competing conceptual representations trigger co-speech representational gestures. Language and Cognitive Processes, 24, 761-775.
Krauss, R. M., Chen, Y., & Chawla, P. (1996). Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us?. Advances in experimental social
psychology, 28, 389-450.
Krauss, R. M., Chen, Y., & Gotfexnum, R. F. (2000). Lexical gestures and lexical access: a process model. In: D. McNeill (Ed.), Language and gesture, pp. 261-283. New York: Cambridge University Press.
McNamara, T., Miller, D.L., & Bransford, J.D. (1991). Mental models and reading comprehension. R. Barr, M.L. Kamil, P.B. Mosenthal, P.D. Pearson (Eds.), Handbook of Reading Research,
Vol. 2, 490-511. Erlbaum, Hillsdale.
McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92, 350-371. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago
Press.
Morrel-Samuels, P., & Krauss, R. M. (1992). Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 18(3), 615.
Pine, K.J., Lufkin, N., & Messer, D. J. (2004). More gestures than answers: Children learning about balance. Developmental Psychology, 40, 1059-1067.
24
Rauscher, F. H., Krauss, R. M., & Chen, Y. (1996). Gesture, speech, and lexical access: The role of lexical movements in speech production.Psychological Science, 7(4), 226-231.
Stevanoni, E., & Salmon, K. (2005). Giving memory a hand: Instructing children to gesture enhances their event recall. Journal of Nonverbal Behavior,29(4), 217-233.
