Francesco Ruotolo1, Tina Iachini1, Luigi Maffei2, Vincenzo Paolo Senese1, Gennaro Ruggiero1, Massimiliano Masullo2, Michela Vinciguerra1, Ortensia D’Errico1
1
Francesco Ruotolo, Department of Psychology, Second University of Naples, via Vivaldi 43, Caserta, 81100, Italia (phone: +39- 0823274770; e- mail: [email protected]).
1Tina Iachini, Department of Psychology, Second University of Naples, via Vivaldi 43, Caserta, 81100, Italia (phone: +39- 0823274770; e-mail:
2Luigi Maffei, Built Environment Control laboratory R.I.A.S., Second University of Naples, Via San Lorenzo, Monastero di San Lorenzo ad
Septimum, Aversa (CE), 81031, Italia (e-mail: [email protected]).
1Vincenzo Paolo Senese, Department of Psychology, Second University of Naples, via Vivaldi 43, Caserta, 81100, Italia (phone: +39-
0823274770; e-mail: [email protected]).
1Gennaro Ruggiero, Department of Psychology, Second University of Naples, via Vivaldi 43, Caserta, 81100, Italia (phone: +39- 0823274770; e-
mail: [email protected]).
2
Massimiliano Masullo, Built Environment Control laboratory R.I.A.S., Second University of Naples, Via San Lorenzo, Monastero di San Lorenzo ad Septimum, Aversa (CE), 81031, Italia (e-mail: [email protected]).
1Michela Vinciguerra, Department of Psychology, Second University of Naples, via Vivaldi 43, Caserta, 81100, Italia (phone: +39- 0823274770;
e-mail: [email protected]).
1Ortensia D’Errico, Department of Psychology, Second University of Naples, via Vivaldi 43, Caserta, 81100, Italia (phone: +39- 0823274770; e-
mail: [email protected]).
Abstract—The aim of this study is to propose a multisensory methodology to assess the impact of the wind turbine noise on
people. To this end, two different methodologies were compared: a traditional methodology, based on the presentation of acoustic stimuli (Audio condition), and an innovative multimodal methodology based on the integration of acoustic and visual stimuli presented by means of an Immersive Virtual Reality System (Audio+Video condition). Participants were assigned to one of the two experimental conditions and had to perform three cognitive tasks while listening to wind turbines sounds. Moreover, participants had to report their degree of perceived noise annoyance. The overall results showed that visual components in the Audio+Video condition modulate the impact of noise on cognitive performances and on perceived annoyance. The theoretical and practical implications of these findings are discussed.
Index Terms— Immersive virtual reality, Wind farm noise, Cognitive performance, Noise annoyance III. INTRODUCTION
The Embodied Cognition approach is progressively gaining consensus in the domain of Cognitive Science: cognition is viewed as originating from the interaction of body and environment and is defined by the characteristics of the body and of the specific environmental situation [1; 2; 3]. The foundation of this body-environment interaction is perception. Contrary to the conventional view, perception is not seen as the sum of isolated sensory inputs or qualities but as a dynamic commingling of sensory possibilities. This would give rise to a dynamic multimodal representation of the environment that influences the way we think and feel. For example, several studies have shown that our judgments about the pleasantness of a sound are influenced not only by the acoustic properties of the stimulus, but also by the characteristics of the visual context in which the sound is heard [4] and by the possibilities to act upon its source [5; 6]. In line with this, Viollon and colleagues [4] showed that the same road traffic noise was rated as less annoying when it was associated with a naturalistic visual setting with respect to when it was associated with a urban visual setting.
The use of the embodied cognition approach has far-reaching consequences for research in applied fields: the planning of new products, infrastructures and procedures should take into account how the brain-body-mind system works. In other words, applied cognitive science should be characterized by a principle of biological plausibility.
Let us consider the assessment of noise impact on people. Traditional methods have mainly focused on the acoustic parameters of the stimuli [6]. For example, in most of the studies participants were seated in a laboratory room and listened to different kinds of pre recorded sounds. Afterwards, they had to indicate the pleasantness of the sound or their degree of annoyance. These standard unimodal procedures reveal a fundamental limit: they reproduce environmental information in a simplified way to the expenses of ecological validity. Instead, procedures respecting a principle of biological plausibility should be based on the multisensory way in which people experience their environment. Furthermore, assessment of noise effects should comprehend objective measures (e.g. cognitive performance) along with self-report measures (e.g. annoyance).
The aim of this study is to propose a multisensory methodology to assess the impact of environmental noise, in particular of the wind turbine noise, on people’s annoyance and cognitive performance. To this end, two different approaches were compared: a traditional approach, based on the presentation of acoustic stimuli (Audio condition), and
170
an innovative multimodal approach based on the presentation of acoustic and visual stimuli by means of an Immersive Virtual Reality System (Audio+Video condition).
On the basis of the literature, we can hypothesize that the impact of noise on cognitive performances and subjective annoyance should be affected by the presence (Audio+Video condition) or absence (Audio condition) of congruent visual scenarios. Therefore, a main effect of Audio vs. Audio+Video condition is expected.
IV. METHOD
A. Participants
Fifty-seven university students (32 Females, 25 Males; age M = 22.9, SD = 2.8) participated into the study. Participants were matched by age and noise sensitivity [7] and randomly assigned to one of two experimental conditions: Audio+Video (n = 29) and Audio (n = 28).
B. Setting and Immersive Virtual Reality Equipment
The experimental session took place in the laboratory of Cognitive Science and Immersive Virtual Reality of the Second University of Naples. The Immersive Virtual Reality laboratory (IVR) is settled in a rectangular room (4.9 × 3.6 × 3.1 m) that allows for extensive movements while participants are connected to the tools of virtualization. It includes a work station linked to the 3-D Vizard Development 2009 Edition Virtual Reality Toolkit Devices of the Integrated VR Setups System. Virtual environments were presented through a nVisor SX (from NVIS, Reston, VA) head mounted display (HMD). The HMD presented stereoscopic images at 1280 × 1024 resolution, refreshed at 60 Hz. The virtual scenario spanned 60 degrees horizontally by 38 degrees vertically. Graphics were rendered by a Intel R core (TM) 2 Quad 9300 2.50 GHz and 1.98 GHz processor with a Nvidia GeForce 8800 graphics card using Vizard software (WorldViz, Santa Barbara, CA). Head orientation was tracked using a three-axis orientation sensor (InertiaCube3 from Intersense, Bedford, MA) and head position was tracked using a passive optical tracking system (Precision Position Tracker, PPT H4 from WorldViz, Santa Barbara, CA). Graphics displayed in the HMD were updated based on sensed position and Running head: Multisensory assessment of acoustic comfort aboard metros orientation of the participant’s head. Two loudspeakers were used to integrate auditory information with the virtual environment.
C. Materials and Measures
Auditory materials. Auditory stimuli were recorded in a wind farm at a distance of 20 m from a wind turbine. The soundtrack consisted of binaural audio signals (16 bit/44.1 kHz) recorded by a portable two-channel device "M-Audio Microtrack 24/96" and binaural headphones “Sennheiser Noise Gard HDC 451”. It was reproduced by means of two loudspeakers placed in front of participants.
IVR stimuli. In the Audio+Video condition, a 3D graphic virtual reality scenario of a wind farm was created. The graphic model was designed by means of the 3D modeling free software Google Sketch Up 7.0 simulating geometrical constructions according to actual dimensions, sizes and colors. Turbines used in wind farm simulation were three- bladed. The blades were colored light gray to blend in with the clouds and ranged in length from 20 to 40 meters (see Figure 1). The tubular steel towers ranged from 60 to 90 meters tall. The blades rotated at 10-22 revolutions per minute. On this scenario, the WordViz software virtual reality development interface allowed for simulating the rotation of blades.
Cognitive tasks. To evaluate the influence of wind farm noise on cognitive processes, three tasks exploring the following cognitive domains were chosen: short term verbal memory (Rey Visual Verbal Learning Test – Rey test), semantic memory (Verbal Fluency by letters test - VF) and executive control (Backward Counting- BC).
Subjective evaluation. Participants were asked to evaluate the degree of noise annoyance on a ten point Likert type scale from 1 (“not at all”) to 10 (“extremely”).
171
Fig. 1. Screen shot of “Wind Farm” virtual reality scenario.
D. Procedure
Audio+Video condition. Participants were led in a pre-marked starting position where they had to wear the HMD. Afterwards, they were immediately immersed in a virtual scenario reproducing a typical wind farm, facing the wind turbines. They stood at a distance of 20 m from the wind turbines and simultaneously heard to the wind turbines noise.
Audio condition. The procedure and materials were the same as in the Audio+Video condition. The only difference concerned the fact that no visual virtual scenario was presented and the items of the cognitive tasks were shown on a blank black screen.
Testing phase. In both conditions, participants had to perform the three cognitive tasks (Rey test, VF, and BC). The beginning and the ending of each cognitive task were indicated with a green and a red square, respectively. The inter- task interval lasted 5 seconds. The first green square appeared 10 sec after the beginning of the scenario. Immediately after, a cognitive task had to be performed. This procedure was repeated for each task, that is three times within each scenario. In the Rey test task the list of 15 words was visually presented at a rate of one per second (15 sec). After that, participants had to reproduce as many words as possible within 15 seconds until the red square appeared. As regards the BC, the starting number was visually presented and participants had to count backward aloud by seven within 20 sec. Finally in the VF task, the target letter appeared and participants had to generate as many words as possible within 30 seconds. The order of cognitive tasks was counterbalanced within each scenario and across participants. In this way any spurious effect deriving from sequence and order factors was prevented. In both Audio and Audio+Video conditions, at the end of each soundtrack participants were required to fill out a self-report questionnaire assessing their degree of noise annoyance.
V. RESULTS
B. To investigate the effect of the experimental conditions on cognitive performances and perceived annoyance, a MANOVA that treated the experimental condition (Audio and Audio+Video) as a 2-level between-subject factor and the cognitive performances or annoyance ratings as dependent variables was performed. Results showed a significant effect of experimental condition, Wilks’ lambda = .680, F(4, 52) = 6.119, p < .001, multivariate η2p = .320. As follow-up, four separate univariate ANOVAs with experimental
Condition as a 2-level between-subject factor (Audio+Video vs. Audio) were carried out on the mean correct responses of the cognitive tasks and on self-report annoyance.
C. As regards the cognitive tasks, results showed a significant difference between Audio and Audio+Video condition in Rey, F(1, 55) = 4.534, p < .05, η2p = .076, and VF task, F(1, 55) = 5.136, p < .05,
η2
p = .085. Participants’ performances were worse in the Audio (Rey, M = 4.43, SD = 1.2; FV, M = 7.07, SD =
2.1) than in the Audio+Video condition (Rey, M = 5.24, SD = 1.6; FV, M = 8.41, SD = 2.4). No significant difference was found for Backward Counting task, F(1, 55) = 1.77, p = .20, η2p = .03). As regards the
annoyance ratings, results showed that the presence of a visual scenario in combination with the acoustic pattern mitigated subjective noise annoyance. Indeed, participants reported higher degree of noise annoyance
172
in the Audio (M = 7.39, SD = 2.1) than in the Audio+Video condition (M = 5.62, SD = 2.3), F(1, 55) = 9.267, p < .01, η2p = .144. In sum these results indicate that participants in the Audio+Video conditions had better
cognitive performances and were less annoyed by wind farm noise with respect to participants in the Audio condition (see Figure 2).
D.
Fig. 2. Comparison between mean number of correct answers at the cognitive tasks and noise annoyance ratings.
Note: a) and b) comparison between mean number of correct answers in Audio vs. Audio+Video condition for Rey test and Verbal Fluency task (VF). Participants were more accurate in Audio+Video than in Audio condition. c) comparison between mean degree of perceived noise annoyance in Audio vs. Audio+Video condition. Participants were less annoyed in Audio+Video than Audio condition.
VI. CONCLUSIONS
Overall, the comparison between Audio and Audio+Video methodology shows that the presence of contextual visual information associated with wind farm noises influences the impact of noise on individuals. Indeed, participants rated wind turbine noise as less annoying in Audio+Video than Audio condition. This result is in line with previous literature showing that the presence of congruent and naturalistic scenario, as the one used in our experiment, can lower the degree of perceived noise annoyance [4]. Furthermore, it is interesting to notice that participants’ performance at the Rey test and at Verbal Fluency task was more accurate in Audio+Video with respect to Audio condition. We may speculate that the better cognitive performance in Audio+Video condition can be considered a consequence of the reduced perceived annoyance. However it is not easy to propose an exhaustive interpretation for the selective effect of noise on the three cognitive tasks used. To our knowledge, there are no published studies exploring the influence of visual features on cognitive tasks performed under noisy conditions. Studies that use unimodal audio methods are often contradictory. In some cases noise seems to produce a negative effect on cognitive tasks, whereas in others the effect disappears [for reviews see 8; 9]. This could be due to the presence of many factors such as characteristics of the acoustic parameters, cognitive demands of the tasks, environmental features and personality traits [8; 7]. Therefore, more studies are needed to better understand how the above mentioned factors may modulate the effects of noise on humans.
From a theoretical point of view, the overall findings support the idea that humans perceive the environment holistically. In turn, they are consistent with the literature showing that the impact of noise does not rely exclusively on auditory information but is influenced by the presence of contextual visual features [4]. Our brain processes environmental information in a multisensory way and combines the different sources of modal features into complex and unitary representations [10].
From a practical point of view, this would imply that acoustic comfort assessment methods should be more biologically plausible, i.e. should take into account the multisensory way in which sounds are processed in real life. In this perspective, the Immersive Virtual Reality technology could offer a more appropriate way to assess the impact of environmental noise on humans. For instance, it allows the reproduction of realistic and vivid embodied experiences where people can actively see, hear and feel the experience as if it were their own, while maintaining experimental control over the variables of interest.
173
REFERENCES
[1] L. W. Barsalou, Grounded Cognition, Annual Review of Psychology, 59, 617–45, 2008
[2] V. Gallese, Embodied simulation: from neurons to phenomenal experience, Phenomenology and the Cognitive Sciences, 4, 23-48, 2005 [3] M. Wilson, Six Views of Embodied Cognition, Psychonomic Bulletin & Review, 9(4), 625–636, 2002
[4] S. Viollon, C. Lavandier, & C. Drake , Influence of visual setting on sound ratings in an urban environment, Applied Acoustics, 63, 493-511, 2002
[5] Z. Bangjun, S. Lili, D. Guoqing, The influence of the visibility of the source on the subjective annoyance due to its noise, Applied Acoustics, 64, 1205–1215, 2003
[6] H. Fastl, Psycho-Acoustics and sound quality, In J. Blauert (Ed.), Communication Acoustics (Chapter 6), Berlin: Springer, 2005
[7] V.P. Senese, F. Ruotolo, G. Ruggiero, & T. Iachini, The Italian version of the Weinstein Noise Sensitivity Scale: Measurement Invariance
Across Age, Gender, and Context. European Journal of Psychological Assessment, in press
[8] G. Belojevic, B. Jakovljevic, & V. Slepcevic, , Noise and mental performance: personality attributes and noise sensitivity, Noise & Health, 6(21), 77-89, 2003
[9] G. Belojevic, E. Öhrström, & R. Rylander, Effects of noise on mental performance with regard to subjective noise sensitivity, International Archives of Occupational and Environmental Health, 64, 293-301, 1992