Neuron, Volume97
Supplemental Information
Supralinear and Supramodal Integration
of Visual and Tactile Signals in Rats:
Psychophysics and Neuronal Mechanisms
A B G F I H C D E A stages 2 - 5 stage 6 traning day
traning day modality
1 6 11 16 21 26 31 36 % correct % correct 40 50 60 70 80 90 100 rats 7 8 9 10 angle 0 45 90
proportion called vertical
0 0.2 0.4 0.6 0.8 1 B C D E K K K G G G G G G G G G G G G G C C C C C C C J J J J C K K K K K K K K K K 0.25 0.5 1 2 4 8 16 32 64 128 -10 0 10 20 30 40 50 60 70 80 frequency (kHz) SPL(dB)
measured acoustic signal reference values
Figure S1: Details on training and testing, related to Figures 1 and 2.
fol-lowing components. A: the main chamber, B: stimulus and stepper motor controller together with digital step counter, C: the reward delivery area and licking sensors, D and E: transparent and opaque panels respectively, F and G: the pumps which drive the syringes loaded with diluted juice, H: the electronic control box, housing the microcontroller-based D/A boards and sensors and lights controllers, I: the high-power infrared light source attached to optic fibers. (B) Acoustic frequency spectrum of
stim-ulus motor. Comparison of the noise of the stimstim-ulus motor with previously reported hearing threshold of rats shows sound cues were below rats’ acoustic sensitivity. Black lines, the hearing threshold of rats according to different references. Blue line, fre-quency content of the stimulus motor noise. K: (Kelly and Masterton, 1977), C: (Cowles and Pennington, 1943), J: (Jamison, 1951), G: Lower frequency range: (Gourevitch, 1965), Higher frequency range: (Gourevitch and Hack, 1966). (C) The performance
through training of a representative rat (number 6). The figure plots the rat’s learning to discriminate 0 and 90 degrees. See Methods for the stages of training. The plot on the right shows the performance across the first 3 days of Stage 6 in separate plots for each modality condition (D) Learning curve during random modality training.
Learn-ing the task in random modality trainLearn-ing was more difficult for the rats compared with the canonical protocol that began with VT. Four rats were trained using this protocol. The curve shows the performance in stages 2-5, with the difference that rats were pre-sented with interleaved modalities from the beginning of their training (from stage 2).
(E) Improvement in a representative rat’s performance (rat 7), is evident from a
grad-ual increase in the slope of the psychometric curve (arrow mark) from first sessions (light colored) to later sessions (darker colored) as well as change in the PSE (less bias in later sessions). Each curve is fitted on the data of 3 consecutive days with 1 overlapping day.
A 1 2 3 4 5 6 7 8 9 10 rat 50 60 70 80 90 100
cumulative session performance
0.49 0.51 0.40 0.38 0.29 0.30 0.45 0.14 0.48 0.49 performance in 90-180 deg performance in 0-90 deg B D C E F G 0 45 90 angle 0 0.2 0.4 0.6 0.8 1
proportion called vertical
H 0 0.2 0.4 max slope 0 0.2 0.4 0.6 normalized count T V VT V vs T p=0.000 VT vs T p=0.000 VT vs V p=0.000 0 0.2 0.4 lapse rate 0 0.2 0.4 0.6 0.8
average across rats
V vs T p=0.000 VT vs T p=0.000 VT vs V p=0.000
35 40 45 50 55
point of subjective equality (deg) 0 0.05 0.1 0.15 V vs T p=0.374 VT vs T p=0.010 VT vs V p=0.028
normalized sigma difference
-0.5 0 0.5 1
nornalized information difference -0.5 0 0.5 1 R = 0.6 response time (ms) 400 200 600 800 1000 1200 75th percentile: 734 ms median: 565 ms 25th percentile: 435 ms σT or σV 12 16 20 24 28 σVT 12 16 20 24 28 σVT vs σT σVT vs σV X=Y 1 2 3 4 5 6 7 8 9 10 11 12 400 600 800 1000 1200 1 2 3 4 5 6 7 8 9 10 11 12 response time (ms) 400 600 800 1000 1200 rat number 1 2 3 4 5 6 7 8 9 10 11 12 400 600 800 1000 1200 tactile visual visual-tactile response time (ms) 500 1000 1500 2000 normalized count 0 0.005 0.01 0.015 0.02 0 45 90 Angle 550 600 650 700 750 Response time (ms) T V VT
Figure S2: Details on the analysis of behavioral data, related to Figure
2.
lapse rate (middle) and point of subjective equality (right) values based on 1,000 rounds of resampling in each modality for all rats averaged together. For all three measures, the modality-specific distributions are visibly distinct from each other. In the VT condi-tion: the slopes are much larger than V-only or T-only, the lapse rates are lower and the points of subjective equality closer to 45 degrees, all indicating multisensory en-hancement. Numbers represent p-values for each test of statistical significance. (B)
Individual rats performed best on bimodal trials, independently of the individual’s pre-ferred modality, as indicated by the decrease in σ in the VT versus V and T conditions. Error bars indicate 95% confidence intervals by bootstrapping. (C) Cumulative
per-formance of 10 rats who were well-trained with stimuli in the range of 0-90 degrees as compared with cumulative performance on trials with stimulus orientations ranging from 90-135 and -45 to 0 degrees (Figure 1B, striped sectors) interleaved with the familiar 0-90 degree trials in the same session. In the first such session, each rat per-formed equally well on the new (black points) versus the familiar stimuli (red points). Numbers indicate p-values for each rat (bootstrap test). Error bars show 95% binomial confidence intervals.(D) Bayesian prediction of underlying threshold (σ, abscissa) and
Mutual Information (ordinate) both suggest that in the majority of rats the signals car-ried by the two sensory channels were not merged independently but were combined in a synergistic manner. Data points in this plot show the normalized difference be-tween the predicted and bimodal σ (x-axis) and the difference bebe-tween sum of V and T information and VT information for each rat. The two measures are larger than zero for most rats and are correlated, linear regression R=0.6. (E) Box plot including the
interquartile range for response times considering all sensory conditions and stimu-lus orientations for all rats. Whiskers indicate variability outside the upper and lower quartiles. (F) Box plot of response times, separately for each rat in each modality.
Circles; median, rectangles; upper and lower quartiles, dots; outliers. (G) Upper:
Re-sponse times averaged across all test sessions for all rats. Lower: Data from V, T, and VT trials are plotted separately. Data show mean±SEM. Response time distributions for all rats in each modality separately (bottom). (H) Performance on catch trials was
not significantly different from the expected chance performance of 50%. Data show mean±SEM.
A C B ID: 200215112702 0 0.05 ID: 20417212002 0 0.02 0.04 ID: 210215111703 0 0.01 ID: 220215111503 0 0.02 ID: 190215111304 0 0.005 0.01 ID: 121214102202 0 0.05 ID: 190215111203 0 0.01 0.02 ID: 20417212502 0 0.01 ID: 310515120602 0 0.05 ID: 200215111803 0 0.05 ID: 200215112402 0 0.05 ID: 190215110402 0 0.01 ID: 200215111103 0 0.005 0.01 ID: 131214102703 0 0.01 0.02 ID: 200215112905 0 5 10-3 ID: 210215113002 0 0.05 ID: 190215112302 0 0.05 0.1 ID: 210215111103 0 0.02 0.04 ID: 121214100302 0 0.01 ID: 200215110403 0 0.02 0.04 ID: 131214101302 0 0.01 0.02 ID: 290515120603 0 0.01 ID: 121214101104 0 0.005 0.01 ID: 190215111305 0 0.01 0.02 ID: 40215103102 0 0.02 0.04 ID: 190215112702 0 0.02 ID: 220215110502 0 0.02 ID: 40215103202 0 0.02 0.04 ID: 190215112402 0 0.05 ID: 121214100102 0 0.01 0.02 ID: 290515121403 0 0.005 0.01 ID: 200215112904 0 0.01 0.02 ID: 131214102202 0 0.005 0.01 ID: 20417213102 0 0.02 ID: 210215112703 0 0.01 0.02 ID: 190215111302 0 0.05 ID: 190215110602 0 0.02 ID: 131214102002 0 0.05 ID: 190215110304 0 0.01 0.02 ID: 210215112003 0 0.02 0.04 ID: 220215112606 0 0.005 0.01 ID: 121214101803 0 0.02 0.04 ID: 121214102803 0 5 10-3 ID: 131214102302 0 0.01 0.02 ID: 131214101503 0 0.01 0.02 ID: 190215110703 0 0.005 0.01 ID: 200215112004 0 0.02 ID: 190215110502 0 0.05 ID: 121214100402 0 0.01 0.02 ID: 210215112202 0 0.02 0.04 ID: 20417211702 0 0.01 0.02 ID: 290515121102 0 0.01 0.02 ID: 200215111903 0 0.01 0.02 ID: 210215112402 0 0.02 ID: 220215112502 0 0.05 ID: 20417210202 0 0.05 ID: 40215100402 0 0.05 ID: 220215110202 0 0.05 ID: 310515121302 0 0.05 ID: 121214100803 0 0.01 0.02 ID: 210215111102 0 0.05 ID: 121214102902 0 0.05 ID: 131214101803 0 0.005 0.01 ID: 121214100602 0 0.05 ID: 40215102103 0 0.005 0.01 ID: 200215110602 0 0.05 ID: 20417211102 0 0.005 0.01 ID: 40215100802 0 0.02 0.04 ID: 121214102703 0 0.02 0.04 ID: 131214100202 0 0.01 0.02 ID: 20417212602 0 0.01 ID: 210215112305 0 0.01 ID: 190215110702 0 0.05 ID: 210215112702 0 0.05 0 0.02 ID: 220215111502 0 50 100 ISI (ms) 0 0.05 0.1 Probability Probability Probability Probability Probability Probability Probability Probability Probability Probability Probability Probability ID: 310515121502 0 50 100 ISI (ms) 0 0.01 0.02 ID: 200215111402 0 50 100 ISI (ms) 0 0.02 ID: 200215110502 0 50 100 ISI (ms) 0 0.05 ID: 190215112602 0 50 100 ISI (ms) 0 0.05 ID: 220215111602 0 50 100 ISI (ms) 0 0.02 ID: 20417210203 0 50 100 ISI (ms) 0 0.005 0.01 ID: 220215111702 0 50 100 ISI (ms) 0 0.02 ID: 210215111802 0 0.05 ID: 40215102302 0 0.02 0.04 ID: 40215100302 0 0.02 0.04 ID: 121214101102 0 0.01 0.02 ID: 131214102902 0 0.05 ID: 210215111803 0 0.02 0 0.05 ID: 20417210802 ID: 310515120903 0 0.02 ID: 121214101502 0 0.05 ID: 290515120803 0 0.01 0.02 ID: 200215111902 0 0.05 ID: 220215112003 0 0.01 0.02 ID: 210215112902 0 50 100 0 0.05 0.1 ID: 121214101902
Figure S3: Implants for electrophysiology, related to Figure 3 and START
Methods on acquisition of neuronal data.
Two different kinds of microwire array configurations used in the electrophysiological investigations. (A) TDT Fixed Microwire arrays (B) CAD rendering of cross-section of
a custom made 3D-printed microdrive for movable microwires. (A, adopted from TDT, B, courtesy of Marco Gigante, SISSA). (C) Inter-spike interval (ISI) histograms for 96
randomly selected neurons.
A
C
B
-1 -0.5 0 0.5
time from first lick (s) 0 0.1 0.2 0.3 0.4 0.5
proportion of selective neurons
modality selective (p<0.01) category selective (p<0.01)
-1 -0.5 0 0.5
time from first lick (s) -1
-0.5 0 0.5 1
category selectivity index
-1 -0.5 0 0.5
time from first lick (s) -1
-0.5 0 0.5 1
modality selectivity index
Figure S4: Selectivity analysis on a population of 622 PPC neurons,
related to Figure 4.
(A) Time course of category selectivity index measured for each neuron (black lines)
from a window of 200 ms neuronal activity with shifting window of 50 ms. (B) Same
analysis as in A for modality selectivity. (C) Time course of proportion of neurons that
show significant (p<0.01, permutation test) category (black) or modality (red) selec-tivity. Taken together, all results indicate poor modality selectivity but strong category selectivity.
B
angle
0 45 90
proportion called/classified vertical 0 0.2 0.4 0.6 0.8 1 behavior classifier 0 45 90 0 0.2 0.4 0.6 0.8 1 behavior classifier 0 45 90 0 0.2 0.4 0.6 0.8 1 behavior classifier A 20 40 60 -1 0 1 day 1 20 40 60 -1 0 1 day 2 20 40 60 -1 0 1 day 3 20 40 60 80 -1 0 1 day 4 20 40 60 80 -1 0 1 day 5 number of neurons 20 40 60 80
neural weights a.u.
neural weights a.u.
-1 0 1 day 6 10 30 50 -1 0 1 day 7 10 20 30 -1 0 1 day 8 10 20 30 -1 0 1 day 9 20 40 60 -1 0 1 day 10 0 45 90 angle 0 0.2 0.4 0.6 0.8 1
proportion called/classified vertical
behavior T behavior V behavior VT classifier T classifier V classifier VT C
(A) Plots are classifier weight associated with each neuron for separate recording
ses-sions (10 days), ordered by magnitude. Almost all neuronal weights were nonzero in each session, indicating that every neuron in the population contributed to the classi-fier’s performance. (B) Psychometric curves based on decoding neuronal responses
(solid lines) compared to the rats’ observed psychometric curves (dashed lines) in each tested modality condition (colors) for all 10 sessions separately. (C) The signal carried
by the PPC population also closely paralleled behavior when the classifier training set included the modality of the test trial. Data are presented as mean±SEM.
Movie S1: Hi-speed video recording from a tactile, visual and visual-tactile trial of a
representative rat, related to Figure 1. Video (slowed 16.5x) showing three example trials of a rat in V, T and VT conditions respectively.
