Chapter 5
Stimulated Emission Depletion Image
Scanning Microscopy
5.1 Stimulated emission depletion microscopy
The vast family of fluorescence optical microscopy techniques stands as an invaluable tool for addressing various biological questions, allowing to observe selectively and often for an extended time a given mechanism of interest, also directly in living cells or tissues. However, the performances of such techniques are hindered by the fundamental limit imposed by the diffraction of light (Abbe (1904)): the maximum achievable resolution, is ⇡ 230nm on the focal plane for visible light (l = 650 nm) and high numerical aperture objective lens (NA = 1.4) (see 3.1). Confocal laser scanning microscopy (see Chapter 3) theoretically overcomes this limitation, but only of a factor ofp2 factor.
In the last ten years, new fluorescence microscopy techniques, collectively termed super-resolved microscopy or optical nanoscopy (Hell et al. (2015)), have succeeded in improving the resolution into the double digit nanometer range - more relevant for many biological ques-tions - by avoiding the simultaneous signaling of molecules closer than the diffraction limit. Stimulated emission depletion microscopy (STED, Hell and Wichmann (1994), Vicidomini et al. (2018)), one of the first of such techniques, achieves super-resolution by leveraging the stimulated emission process, within the framework of a fluorescence laser scanning microscope. The stimulated emission phenomenum describes how a properly-energetic stimulating photon may interact with an excited fluorophore, inducing it to relax to the ground state with the emission of another photon, which is "identical" to the stimulating one. STED microscopy relies on an additional laser beam (the STED beam) which focal intensity distribution is shaped as a doughnut, to induce stimulated emission of all the fluorophores
5.1 Stimulated emission depletion microscopy 49 lying in the periphery of the excitation spot; in other words, only molecules situated in the centre of the conventional Gaussian excitation region can emit fluorescence. By properly choosing the stimulating wavelength, the stimulated photons emitted from the periphery can be spectrally separated by the fluorescence signal originated from the center of the excitation spot, thus leading to fluorescence images with much improved spatial resolution (Fig. 5.1). Notably, the higher the STED beam intensity, the smaller the effective fluorescent region, and ultimately the better the spatial resolution.
The relation between the intensity of the STED depletion beam and the achieved spatial resolution can be obtained analytically. Consider a STED microscope configuration for which both excitation and STED laser beams are pulsed at the same frequency, and the STED pulse - of durationD - immediately follows the excitation pulse. The photo-physical phenomena interesting the fluorescent molecules can be simplistically described with a system of 3 energetic levels (Fig. 5.1a). Considering the time dependence of the normalized population N1of the first excited state S1:
dN1
dt = (rST ED+rspEm)N1(t), (5.1) where rST ED=sST EDIST EDis the rate for stimulated emission,sST EDis the cross-section
for stimulated emission and IST EDis the intensity of the STED beam; rspEmis conversely the
rate for spontaneous fluorescence emission, the inverse of the fluorescence lifetimet. In the initial condition N1(0) = 1, if t = 0 is the onset of the STED pulse, the solution of Eq. 5.1 is:
N1(t) = e (rST ED+rspEm)t. (5.2)
It is convenient to define a suppression factor, which accounts for the reduced fluorescence signal caused by the depletion beam:
h(IST ED) = N1N(D)I=IST ED 1(D)I=0 =e
rST ED(D)=e sST EDIST EDD. (5.3)
In order to derive the resolution improvement provided by the depletion effect, we must consider the intensity profile of the STED beam, which we may approximate - close to the central peak of the excitation focus - to a parabola: IST ED(x) = 4IST EDmaxa2x2. Here, it
is convenient to express the resolution of the STED microscope leveraging the PSF of a confocal laser point scanning system, kCLSM (3), that we may approximate to a Gaussian
distribution with FW HM = dCLSM. Indeed, since these two architectures differ only in the
illumination PSF, we can obtain the effective PSF of a STED system kST ED by imposing the
5.2 Stimulated emission depletion image scanning microscopy 50
kST ED(x) = kCLSM(x)h IST EDmax,x = e 4ln2x2
d2CLSMe 4ln2a2x2ISTEDmaxIS
. (5.4) where IS=ln2/(sST EDD) is the saturation intensity, i.e., the intensity at which half
of the molecules are quenched to the ground state by the depletion beam: h(IS) =1/2 =
exp( ln2sST EDISD). Under the previous approximations, kST ED is again a Gaussian
distri-bution. Its FWHM - corresponding to the effective resolution of the STED microscope - is as follows:
dST ED= q dCLSM
1 + d2
CLSMa2 IST EDmaxIS
. (5.5)
According to this formulation, the effective resolution of a STED microscope is mainly influenced by the saturation intensity ISof the fluorophore and by the intensity of the STED
beam IST EDmax.
5.2 Stimulated emission depletion image scanning microscopy
In the previous sub-chapter I briefly introduced Stimulated Emission Depletion microscopy, one of the first and most widely utilised nanoscopy techniques. As described, this approach relies on the additional STED laser beam to confine the fluorescence signal only in the very centre of a conventional Gaussian excitation region. Notably, the higher the STED beam intensity, the smaller the effective fluorescent region, and ultimately the better the spatial resolution. Whereas STED microscopy attains single-digit nanometer resolution for some particularly tailored implementations (Giske (2007)), its practical performance is hindered by the non-negligible chance of photo-damaging the sample with the high-intensity STED beam. Despite several strategies to mitigate this problem have been proposed, such as time-resolved STED (Lanzanò et al. (2015); Vicidomini et al. (2011)) microscopy, adaptive-smart scanning schemes (Goettfert et al. (2017); Heine et al. (2017); Staudt et al. (2011), and far-red illumination (Kilian et al. (2018)), it still holds as a significant concern when one performs a live-cell or long-term measurements. Therefore, the scientific community is still looking for strategies to reduce the STED beam intensity needed to achieve a target resolution.
Image scanning microscopy (ISM) stands as a promising candidate to meet this need: as discussed in Chapter 3, this technique boosts the spatial optical resolution and the signal-to-noise Ratio (SNR) of confocal microscopy, and can potentially provide a similar enhancement in other microscopy approaches, e.g. two-photon excitation microscopy or STED microscopy.
5.2 Stimulated emission depletion image scanning microscopy 51
Figure 5.1 The STED principle. a) Fluorescence and stimulated emission phenomena. Fluorescence: a fluorescent molecule can be excited from the ground state S0 to a higher
vibrionic level Svib1 by an appropriate photon. In a time range in the order of ps the molecule decays non-radiatively to the lowest energy level of S1. The excited molecule later relaxes to
the ground state via the spontaneous emission of a fluorescence photon. Stimulated emission: if the molecule is in the excited state, a stimulating photon that matches in energy the gap between S1and Svib0 may induce the fluorophore to relax to the ground state, emitting a photon
in the process that is identical to the stimulating one. b) Cartoon of confocal and STED imaging. In STED, the depletion beam is superimposed to the excitation beam, causing only the fluorophores situated in a sub-diffraction spot to emit via spontaneous emission, and thus improving the spatial resolution. The super-resolved image is obtained scanning the two beams on the the sample. c) Comparison between confocal and STED imaging of tubulin structures in fixed HeLa cells.
I have previously presented our ISM implementation that relies the SPAD array detector, instead of the typical single-element detector. A single scan results consequently in a set of ”similar” images, the so called scanned images, that are in principle only shifted with respect to the reference image generated by the central detector of the array. The simple pixel-reassignment (PR) operation, i.e., shifting back and summing all the images up, grants the spatial resolution and the SNR boost of the final ISM reconstruction. Here I discuss how this enhancement, in the context of STED microscopy, allows to reduce the STED power
5.3 Material and methods 52 needed to achieve a target resolution. Notably, the proposed combination of STED and image-scanning microscopy (i) requires only minimal changes in the conventional STED microscopy architecture: an extra magnification on the image plane and the introduction of the SPAD array instead of the single-element detector, (ii) is fully compatible with all the approaches for photo-damage reduction mentioned above, and (iii) preserves all the other functions of STED microscopy, such as multi-color, three-dimensional, and fast imaging. The correct amount of shift – the shift vector – imposed to a given scanned image for the PR reconstruction method is key for a successful resolution and SNR enhancement. In first approximation, and in absence of stimulating photons, i.e., no STED imaging, the shift-vectors are theoretically related to the physical geometry of the detector array, and their modules equal half of the distance between the given sensitive element of the detector array and the central one, projected onto the sample plane, as described previously (see Chapter 3); however, the shift-vectors are practically influenced by the spectral properties of the fluorophores (De Luca et al. (2013)), and by any sample- or system-induced aberration or misalignment of the optical system. Additionally, in the case of STED microscopy, the shift-vectors show a strong dependence on the STED beam intensity: the higher the photon dosage, the smaller the effective fluorescent spot, and the shorter the shift-vectors. In essence, the overall PSF of a STED microscope in the case of high STED intensities depends mainly on the excitation PSF, whilst the influence of the emission PSF becomes negligible, and thus the scanned images differ each other in SNR but not anymore in position. With this insight, we show through simulations (Fig. 5.2) that the pixel reassignment operation is beneficial for STED microscopy for a given range of STED intensities; more precisely, the spatial resolution and the SNR of the STED-ISM reconstruction are enhanced with respect to the raw STED counterpart, for STED intensities below a given threshold. Increasing the STED beam power more leads to very short shift-vectors and ultimately to negligible benefits from the pixel reassignment operation.
5.3 Material and methods
5.3.1 Setup
To achieve the STED-ISM implementation, we updated the ISM setup described in Chapter 3 with a STED beam laser line (Fig. 5.3). Briefly, the STED beam was provided by a femtosecond mode-locked Ti:Sapphire laser (Chameleon Ultra II, Coherent) running at 775 nm. We coupled the laser beam into a 100 m long polarization maintaining fiber (PMF).
5.3 Material and methods 53
Figure 5.2Simulation of a STED-ISM experiment with increasing saturation factor z. a, Each element of the SPAD array is characterised by a PSF which is, in first approximation, shifted with respect to the central one. The 25 PSFs are represented in the top row, normalized to themselves (top left), or to the central one (bottom right). Each PSF is shifted back, accordingly to the values retrieved by our phase correlation algorithm (bottom row), and the summed up to obtain the STED-ISM PSF (central row, right). The raw STED PSF is alternatively obtained by ignoring the shifts and only summing all the PSFs up (central row, left, normalized and non-normalized versions reported). FWHM values are also reported. As the saturation factorz increases (from left to right) and consequently the effective excitation PSF shrinks, the shift values reduce and the benefits of STED-ISM with respect to the raw STED counterpart become less and less evident. b, STED-ISM grants a resolution improvement (top, FWHM analysis) as well as a SNR increase (center) ifz is lower than a given value. Bottom, the average of the four shortest shift values is decreasing as a function ofz. Excitation, emission and STED wavelengths: 640, 670 and 775 nm respectively; STED repetition rate: 80 MHz; STED pulse length: 200 ps; SPAD array semi-length: 1.4 Airy Units.
Before injection into the PMF, the beam passed through two 20 cm long glass rods to temporally stretch the pulse-width to few picoseconds in order to avoid unwanted nonlinear effects and damages during the fiber coupling. We used a half-wave plate (HWP) to adjust beam polarization parallel to the fast axis of the PMF. The combination of glass rods and PMF stretched the pulses of the STED beam to ⇡ 200 ps. We controlled the power of the STED and excitation beams thanks to two acoustic optical modulators (AOM, MCQ80-A1,5-IR and
5.3 Material and methods 54
Figure 5.3The STED-ISM setup. HWP:half-waveplate;AOM:acousto- optic modulator; AD:achromatic doublet; PMF: polarized-maintaining fiber; FI: Faraday isolator; GR: glass-road; 3A-PS: three-axis piezo stage; SL: scan- ning lens; GNs: galvanometric mirrors; DM: dichroic mirror; QWP: quarter- wave plate; BPF: band pass filter; MMF: multi-modes fiber; APD: avalanche photo-diode; PM: phase-mask; GTP: Glan-Thompson polarizer; CUP: clean-up filter. The asterisk denotes the plane conjugate to the image plane. The double asterisks denote the plane conjugate to the objective back-aperture.
MT80-A1-VIS, respectively, AAopto-electronic). The Ti:Sapphire laser (master) runs at 80 MHz and provides an electronic reference signal which we used to synchronize electronically the excitation laser diode (slave). We used a picosecond electronic delayer (Picosecond Delayer, Micro Photon Devices) to temporally align the excitation pulses with respect to the depletion pulses. The STED beam emerging from the PMF is collimated, filtered in polarization by a rotating GlanThompson polarizing prism and phase-engineered though a polymeric mask imprinting (0,2p) helical phase-ramps (VPP-1a, RPC Photonics). We rotated a quarter-wave plate and a half-wave-plate to obtain circular polarization of the STED beam at the back-aperture of the objective lens. We co-aligned the excitation and STED beam using two dichroic mirrors (T750SPXRXT and H643LPXR, AHF Analysentechnik). After combination, the excitation and STED beams were deflected by two galvanometric scanning mirrors (6215HM40B, CT Cambridge Technology) and directed toward the objective lens (CFI Plan Apo VC 60×, 1.4 NA, Oil, Nikon) by the same set of scan and tube lenses used in a commercial scanning microscope (Confocal C2, Nikon). The fluorescence light path and the acquisition hardware and architecture are not changed from what described in Chapter 3.
5.4 Results 55
Figure 5.4STED-ISM imaging of 20 nm sized fluorescent beads with increasing STED power. The visual inspection of the details (top) suggests the superior performance of STED-ISM, confirmed by the FRC analysis (right). Dwell time: 50µs; pixel size: 20 nm; STED powers: 0, 13, 50, 160 mW; details format: 150 x 150 pixels.
5.4 Results
5.4.1 Calibration samples and fixed cells
We present various experiments to validate our STED-ISM implementation with a SPAD array. We started imaging 20 nm diameter fluorescent beads with increasing STED intensities (Fig. 5.4): as expected from simulations (Fig. 5.2), the STED- ISM image shows better SNR when compared to the raw STED counterpart, obtained summing all the 25 channels up. The spatial resolution, again measured using Fourier Ring Correlation (see Appendix A), is also enhanced within a given STED intensity range. We confirmed these results by performing STED-ISM measurements of a sample of tubulin-labelled fixed Hela cells (Fig. 5.5). It is important to note here that the successful STED-ISM reconstruction relies heavily on the blind, parameter-free phase-correlation algorithm that we introduced previously (see Chapter 3), capable of retrieving the pixel reassignment shift-vectors directly from the scanned images. Although in general this strategy compensates for eventual aberrations and misalignments of the optical system, in the context of STED-ISM it also accommodates implicitly the strong dependency of the shift-vectors on the STED intensity. Conversely, other all-optical ISM implementations – both single-spot (De Luca et al. (2013); Gregor et al. (2017); Roth et al. (2013)) or eventually multi-spots (Azuma and Kei (2015); York et al. (2013)) architectures – should rely on proper modelling or prior calibration of the STED microscope’s effective PSF, as a function of the STED beam intensity and of the general experimental conditions.
5.4 Results 56
Figure 5.5STED-ISM measurements of fixed cells with increasing STED power. The resolution and the SNR of are enhanced by STED-ISM when imaging tubulin-labelled fixed Hela cells. Retrieved FRC resolution values: 278 and 189 nm, 202 and 180 nm, 192 and 177 nm, 136 and 136 nm, for raw STED and STED-ISM, respectively. Dwell time: 100µs; pixel size: 40 nm; STED powers: 0, 15, 23, 137 mW; format: 500 x 500 pixels; details format: 150 x 150 pixels; all scale-bars: 1µm.
5.4.2 Live-cells STED-ISM time-lapses
To investigate the concrete advantages of STED-ISM over STED microscopy, we explored the conditions for which one is usually concerned about the maximum amount of stimulating photons delivered to the sample: live cell imaging (Fig. 5.6). Also in this case, we report a SNR and spatial resolution enhancement of the resulting STED-ISM images with respect to the raw STED counterparts. The result is further improved applying the multi-image
5.4 Results 57
Figure 5.6 STED-ISM imaging of living cells. Raw STED (top), STED-ISM and the multi-image deconvolution result STED-ISM+ (center) details of SIR tubulin labelled living
Hela cells with increasing STED powers (from left to right). The line profiles (bottom) proof the superior performances of STED-ISM with respect of the raw STED counterpart. Dwell time: 50 µs; pixel size: 40 nm; STED powers: 0, 10, 20, 45 mW; details format: 150 x 150 pixels.
deconvolution algorithm described previously, in this context completely parameter-free thanks to the PSFs estimation via FRC analysis (see Appendix A). Moreover, we were able to perform extended STED-ISM time lapses of live Hela cells without inducing any noticeable photo-bleaching effect, given the reduced STED intensity necessary to obtain the target resolution (Fig. 5.7).
5.4 Results 58
Figure 5.7STED-ISM time lapses. a, Two extended time lapses (20 frames, 500 and 300 s) of Sir-Tubulin labelled Hela cells. Different frames are reported to show the negligible photo-bleaching. b, The FRC analysis confirms, for both time-lapses, the superior performances of STED-ISM with respect to raw STED.c, Details from the time lapses, showing the image recorded by the central element of the SPAD array (raw central STED), the result of the summing operation (raw STED), the result of the pixel reassignment operation (STED-ISM) and the deconvolution image (STED-ISM+). Dwell time: 25µs; pixel size: 70 and 67 nm;
format size: 1000 x 1000 and 750 x 750 pixels; details format size: 200 x 200 pixels; 635 nm excitation laser power: 8µW; 775 nm STED laser power: 21 mW.
Chapter 6
Separation of Photons by Lifetime
Tuning for pSTED Microscopy
6.1 Time-resolved STED microscopy
Stimulated emission depletion (STED) microscopy was one of the firstly introduced super-resolution techniques (Hell and Wichmann (1994); Vicidomini et al. (2018)), as discussed in the previous Chapter. In a nutshell, the standard STED microscope co-aligns the conventional Gaussian excitation beam with the so-called STED beam, engineered to generate a donut-shaped intensity distribution at the focus, i.e., with a "zero"-intensity point in the center. The STED beam forces all fluorophores, except those in the "zero"-intensity point at the center of the excitation spot, to de-excite to the ground state via stimulated emission, thus reducing the region from which the fluorescence signal is emitted and then recorded. The resolution of STED microscopy can theoretically reach the molecular size by increasing the intensity of the STED beam; however, it is practically limited by other factors, such as the noise (Tortarolo et al. (2018)) and the amount of laser power that can be delivered to the sample to prevent photo-damaging effects (Laissue et al. (2017)). For the latter reason, much effort has been spent on reducing the peak power of the STED beam necessary to reach a given spatial resolution. I already shown how, by substituting the single element detector of a traditional STED setup with our SPAD array, it is possible to exploit the extra spatial information to reduce the STED intensity needed to achieve a target resolution, in a given range of STED powers (see Chapter 5, Results). Notably, another possible approach to achieve a similar result is to explore the temporal information related to the time-of-arrival of fluorescence photons. Historically, the turnkey insight has been the comprehension of the variations in fluorescence temporal dynamics induced by the STED beam itself. Indeed,
6.1 Time-resolved STED microscopy 60 the stimulated emission process opens for each fluorophore a new de-excitation pathway, which rate (instantaneous probability) strongly depends on the intensity of the STED beam. The STED beam intensity is spatially distributed as a donut, hence the fluorescence temporal decay of each fluorophore encodes its position with respect to the center of the excitation spot: the decay is slower at the center of the excitation spot, and faster at the donut crest. This understanding has been exploited to reduce the peak intensity of the STED beam needed to achieve a target resolution, enabling effective sub-diffraction resolution also in such conditions where the STED beam intensity is not sufficient to obtain a complete fluorescent depletion/quenching (Moffitt et al. (2011); Vicidomini et al. (2011, 2013)).
The first application of this principle is the so-called gated continuous-wave (CW) STED microscope (Vicidomini et al. (2014, 2011)). If the STED beam is implemented with a CW laser, the peak intensity reduces, and the effective resolution as well. However, the further is the fluorophore from the focal point, the shorter is its (perturbed) excited state lifetimetST ED
(i.e., the average time that the fluorophore spends into the excited state). Thus, by using pulsed excitation and a time-gated detection scheme - i.e., photons are collected only after a delay from the excitation events - the fluorescence at the periphery due to the incomplete depletion is removed, and the resolution improves. On the other hand, time-gating rejects also a portion of "wanted" photons from the center of the focal spot, resulting in a reduced signal-to-noise ratio (SNR) of the final image that may cancel out the resolution enhancement (Tortarolo et al. (2018); Vicidomini et al. (2013)). For implementations based on pulsed STED (pSTED) beams, the benefits of exploring the fluorescent temporal dynamics and of the time-gated detection depend on the pulse-width of the STED beam itself (Vicidomini et al. (2013)). Early pSTED microscopes used pulse-width below 200 ps, thus much shorter than the excited-state of typical organic fluorophores (tf l ⇠ 1-10 ns). This temporal condition
makes the fluorescence emitted during the action of the STED beam and the incomplete depletion negligible, thus the time-gated detection nearly useless (Vicidomini et al. (2012)). More recently it has been shown that pSTED microscopy based on sub-nanosecond (⇠ 600-1000 ps) pulsed lasers substantially reduces the photo-bleaching compared to early pSTED implementations (Castello et al. (2016)): photo-bleaching is supra-linear with the STED beam intensity (Dyba and Hell (2003); Oracz et al. (2017)). In this case, since the pulse-width is comparable with the excited-state lifetimetf l and the peak intensity reduces
(for a given average intensity of the pulse), the fluorescence emitted during the action of the STED beam is not anymore negligible and the benefit of time-gating is relevant. For these reasons, current pSTED microscopy implementations - including the commercial systems - rely on sub-nanosecond fiber laser and implement a time-gated detection (gated-pSTED
6.1 Time-resolved STED microscopy 61 microscopy). However, similarly to gated-CW-STED microscopy, also for gated-pSTED microscopy the SNR of the final image is reduced.
A-posteriori approaches, such as multi-image deconvolution (Castello et al. (2014)) and separation of photons by lifetime tuning (SPLIT, Lanzanò et al. (2015)), can solve this problem. In particular, the SPLIT method analyses the pixel fluorescent temporal decays within the phasor framework, and represents a straightforward approach to separate all the photons emitted from the long lifetime fluorophores located in the focal point ("wanted" photons), from the "unwanted" photons emitted from the short lifetime fluorophores located in the focal periphery. In this Chapter, we will extend the SPLIT method to pulsed STED microscopy (pSTED-SPLIT) architectures, showing how to recover the resolution hidden by the incomplete depletion/quenching, but without impacting the SNR.
To this aim, we leverage the phasor analysis, a powerful tool able to describe the evolution of a signal (as a function of a variable, such as the time) as a single point in a plane with coordinates g (the cosine transform) and s (the sine transform, Digman et al. (2008)). The phasor approach is performed on every pixel of a time-resolved STED measurement (e.g., on the histogram of the photon-arrival times associated to any pixel) to discriminate molecular species with different temporal fingerprints. In particular, the phasor analysis is used to separate the photons emitted by the fluorophores localised in the center of the excitation volume from the photons emitted by the fluorophores localised in the peripheral region, leveraging their different fluorescent decay dynamics. In brief, it can be used to shrink a posteriori the volume from which fluorescence signal is considered, thus improving the effective spatial resolution of the STED microscope.
Our initial implementation (Sun et al. (2017)) was successively improved by a successive work (Wang et al. (2018)), which used the phasor representation to generate a pixel segmented version of the raw STED data: the segmentation selects the pixels characterised by a slow fluorescence temporal dynamic, thus composed primarily by "wanted" photons, and discards the pixels identified by a fast fluorescence temporal dynamic, thus composed primarily by "unwanted" photons. On the contrary, the pSTEDSPLIT presented in this Chapter selects -from each pixel - only the "wanted" photons, rather than performing a simple binary pixel classification. In essence, the pSTED-SPLIT method that we propose - compared to the phasor-plot based segmentation methods - sorts photons and not pixels, thus providing more quantitative and artifact-free pSTED images. In the context of quantitative imaging, it is important to highlight that the SPLIT imaging technique preserves the linearity in the image, i.e., the pixel intensity values are linear with the fluorophore concentration (Lanzanò et al. (2015)).
6.2 Material and methods 62 In the following sub-chapters I will describe the details of our pSTED-SPLIT approach, and I will demonstrate - through simulations and experiments on fixed cells - the increased performances with respect to the RAW STED counterpart.
6.2 Material and methods
6.2.1 Setup
For the experiments reported in this chapter, we performed measurements using an ISS Alba confocal/STED laser scanning microscopy system coupled with a Nikon Te2000 microscope (Sun et al. (2017)). The excitation source is a 640-nm picosecond pulsed diode laser (Becker Hickl, BDL-SMN-640, ⇡120 ps pulse width). A 775-nm sub-nanosecond pulsed fiber laser (OneFive, Katana 775, ⇡600 ps pulse width) provides the STED beam. The two lasers are synchronised by either (a) using the depletion laser at the 40 MHz repetition rate (master) to trigger the excitation laser (slave); or (b) using the excitation laser at the 50 MHz repetition rate (master) to trigger the depletion laser (slave). Both lasers are also synchronised to the FastFLIM module to perform time-resolved STED measurements. The 640-nm excitation laser is mounted on the ISS 3-diode laser launcher to control its intensity, and then delivered to the Alba system via a single mode polarisation maintained fiber (QiOptics). The 775-nm STED laser intensity is controlled by the ISS intensity control unit consisting of a custom-made motorised rotating half-wave plate and a fixed Glan-Thompson polariser. We use an optical delay line (custom made) for the fine tuning (in the order of picosends) of the temporal delay between the STED pulses with respect to the excitation beam; the STED laser is then delivered via a single mode PMF (Thorlabs) to the STED beam conditioning module (custom made), to generate the donut-shaped profile of the depletion beam. Inside the Alba module, the excitation and the STED beams are combined using a 670 long-pass dichroic mirror (Semrock); we use a pair of galvanometric mirrors to scan both beams in the plane of the sample perpendicular to the optical axes. The objective is the highly numerical aperture Nikon Plan APOl 60X/1.4NA oil. The scanning device is synchronised to the data acquisition unit (FastFLIM by ISS) for time-resolved Digital Frequency Domanin (DFD) measurements. We use another dichroic mirror (custom made by Chroma) to separate the descanned fluorescence signal from the excitation and STED light. We filter the emitted fluorescence by both a 720 nm short-pass NIR light blocking filter (Optical Density 8, Chroma), and a 679/41 nm band-pass emission filter (Semrock). We set the motorised confocal pinhole (from ⇠20 µm to 1 mm) placed before the detector to 60 µm in diameter (⇠1 Airy unit) for STED
6.2 Material and methods 63 measurements. The detection unit is the single photon counting module avalanche photodiode (SPCM-ARQH-15 by Excelitas). Data acquisition is performed using the ISS VistaVision 64-bit software; phasor analysis is performed by a custom written Matlab code.
6.2.2 The pSTED-SPLIT approach
In this Section I will describe in details our pSTED-SPLIT approach based on the phasor analysis. We start by considering a single fluorophore and a CW-STED microscopy archi-tecture, i.e., the STED beam runs in CW. In the condition for which the gaussian-shaped excitation and the donut-shaped depletion beams are perfectly co-aligned (i.e., the maximum of the excitation profile coincides with the true "zero" of the depletion profile), a fluorophore in the very center of the excitation spot (r = 0) does not interact with stimulating photons from the STED beam: hence, it emits fluorescence according to its unperturbed lifetimetf l.
The decay dynamics of such fluorophore (thus, the histogram of the photon-arrival times) may be described with a single exponential exp( t/tf l)and corresponds to a single point
in the phasor space, that lies on the semicircle of radius 1/2 with center (1/2,0) (Fig. 6.1). On the contrary, if the fluorophore is located in the ring described by r = ˆr, it interacts with STED photons and it is likely to be quenched to the ground state. However, as a result of the not perfect efficiency of the stimulated emission process, it may emit fluorescence nonetheless, but following the faster single exponential decay law exp( t/tST ED(ˆr)), where
tST ED(ˆr)µ 1/IST ED(ˆr) and IST ED(r) is the spatial intensity distribution of the STED beam
(we suppose radial symmetry). The corresponding point in the phasor space is still laying on the same semicircle, but shifted toward higher g values; the limiting case of infinite STED intensity is the point (1,0) (Fig. 6.1).
The key difference between pSTED and CW-STED implementations resides on the temporal dynamics of the fluorescence photons from the peripheral region of the excitation spot: the intensity temporal decay of a fluorophore illuminated by the pulsed STED beam, that can be described with a piece-wise function, is faster during the STED pulse (0 t < TST ED), and
slower when the STED beam is off (t TST ED):
I(t)µ 8 < : exp t/tST ED if 0 t < TST ED exp TST ED/tST EDexp (t TST ED)/tf l if t TST ED (6.1)
As a result, a fluorophore in the periphery of the excitation spot is represented in the phasor space by a point that is lying inside of the semicircle, conversely to the CW-STED
6.2 Material and methods 64 implementation. In particular, this point is lying on a trajectory which again moves from the point associated to the unperturbed fluorophore (no stimulating photons) to the point (1,0), for increasing STED intensities. Notably, the shape of the trajectory in a pSTED system depends on the ratiog = TST ED/tf l, where TST ED is the pulse-width of the STED beam, and
tf l is the natural lifetime of the fluorophore. The two considered limit cases forg are (i) very
small pulse-width,g = 0.2: the trajectory is a chord in the semicircle (Fig. 6.1b, light blue trajectory); infinite large pulse-width,g >> 1 (equivalent to a CW-STED implementation): the trajectory is an arc of the semicircle (Fig. 6.1b, green trajectory).
In a real STED imaging experiment, the fluorescent signal registered by each pixel is the sum of the signals provided by all the fluorophores located within the detection region associated with the pixel. The phasor analysis isolates, from the fluorescent signal of each pixel, the component associated with the fluorophores located in the center of the excitation region, i.e., the slower components. It thus allows to reduce the effective detection region and therefore to improve the resolution, without increasing the STED intensity.
The dependencytST ED(r) yields to a continuous population of emitters with increasing
dis-tances from the center of the excitation spot, characterised with increasingly faster temporal decays. It is however convenient to approximate this population considering only two species: the emitters situated in the center of the excitation spot (species 1,P1in the phasor space);
and the emitters related to the periphery (species 2,P2in the phasor space), that originates the
undesired signal. Now, the problem becomes to isolate the component of the signal generated from the first species. As per the rules of phasors, a combination of such two species will be represented in the phasor space as a point lying on the lineP1P2. As described in the SPLIT
technique (Lanzanò et al. (2015)), the contributes from these two species can be separated via a linear decomposition approach, to extract only the signal originated by fluorophores in the center of the excitation spot.
We now consider a time-resolved measurement. The total number of collected photons N in a given pixel is the sum of photons N1 and N2, emitted by molecules in the center
and in the periphery of the effective excitation spot, respectively. As described before, the corresponding vector in the phasor space P can be described as the linear combination of the two vectors P1= (g1,s1) and P2= (g2,s2) describing the two pure lifetime species:
P = (N1P1+N1P2)/N = f1P1+ f2P2, where f1 and f2 are the fractional components of
the detected photons. We can write this linear system in the matrix form P = M f, where f = ( f1,f2), and M = (P1,P2)is the matrix containing the temporal dynamics of the two
species in the phasor domain. The solutionf = M 1P allows separating photons emitted by
6.3 Results 65 of peripheral molecules N2= f2N. The final image with higher resolution is obtained by
iterating this process for each pixel of the raw time-resolved image. Notably, the phasor analysis presented here is not restricted to the case of a single-exponential decay - as for the CW-STED implementation - but it is valid in general for each signal evolution, thus also for the signal generated by the fluorophores in the case of pSTED microscopy.
6.3 Results
6.3.1 pSTED-SPLIT simulations
We first applied the SPLIT method on the synthetic pSTED image of a single fluorophore. This simulation allows us also to derive the point-spread-function (PSF) of the pSTED-SPLIT system (Fig. 6.2a). In this context, it is important to remember that the pSTED-SPLIT-STED microscope is - in general - a linear and space-invariant system (Lau et al. (2012)), thus its PSF fully describes the imaging characteristics of the system: e.g., the spatial resolution is often measured as the full-width at half maximum (FWHM) of the PSF itself, as mentioned in the previous Chapters. Given the gaussian- and donut-shaped focal intensity distributions of the excitation and depletion beams, respectively, and considering a STED pulse width of TST ED=600 ps, we calculated the expected fluorescence decay for each pixel of the image
(Eq. 6.1), i.e. the so-called temporal effective PSF (tE-PSF) of the pSTED microscope. In essence, the calculation of the tE-PSF is equivalent to simulate molecules located at different distances between the center of the excitation spot (r = 0), where the STED power is theoretically null, and rcrest, where the STED intensity is maxed.
Successively, for each pixel, we computed the phasor values and we reported them in the phasor plot. The corresponding family of phasors describes a trajectory that: (i) starts (phasor corresponding to the point r = 0) in a position located inside the semicircle, since we simulated a realistic situation in which the center of the donut is not a perfect "zero"-intensity point; (ii) does not terminate (phasor corresponding to the point rcrest, where the STED
intensity is maxed) in the limiting position (1,0), due to the limited intensity of the STED beam (Fig. 6.2b).
We then applied the linear decomposition described above. As P1 component ("wanted"
central signal), we chose the starting point of the trajectory, the one associated to the slowest dynamics, and as P2("unwanted" peripheral signal) a point located at the tail of the trajectory.
Results show that the method is effective in rejecting the contribution of fluorophores from the periphery of the excitation spot, thus leading to a reduction of the effective PSF (Fig.
6.3 Results 66
Figure 6.1 Phasor plot trajectories of a single molecule exposed to increasing STED beam intensities, for different STED beam pulse-widths. (a) Three cases of increasing g = TST ED/tf l are represented, where TST ED is the STED beam pulse-width and tf l the
unperturbed fluorescence lifetime:g = 0.2 (light blue), g = 1 (red) and g = •, the continuous-wave STED laser configuration (green). (b) A synthetic pSTED experiment of a single molecule exposed to increasing doses of stimulating photons, for the three pulse-widths cases depicted in (a). All trajectories in the phasor space obtained increasing the STED intensity start from the point in the semicircle describing the unperturbed fluorescence lifetimetf l, and
end in the point (g,s) = (1,0) describing instantaneous decay. Ifg = 0.2 (very short STED pulse-width, light blue), the trajectory is a chord in the semicircle; in the limit case ofg = • (CW configuration, green), the trajectory is an arc of the semicircle. Values ofg 2 (0.2,•) lead to trajectories lying between the two limiting cases. (c) We now consider the condition g = 1 (red trajectory): if the molecule lies in the very center of the excitation spot (left, dark blue star) it does not interact with stimulating photons, thus its unperturbed temporal decay is represented in the phasors space as the corresponding point in the semicircle. Increasing the distance of the molecule from the center of the excitation spot leads to faster decays, due to higher STED intensities (center, blue star and right, light blue star): the corresponding points in the phasor space are thus closer to (g,s) = (1,0). Repetition rate of the simulated excitation and STED beams: 60 MHz.
6.3 Results 67 6.2a). To quantitatively assess the resolution enhancement, we iterated the above-described simulation for increasing values of the saturation factorVST ED=IST ED/Is, where Is is the
STED saturation intensity, i.e., the intensity at which the rate of stimulated emission kST ED=
1/tST EDduring the STED pulse equals the spontaneous rate of de-excitation kf l =1/tf l.
We conclude that the pSTED-SPLIT approach allows using notably less STED intensity to achieve a given resolution when compared to the raw STED counterpart (e.g., 50% IST EDfor
a target FWHM of 140 nm, Fig. 6.2c).
6.3.2 pSTED-SPLIT imaging of fluorescent beads
We successively validated the pSTED-SPLIT using a phantom sample of fluorescent beads and a biological sample.
We first tested the performances of our setup by performing a series of time-resolved pSTED measurements with increasing depletion power, imaging a sample of 60 nm sized Crimson fluorescent beads (Fig. 6.3). The cluster of phasors, related to all pixels in the image, lies close to the semicircle when the STED beam is off (the not perfect overlapping may be a consequence of the high concentration of fluorophores on the beads, which induces a self-quenching phenomenon (Vicidomini et al. (2013)). As the depletion power increases, and the spatial resolution of the resulting images improves, the cluster elongates toward the point (1,0), following the simulated expected trajectory.
We then tested the linear decomposition approach on the same of 60 nm sized crimson fluorescent beads. It is worth mentioning here that, in real experiments, the choice of the two points P1and P2 in the phasor plot represents a critical aspect of the pSTED-SPLIT
approach. In this work we applied the following protocol for each measurement (Fig. 6.4): (i) perform a time-resolved confocal measurement (the STED beam is off) to retrieve the unperturbed fluorescence lifetime of the fluorophore; fixPnas the corresponding phasor on
the semicircle at the shortest distance from the centroid (center of mass, CoM) of the resulting cluster of phasors; (ii) simulate via a custom-made Matlab tool a molecule/fluorophore with a lifetime described byPn; we then consider a STED impulse of duration TST ED= 600 ps, and
calculate the different temporal behaviours of such molecule when interacting with increasing doses of stimulating photons: the different decays are described by a family of points in the phasor plot, or expected trajectory; (iii) perform a time-resolved pSTED measurement of the same area of step (i): the phasor analysis reveals a distribution of points in accordance with the expected trajectory; (iv) select bothP1andP2on the simulated trajectory. A pointP1
6.3 Results 68
Figure 6.2The pSTED-SPLIT method applied to synthetic data. (a) Simulation of the temporal effective point spread function (tE-PSF) of a pSTED experiment, equivalent to simulate the temporal dynamics of molecules located at different distances r with respect to the center of the excitation spot. The g and s images of the raw STED data indicate different temporal dynamics as a function of r. Notably, also for r = 0 (center of the excitation spot), the temporal decay of the molecule is perturbed, since the depletion intensity profile features a non-zero minimum to mimic real-life pSTED experiments. Thus, the point P1representing
such molecule in the phasor space (b) does not lie exactly on the semicircle. As expected, increasing r values are related to point closer to (g,s) = (1,0). Given two points P1and P2
related to central molecules and peripheral ones respectively, the pSTED-SPLIT method removes the fluorescence signal arising from the latter (a, incomplete depletion) via a linear decomposition approach. The result (a, pSTED-SPLIT) shows a shrunk PSF compared to the raw STED counterpart. (c, left) Simulations of pSTED experiments with increasing saturation factorV: the resolution enhancement of the pSTED-SPLIT method over the traditional raw STED counterpart is evaluated measuring the full-width at half-maximum (FWHM) of the corresponding PSFs. The ratio of the STED intensity needed for a pSTED-SPLIT over traditional STED measurement is plotted as a function of the achieved resolution (c, right).
6.3 Results 69 "zero" intensity condition; (v) force all the phasorsP(x,y) to the line in between P1P2and
the expected trajectory; then find the fractional componentsf(x,y) = M 1(x,y)P(x,y). We
finally obtain the pSTED-SPLIT image as N1(x,y) =f1(x,y)N(x,y) (Fig. 6.4b, top).
Following this procedure, the pSTED-SPLIT approach succeeds in rejecting contributions from the periphery of the excitation spot and thus shrinks the size of the effective PSF when compared to the raw STED counterpart, i.e., the raw data collected during the time-resolved pSTED measurement, before applying the decomposition algorithm (FWHM of the gaussian fitting, averaged over n = 300 beads: 79 ± 28 nm and 69 ± 4 nm, for the raw STED and pSTED-SPLIT images, respectively). The reduced effective PSF of the pSTED-SPLIT method leads ultimately to better resolved images. Notably, the maximum counts of the raw and pSTED-SPLIT images are similar, which indicate that the SNR is not reduced during the process.
We continued comparing the pSTED-SPLIT approach with the phasor-based segmentation that we introduced in a previous work (Sun et al. (2009)). In short, we used the CoM of the cluster of points in the phasor plot to generate two circular regions of interest with the same radius and touching - externally tangent - in the cluster’s CoM. The first region - centered in the point (1,0) - selects pixels with a shorter fluorescence temporal decay, thus composed primarily by "unwanted" photons. The second region selects pixels with a longer fluorescence temporal decay, thus composed primarily by "wanted" photons. By back-projecting the pixels lying in the second region we obtained a segmented pSTED images (Fig. 6.4b, bottom). Notably, the method proposed by (Wang et al. (2018)) better selects the two regions on the phasor plot ("abandoned" and "selected" area), since it allows considering all the pixel contained in the semicircle, but the final pSTED result is still a segmented version of the raw image. Importantly, the non-linear nature of the segmentation operation (i.e. the intensity of the final images is not anymore linear with the fluorophores concentration) suggests that the measured "PSF" may not be indicative of the performance of the system. For these reasons, even if the pSTED-SPLIT method shows only a marginal improvement over the phasor-based segmentation method on fluorescent beads imaging, it performs substantially better when imaging more convoluted structures, as I will show in the next sub-Chapter.
6.3.3 pSTED-SPLIT imaging of fixed cells
We compared the proposed pSTED-SPLIT method with the phasor-based segmentation method on fixed HeLa cells with TOM20-labeled mitochondria (Fig. 6.5). Thanks to the pho-tons separation - rather than the pixel separation - the pSTED-SPLIT method is able to reveal
6.3 Results 70
Figure 6.3Time-resolved pSTED measurements of 60 nm sized fluorescent beads with increasing STED power. Images clearly reveal a resolution improvement when increasing the STED power, from confocal imaging (PST ED= 0 mW, left) to pSTED imaging (PST ED
= 25 mW, center; and PST ED = 91 mW, right). Magnified views of the recorded images
are also shown. The phasor analysis (bottom) unveils the increasingly perturbed temporal dynamics of incomplete-depleted fluorophores, as expected from simulations (dotted red line). Pixel-dwell time: 100µs. Pixel-size: 20 nm. Format: 512 ⇥ 512 pixels. Scale bars: 1 µm.
the localisation of the TOM20 proteins in the membrane of the mitochondria with higher spatial resolution (with respect to confocal) and without introducing artefacts. On the con-trary, the phasor-based segmentation method tends to generate spotty structures, which may not adequately represent the real morphology of the mitochondria membrane. Similarly to
6.3 Results 71
Figure 6.4pSTED-SPLIT analysis of fluorescent beads. Results of the linear decomposi-tion and the pixel segmentadecomposi-tion approaches for time-resolved pSTED measurements of 60 nm sized crimson fluorescent beads. (a) Resolution is increasing from confocal imaging (top) to pSTED imaging (bottom); magnified views of the original measurements are reported. The phasor analysis (a, right) shows how the STED beam perturbs the temporal dynamics of the fluorophores. (b,top) The pSTED-SPLIT method shown for magnified views of recorded data: points P1and P2are chosen from the phasor plot on the simulated trajectory (red dot
line) calculated for point Pn. For each pixel, "wanted" and "unwanted" photons are sorted
thanks to the linear decomposition approach: the corresponding fractional components f1
and f2allow to generate the final pSTED-SPLIT result and the image obtained solely by the
rejected photon, respectively. Notably, the signal to noise ratio of the raw pSTED image is preserved by the pSTED-SPLIT analysis. (b, bottom) The pixel segmentation approach: in the phasor space, we define a circular region of interest (RoI) centered in the limiting point Pl = (1,0) passing through the center of mass (CoM, black star) of the distribution of
points (brown circle). A second circular RoI with the same radius is defined to be tangent with the first one and with the center lying on the straight line identified by CoM-Pl (green
circle). The two resulting binary maps are related to pixels dominated by peripheral and central fluorophores, respectively. The final segmentation result is then obtained by applying the binary map on the raw pSTED intensity image. Pixel-dwell time: 100µs. Pixel-size: 20 nm. Format: 512 ⇥ 512 pixels. Scale bars: 1 µm.
intensity-based segmentation approaches, the phasor-based segmentation methods highlight the brighter pixels, producing good-looking results when imaging well separated point-like structures such as fluorescent beads, but tending to loose morphological information for other structures.
6.3 Results 72
Figure 6.5pSTED-SPLIT analysis of fixed cells. Confocal image (left) and the results of the pixel segmentation (center) and linear decomposition analysis (right) of time-resolved pSTED measurements of a sample of a fixed cell with Atto 647N labeling mitochondria via the TOM20 protein. Magnified views (central row) and phasor plot analysis (bottom) are also shown. Pixel-dwell time: 100µs. Pixel-size: 24 nm. Format: 512 ⇥ 512 pixels. Scale bars: 1 µm.
Chapter 7
Single Molecule Tracking with a SPAD
Array Detector
7.1 Introduction and state of the art
In the previous Chapters, I introduced a novel microscopy platform based on the SPAD array detector, and showed the advantages that this platform grants in the context of various imaging applications. In particular, I showed how the new spatial information provided by the detector array can be used by meas of the image scanning microscopy method to improve the spatial resolution and signal-to-noise ratio of confocal laser scanning microscopy, fluorescence lifetime imaging and STED microscopy. Here, I decline our architecture to explore the extra spatial information provided by the SPAD array detector in the context of a different application: single molecule/particle tracking (SMT or SPT). In this first Section, I will briefly present the state of the art for the family of tracking techniques, while in the following Sections I will introduce our novel, real-time, feedback based implementation of a SMT system based on the SPAD array detector.
Single molecule experiments have several advantages over ensemble experiments, including the detection of sub-populations, investigation of the sample heterogeneity and direct visu-alisation of dynamic processes (Dupont and Lamb (2011)). Single particle tracking (SPT) - firstly proper to the astronomy field as a tool to follow the planets in the sky - was intro-duced to the study of biological systems in 1971, allowing the three-dimensional tracking of individual bacteria (Berg (1971)). Since then, SPT, or better SMT, successfully allowed to investigate the movement of different bio-molecules, such as the stepping mechanism of molecular motors (Gelles et al. (1988)) and the Brownian motion of lipids in a bilayer (Lee et al. (1991); Schmidt et al. (1995)). In living cells, SPT was first demonstrated by tracking
7.1 Introduction and state of the art 74 gold nanoparticles injected into the cell cytoplasm (Geerts et al. (1987)). Later, fluorescence based SPT methods were used to investigate the motion of lipids in the plasma membrane (Fujiwara et al. (2002)), to follow the entry process of adeno-associated viruses along their infection pathway (Seisenberger et al. (2001)) or to track other biomolecules of interest inside the cell (Kubitscheck et al. (2000)).
From the point of view of the implementation, the several SPT/SMT techniques introduced in the recent years may be divided into two main strategies: (i) a-posteriori approaches, for which the image acquisition is later followed by a data analysis step to extract the particles trajectories; (ii) real-time, feedback-based approaches.
The a-posteriori family of approaches relies on the acquisition of a time series of multiple images captured at different depths of the sample, either via a confocal or a wide-field microscope. After the stack is recorded, the trajectories of individual particles are extracted from the data, by means of automatic or semi-automatic tracking software routines. It is later possible to extract various parameters from the retrieved paths, such as the mean squared-displacement, which provides information about the type of diffusion behaviour the particle undergoes, the diffusion coefficient, the instantaneous or average velocity and others. This family of techniques is undoubtedly characterised by an high data throughput, since it allows to build many particle tracks from a single measurement. However, such an approach presents a few drawbacks: (i) the tracking space is limited to the volume of investigation decided at the beginning of the experiment. In other words, it is not possible to follow a particle that exits the boundaries of the pre-defined region of interest; (ii) the temporal resolution is limited, particularly when considering laser scanning architectures such as the confocal microscope, as a direct consequence of the scanning approach. Although the choice of using a wide-field architecture obviously improves the temporal resolution - thus enabling the observation of faster dynamics - the resulting recorded images are characterised by a lower signal-to-background ratio (SBR). Importantly, wide-field implementations also hinder the straightforward combination with essays leveraging the photon-arrival temporal information, and as such severely limit the information content potentially extracted from the data. The second class of SPT/SMT techniques relies on a real-time, feedback-based strategy in the context of a laser scanning microscopy system. The paradigm here is different: only one particle is "observed" at a time, and the observation volume - in general much smaller compared to the field-of-view of the a-posteriori family - is moved to follow the observed particle. Clearly, this strategy relies on some method to assess the position of the particle with respect to the observation volume, which is then fed to a control system to update the position of the observation volume and ultimately to perform the tracking. Notably,
7.1 Introduction and state of the art 75 it is no longer necessary to define a region-of-interest a-priori, since the particle can be tracked indefinitely - within the maximum volume allowed by the scanning system. More precisely, the only limitations related to the duration of the tracking experiment are the photo-physical effects that can prevent the molecule to emit photons, such as bleaching or blinking of the fluorophore. The data throughput of the real-time techniques is surely inferior with respect of the a-posteriori family of approaches, considering that it allows to track only one particle at a time; however, the temporal resolution may be substantially increased, particularly when compared to a-posteriori techniques based on confocal architectures. More importantly, the real-time approach favours the potential combination with essays leveraging the photon-arrival temporal information. One of the most common real-time implementa-tions of SPT is the so-called orbital tracking (Hellriegel and Gratton (2009); Wells et al. (2008)): the confocal laser spot (detection volume) is scanned around the particle of interest in a circular orbit, with radius usually half the width of the point spread function, and the fluorescence signal is recorded with a single element detector. If the particle is exactly in the center of the orbit, the registered fluorescent signal is constant over one scanning period. Conversely, if the particle is displaced from the center of the orbit, the fluorescent signal is modulated according to its position relative to the orbit’s center. Phase and modulation of the fast Fourier transform of the intensity signal measured during a orbiting cycle can be related to the particle position, and the calculation is fast enough that it can be used for real-time feedback. The axial coordinate of the particle is determined by the difference of the intensity signal measured in two different z-planes, separated by half the axial width of the point-spread-function. More recently, the extension of the orbital also along the axial direction is introduced thanks to a fast electrically tunable lens (Annibale et al. (2015)). The feedback loop, operating on the scanning device, directs the laser beam to orbit around the new estimated position of the particle, and ultimately the entire trajectory of the fluorescent particle is recorded in real-time. Other feedback strategies rely on variations of the confocal geometry, such as the introduction of four slightly offset optical fibers in detection - acting as four confocal pinholes - each coupled with an avalanche photo-diode (APD) (Levi et al. (2005)). In this study, one pair of fibers define the x-direction, while the second pair of fibers is aligned along the y-direction. By comparing the intensities collected in the four channels, it is possible to assess the particle position, and move the sample stage accordingly. Although this approach allows to follow the particle without the need of performing orbital scanning, it relies on a complex detection architecture. Moreover, the usage of four optical fibers may lead to a reduced overall photon detection efficiency. Other methods based on CCD cameras as detecting hardware have been proposed to address these limitations (Juette
7.2 Material and methods 76 and Bewersdorf (2010)). In this study, only a subset of pixels is read from the camera - with a frame rate up to 3.2 kHz - and used to estimate the particle position. Albeit relaxing the architectural complexity of the tracking system, the usage of a CCD camera notably hinders the photon-arrival time information.
Here, we present a novel, real-time, feedback-based SMT implementation based on the SPAD array detector. In brief, each sensitive element acts as a confocal pinhole, and the recorded intensity distribution mirrors the position of the particle with respect to the center of the excitation volume: for example, if the particle lies at the center of the confocal laser spot, the maximum signal is recorded by the central element. Any displacement of the particle leads to a variation of the distribution of recorded signal, thus allowing the estimation of the particle position. Notably, this approach allows to track particles without the need of performing orbital scanning; moreover, it is implemented leveraging our image scanning microscopy platform (see Chapter 3), a confocal microscope equipped with the SPAD array detector: as such, the architectural complexity is reduced with respect to previously described approaches. Moreover, the high count rates allowed by the SPAD array (up to 20 Mcount/s), combined with the asynchronous implementation of both the detector and the data acquisition electronics, represents an advantage over camera-based approaches. This implementation allows to potentially combine real-time SMT with spectroscopy analysis, e.g., Fluorescence Correlation Spectroscopy (FCS). Most importantly, the single-photon timing capabilities of the SPAD array detector - already explored in the imaging context (see Chapter 4) - would allow for the combination of the SMT with time-resolved essays such as fluorescence lifetime and smFRET, hence greatly increasing the information content of the recorded data.
7.2 Material and methods
7.2.1 Optical setup
The SMT implementation based on the SPAD array detector presented in this Chapter relies on the same optical setup presented in Chapter 3. The only difference is the addition of a three-axis piezo-positioning sample stage (MAX302, Thorlabs) coupled with the adeguate controller (MDT693B, Thorlabs).
7.2.2 The SMT implementation based on the SPAD array detector
I implemented the single molecule tracking (SMT) architecture using the software Labview and the module Labview FPGA. The project is composed mainly of two modules: (i) the
7.2 Material and methods 77 high-level program, running on the host computer; this module presents a graphical user interface (GUI), allowing to set all the needed parameters, to start and monitor a SMT measurement, and to analyse the results (Fig. 7.1). (ii) The low-level firmware module running on the FPGA, which hosts the real-time, feedback-based control at the core of the SMT implementation. In a nutshell, the system monitors the signals from the 25 sensitive elements of the SPAD array, and retrieves the position of the particle. It then – by moving the pair of galvo-mirrors – redirects the excitation laser light to the calculated position, thus closing the feedback control loop.
I will now describe in more details the functional components of the proposed architecture. The two key controls are the "particle evaluation strategy" and the "moving strategy" param-eters. The former defines which algorithm is used in the FPGA to retrieve the position of the particle from the small 5⇥5 image from the SPAD array, in real-time. Currently, two modalities are available: one leverages the position of the sensitive element collecting the maximum number of photons, the other calculates the ceter-of-mass (CoM) of the image. On the other hand, the "moving strategy" parameter allows to select different approaches to update the position of the excitation beam: (i) as soon as a particle is detected (i.e., when the number of photons recorded during a pre-defined "waiting time" is major than a selected threshold), or (ii) at a fixed frame rate. Notably, regardless of the selected strategy, if the real-time system fails to detect a particle within the user-defined "waiting time", it doesn’t update the position of the excitation light, and starts again to check the presence of the particle. This prevents the system to track noise, if the particle is lost or not in the field of view.
The core of the SMT project relies in the FPGA firmware, composed of different modules, each with a specific function:
• The photon counter module, consisting in 25 very high-frequency loops (200 MHz), each monitoring one of the 25 digital signals produced by the SPAD array detector. Every time a photon is detected by any of the elements of the detector, a Transis-tor–transistor logic (TTL) signal reaches the FPGA board and the dedicated register is updated. The asynchronous nature of the detection hardware is exploited by the usage of 25 parallel loops, each coupled with a different register. Moreover, the frequency of this module is the highest in the project, ensuring that the correct value of any register is available everywhere else in the firmware at any given time;
• The main state-machine, hosted inside a 25 MHz single-cycle timed loop. The three states are defined as follows: "collecting", "calculating position" and "moving". The
7.2 Material and methods 78 first state monitors the acquired photons and, for each cycle, applies a simple threshold algorithm to assess whether a particle is detected or not. The detection result is evaluated, among with the user-defined parameters such as "moving strategy" and "maximum waiting time", to decide if to keep collecting photons, or to enter the second state of the machine, "calculating position". Notably, if the maximum waiting time elapses and no particle has been detected, the system set all the counters to 0 and stays in the "collecting" state. In the second state, "calculating position", the system analyses the 5⇥5 image obtained from the previous step with the user defined algorithm (i.e., calculating the center-of-mass or the position of the maximum value). The result of the analysis is then used in the "moving" state, to update registers containing the last calculated position of the particle; then the system enters again the "collecting" state; • The actuation module, which constantly monitors the latest particle position stored in the proper register; as soon as a new value is available, it updates the excitation laser light position accordingly;
• The communication module, which prepares and sends to the high level program, eachµs, all the information needed: the 5 ⇥ 5 current image; the current calculated position of the particle in the sample plane (and thus of the scanning system); and the information of whether the particle is currently detected or not.
7.2.3 The simulation platform
To validate the proposed SMT platform, i.e., to assess if the system is capable of correctly calculating the position of a given particle and to track its movement, I decided to implement a "simulation" environment. In this framework, the observed particle is fixed in the sample (e.g., a 20 nm gold beads fixed on a coverslip), but the whole sample is moved by means of an additional system, in this case a controllable 3-axis piezo-electric stage, hosting the sample. Feeding the piezo stage with known coordinates results in the observed particle moving of a known trajectory; therefor, the validation of the SMT platform relies on the comparison between the known trajectory imposed to the particle through the piezo-stage, and the retrieved trajectory obtained by the SMT architecture.
The simulation platform is built within the SMT architecture, and it is composed as follows: (i) the high level module, generating the spatio-temporal coordinates of the simulated track; and the (ii) low-level FPGA firmware, receiving the coordinates from the high level trough a dedicated host-to-target first-in-first-out structure (H2T FIFO), and actuating the piezo stage
7.2 Material and methods 79
Figure 7.1The SMT tracking graphical user interface. The GUI is divided into three main modules: (i) "SPT parameters", with which the user may set all the parameters related to the SMT experiment. It is here possible to select which elements of the SPAD array detector to use for assessing the position of the particle. (ii) "simulation", with which the user can define the type of the imposed trajectory (currently, line or circle) and other parameters such as the (constant or variable) particle velocity, the dwell-time (i.e., the amount of time after which the position is updated), and others. (iii) "running experiment", with which the user can monitor the running experiment. Information about the simulated track, the retrieved track, the SPAD array image, and the flag for the detected particle are shown here. While the experiment is running, all raw data (regarding the eventual simulation and all the information from the real-time FPGA module) is saved.
when needed via dedicated analog outputs. For the sake of synchronization, I decided to implement the low-level FPGA "simulation" module in the same FPGA firmware hosting the SMT platform. Notably, the two modules are completely independent one to each other, thus ensuring that the performances of the SMT architecture are not influenced by the ongoing simulation. In other words, the SMT module is never aware of the eventual simulation, and doesn’t make any assumption on the nature of the movement of the particle.
7.2 Material and methods 80
7.2.4 The communication protocol
Here I would like to describe more in details the communication protocol I implemented for sending the data from the FPGA to the high level program. As mentioned before, the information to be sent consists of: 25 intensity values, i.e., the 5x5 image from the SPAD array; the (x,y,z) coordinates related to the last calculated particle position (and thus the current center of the excitation spot), and a flag representing whether or not the particle is currently detected.
To achieve the best data transfer-rate possible, I decided to use a target-to-host (H2T) FIFO over the USB 2.0 connection provided by the FPGA board in use (the USB-7856R board by National Instruments). I performed characterization tests to assess the effective maximum transfer rate through this channel, which resulted in 320 Mbit/s, inferior to what was expected from specifications of the USB BUS of 480 Mbit/s. To maximize the versatility of the implementation, I opted to send the data packets at the maximum frequency possible, under the following constrains: (i) the maximum intensity value for each pixel of the SPAD array image must be greater than the counts that would be obtained having the detector in saturation, thus ensuring that no detected photon is lost in the communication process; (ii) the (x,y,z) coordinates must cover a tracking range in the order of millimeters, thus far exceeding the expected field of view of a reasonable SMT measurement; moreover, the minimum distance between two adjacent points must be small enough to accommodate eventual position retrieving algorithms with high localisation precision; (iii) the data transfer rate must not exceed 80% of the actual maximum.
Therefore, the final communication protocol is as follows (Fig. 7.2): • The message packet is sent at a frequency of 1MHz.
• Each packet is built of four unsigned 64 bits words, from W0 to W3; W0 hosts the intensity values of 10 elements of the SPAD array (from d0 to d9); W1 hosts other 10 elements (from d10 to d19); W2 hosts the remaining 5 intensity values (from d20 to d24), and the z coordinate; W3 hosts the x and y coordinates, and the flag related to the particle detection. In addition, each word has a unique identifier, to detect eventual data loss and avoid data misinterpretation.
• The intensity value of each pixel of the SPAD array is represented by 6 bits, allowing a maximum value of 63 counts (more than three time the counts produced by a detector with 50 ns of hold-off in saturation, in 1µs);
7.3 Preliminary results 81
Figure 7.2The SMT communication protocol for the target to host FIFO. In light blue, the counts of each sensitive element of the SPAD array detector (6 bits each). In blue, the (x,y,z) coordinates (25 bits each). In orange, the word identifiers (2 bits). In grey, unused space for future developments. The packets built of these four words is sent to the high level module at the frequency of 1MHz.
• The (x,y,z) coordinates are represented with 25 bits each, allowing a range of roughly 2 mm in each dimension, with a minimum distance of 1 nm.
This communication protocol results in a constant transfer rate of 256 Mbit/s, which I consider safe to avoid any data loss. I validated this by performing experiments of know and various duration, to ensure that the system was receiving all the expected packets. The reasoning behind transferring data the fastest, and not syncing the transfer of each packet with a new particle position, is that in this way the two processes are completely independent. The update of the excitation laser position may be easily monitored a posteriori, by observing the coordinates sent within the packet. It is of course true that, in case of updating the particle position with a frequency higher that 1 MHz (which is allowed by the platform), some calculated positions will be not sent to the high level and thus stored. However, this doesn’t represent a major loss, since the coordinates in the packet are absolute and not differential. Moreover, the condition for which it is feasible and beneficial to calculate the position of any given particle at such a high frequency is extremely unlikely.
7.3 Preliminary results
In this Section, I will briefly present the preliminary results obtained with the proposed SMT platform. I decided to use a sample of gold nanoparticles, and to work in reflection mode, i.e., the intensity signal recorded by the SPAD array detector is the reflection of the "excitation" laser source on the gold particle themselves. At this stage, I avoided the fluorescence modality to ignore the probable variability of the fluorescence intensity signal, to be expected mostly because of bleaching effects of the observed molecule. Conversely, the reflection modality allows potentially to track indefinitely the observed gold particle, and
