LITHUANIAN UNIVERSITY OF HEALTH SCIENCES MEDICAL ACADEMY NEUROSCIENCE INSTITUTE

LITHUANIAN UNIVERSITY OF HEALTH SCIENCES

MEDICAL ACADEMY

NEUROSCIENCE INSTITUTE

KOLCHEVSKIY ALEKSANDR

SYSTEMATIC REVIEW ON CELL MORPHOMETRY

METHODS IN DIGITAL IMAGING: MATHEMATICAL

MORPHOLOGY BASED ESTIMATES OF CULTURED CELL

BEHAVIOUR

SUPERVISOR: DR. ROBERTAS PETROLIS

TABLE OF CONTENTS:

6. AIM AND OBJECTIVES OF THE THE THESIS...9

7. LITERATURE REVIEW...10

7.1 Single cell migration...11

7.2 Collective cell migration...12

7.3 Mathematical morphology applications in digital imaging...14

7.4 Importance of cell motility analysis ...18

8. RESEARCH METHODOLOGY AND METHODS...20

9. RESULTS...23

10. DISCUSSION OF THE RESULTS...31

11. CONCLUSIONS...33

12. REFERENCES...34

13. ANNEXES...39

13.1 Microscopy...39

1. SUMMARY

Aleksandr Kolchevskiy „Systematic review on cell morphometry methods in digital imaging: mathematical morphology based estimates of cultured cell behavior”.

The aim of this work is to review mathematical morphology (MM) based cell migration analysis techniques and highlight their advantages together with their limitations.

Objective:

 Analyze scientific article databases and look for suitable studies for the systematic review.  Assessment of methods described in articles selected for the review.

 Interpret the usage of mathematical morphology (MM) methods in selected articles and highlight MM advantages together with their limitations.

Methods:

A literature search was conducted in the available databases and search engines with the principles outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. After the selection of 15 publications all the gathered data was analyzed and prepared for detailed description.

Results:

Since the appearance, behavior and living environment of cells or their compounds can be quite different, therefore image analyze techniques developed to estimate their behavior are also quite different. The most common procedures of analyzed algorithms were described which were using MM, to understand the importance of the methodology.

Conclusions:

MM methods is a powerful mathematical tool for analyzing and solving problems that occur during implementation of cell motility analysis algorithms.

MM methodology is widely used among the variety of developed methods and even some combinations of MM operations are „golden standard“ for unwanted distortion elimination during analysis till now, even though principles of it were published in 1980‘s.

In analyzed articles MM operations are only one of the methods parts for cultured cell behavior analysis, nevertheless they play a critical role in the end result.

1. SANTRAUKA

kurie skirti ląstelių gyvybingumui analizuoti. Išryškinti MM metodų taikymo sritis bei apribojimus analizuojamuose algoritmuose, kurie buvo rasti mokslinių straipsnių duomenų bazėse, remiantis straipsnių atrankos gairėmis, pagal - PRISMA (Preferred Reporting Item for Systematic Review and Meta-Analyses) metodinius nurodymus.

Pagrindiniai žodžiai:

Matematinė morfologija; Ląstelių gyvybingumas; Vaizdų analizė; Ląstelių judrumas; Ląstelių segmentavimas.

2. ACKNOWLEDGEMENTS

This „Systematic review on cell morphometry methods in digital imaging: mathematical morphology based estimates of cultured cell behavior” is a part of my integrated studies in the Lithuanian University of Health Sciences (LSMU) as a Final Master Thesis, during my 6th year of Medical Academy studies.

I would like to thank personally DR. ROBERTAS PETROLIS who helped, taught and advised me throughout the whole process of „Systematic review on cell morphometry methods in digital imaging: mathematical morphology based estimates of cultured cell behavior” work.

3. CONFLICT OF INTEREST

4. ABBREVIATIONS

 MM – Mathematical morphology  ROI – Region of interest

 SE – Structuring element

 RGB – color model (Red, Green, Blue)  TEM - Transmission electron microscopy  SEM - Scanning electron microscopy

 LSMU- Lithuanian University of Health Sciences  MLK- Mokomasis Laboratorinis Korpusas  COVID-19- Coronavirus Disease 2019

5. INTRODUCTION

Cell migration is a fundamental biological process that is essential from development to wound healing [1]. Movement itself is composed of multiple dynamical processes such as surface attachment and detachment cycles, development and collapse of filopodia, movement of the cell body center, and maintenance of cell morphology [1]. All of the cell motion can be divided between spontaneous movements and tactic responses to environmental signals. Spontaneous cell movement is a random motion under no external guiding cues, which accompanies large fluctuations in the dynamical localizations of corresponding molecular components in order to coordinate function.

Tactic behaviors are achieved by biasing the cell movement in a sensitive and stable manner in response to environmental signals [2], thus playing an essential role in various cellular functions.

Depending on the stage of morphogenesis cell locomotion can be viewed differently. For example from large-scale migrations of epithelial sheets during gastrulation [3], to the movement of individual cells during development of the nervous system [3]. Although cell migration is crucial for normal development and morphogenesis of body plans and organ systems, abnormal cell migration underlies pathological states such as invasion and metastasis of cancer [1], vascular disease and inflammatory disease [3]. The molecular mechanisms through which individual cells or cell compounds migrate have been extensively studied [4]. Nevertheless a wide variety of biological processes that are directly dependent on cell motility lack complete understanding of how individual cell behaviors lead to emergent collective properties of cell groups. Because of this complex and multicontextual cells interaction has been increasingly appealed and studied a wide variety of methods and models have been proposed for such analyses [5]. Since there is no “gold standard” or universally best way to evaluate cell movement, a great variety of methods exist for different scenarios (for example: Fluorescent Microscopy [5], Phase Contrast microscopy [5], Long Term [6], or Short Term Observation [2]). The knowledge obtained from these analyses are likely to yield an early abnormality detection and generate new insights into function and malfunction of biological processes, which will eventually lead to reducing deaths and morbidity, provision of prompt advice, and opportunities for innovative treatments. In this master‘s thesis called „Systematic review on cell morphometry methods in digital imaging: mathematical morphology-based estimates of cultured cell behavior” we review the most common cell motility analysis techniques and highlight their advantages together with their limitations.

6. AIM AND OBJECTIVES OF THE THESIS

Aim:

To review mathematical morphology(MM) based cell motility analysis techniques and highlight their advantages together with their limitations.

Objectives:

 Analyze scientific articles databases and look for suitable studies for the systematic review.  Assessment of methods described in articles selected for the review.

 Interpret the usage of mathematical morphology (MM) methods in selected articles and highlight MM advantages together with their limitations.

7. LITERATURE REVIEW

Cell movement is a complicated and dynamic process in development and maintenance of human body organisms and is one of the fundamental features of a living cell.

During this activity cell morphology changes by reorganizing the actin cytoskeleton and modulating cell adhesions, these changes are invoked from a wide variety of external physiological and chemical signals. Cell motility is responsible and plays an important role in various biological processes from embryo development till immune responses (for example: wound healing). However, derangement of proper cell movement is associated with consequences, like various pathological processes (for example: Tumor formation, Inflammations or even Vascular Disease) and disorders of immune response.

In general, there are three causative factor that lead to a cell movement. First there is a random cell move (chemokinesis) due to absence of physiological and chemical signals, this kind of motility has no directional component. Second factor is chemotaxis - a directional cell movement along a positive concentration gradient of chemoattractants.

It is a fundamental form of cell behavior that involves a complex response of a cell to an external stimulus. Sensing and measuring the concentration of the chemoattractant, transmitting the information to biochemical reaction, and exhibiting the motility and adhesive changes associated with the response are critical factors involved in chemotaxis. The last factor that leads to cell movement is haptotaxis. It is a directional cell movement in response to adhesive substrates. During normal development or under pathological conditions, cells can migrate in two main ways:

 Single cell migration  Collective cell migration

Fig. 1. Cell migration types (single cell white arrows, collective black arrows)(Adopted from laboratory archives in LSMU [7])

7.1- Single cell migration:

This type of migration is described as single cell movement in particular direction to specific locations. This migration allows cells either to cover local distances and integrate into tissue (for example: Neural Crest Cell Migration) or to move from one location in the body to another and fulfill effector functions (for example: Immune Cell Trafficking).

Single cell migration can be described as a cyclical process. It begins with a cell’s response to an external signal that leads to the polarization and the extension of a protrusion in the direction of movement. The formation of adhesions attaches the protrusion to the substratum on which the cell is migrating. These adhesions serve, in part, as traction points for migration, and they also initiate signals that regulate adhesion dynamics and protrusive activity [8]. Contraction then moves the cell body forward and release of the attachments at the rear as the cell retracts completes the cycle. While relatively slow-moving cells like Fibroblasts, show these very distinct steps, the different stages are less obvious in other cell types. Rapidly migrating cells, like Keratocytes and Leukocytes, appear to glide over the substratum by protruding and retracting smoothly without forming noticeable attachments [8]. Cells migrating in sheets show features of single-cell movement also. Those at the front have protrusions while the cells at the rear of the sheet show features of retraction. The plasticity in migration mechanisms is pointing in few examples like:

1. The migration of cells in vivo differs from migrating in vitro. Migration in vivo is much more directed than that in vitro with cells forming long, stable protrusions pointed in the direction of migration [9].

2. Tumor cells also show a plasticity that depends dramatically on the environment. Under some conditions they polarize and migrate along collagen bundles, whereas under others, they become more amoeboid and use different migration mechanisms [9].

Single cell movement during morphological analysis can be characterized in to two different movement types [10]:

1. Amoeboid cell migration: This motility is the most primitive and, in the same time, the most efficient mode of migration of single tumor cells (Figure 2(A)) [1]. Commonly refers to the movement of rounded or ellipsoid cells that lack mature focal adhesions and stress fibers. There are two subtypes of Amoeboid movement: The first is the rounded, blebby migration of cells that don’t adhere or pull on substrate but rather use a propulsive, pushing migration mode.

The second occurs in slightly more elongated amoeboid cells that generate action-rich filopodia at the leading edge that engage in poorly defined, weak adhesive interaction with the substrate [10].

2. Mesenchymal cell migration: Individual cells with high levels of attachment and cytoskeletal contractility develop mesenchymal migration (Figure 2(B)), which involves focalized cell-matrix interactions and movement in a fibroblast-like yielding varying migration speeds, such as the fast migratory scanning of single leukocytes, the relatively slow invasive migration during organ formation [10].

Fig. 2. Modes of single cell migration: (A) Amoeboid, (B) Mesenchymal (Adopted from [8]).

7.2- Collective cell migration:

This type of migration is described as movements of group of cells and the emergence of collective behavior from cell-environment interactions and cell-cell communication. Cells can migrate as a cohesive group (for example: Epithelial Cells) or have transient cell-cell adhesion sites (for example: Mesenchymal Cells). They can also migrate in different modes like sheets, strands, tubes, and clusters. Collective cell migration is the prevalent mode of migration during development, wound healing, and tissue regeneration. Collectively migrating cells use similar mechanisms as single cells to protrude, polarize, contract, and adhere to the surrounding matrix.

However, their ability to interact with each other both chemically and mechanically provides cells within the moving group with additional mechanisms to migrate while [1]:

 Maintaining tissue cohesiveness and organization.  Regulating tissue paracellular permeability.  Creating large gradients of soluble factors.

 Distributing tasks between specialized mobile and non-mobile cells.  Propagating mechanical signals via cell-cell junctions.

 In the case of cancer, protecting metastatic clusters from an immune assault

Collective cell migration has been observed in the development and progression of breast and endometrial cancer, prostate cancer, colorectal cancer, large cell lung carcinoma, rhabdomyosarcoma, melanoma, as well as most squamous cell carcinomas [3]. The mechanisms of collective cell migration (Figure 3) is mainly based on adhesion.

To move as cohesive groups they require both cell-matrix and cell-cell adhesions (adherents junction, tight junction, desmosomes and gap junctions).

Fig. 3. Collective cell migration (Adopted from [8]).

Table 1: Key features of single and collective migration modes (Established according [8]). Migration category Circulating form Mode of migration Key features

Single Circulating tumor

cells (CTC)

Mesenchymal Strong stress fibers, polarization,

leading/trailing edges Collective Circulating tumor

microemboli (CTM) Ameboid, Sheets, strands, tubes, Clusters Blebbing, weak adhesions, rapid motility Intact cell– cell junctions

7.3-Mathematical morphology applications in digital imaging:

Mathematical morphology (MM) is a technique for the analysis and processing of spatial structures in other words it is the study of shape. MM is usually applied to digital images, but it can be also used for other data spatial structures analysis. The main idea of this technique is to investigate the interaction between analyzed image and a particular structuring element (SE). MM main goal is to extract relevant special structures of SE set while ignoring image’s pixel values [11]. Dilation and erosion are two basic operators in MM methodology (Figure 4).

They are typically applied to binary image but you can find applications suitable for grayscale images also. The basic effect of dilation on a binary image is to progressively enlarge the limits of anterior regions pixels (usually white regions). Resulting the growth in size of anterior region while holes within the region become smaller. The basic effect of erosion is opposite than dilation operators. It erodes away the boundaries of anterior region (usually white regions) and results as shrinking the anterior region which enlarges holes in the foreground area [11].

Fig. 4. Example of erosion and dilation operation for binary image (Adopted from [11])

Effect of erosion 3x3

structuring element (square)

Effect of dilation 3x3

Opening and closing (Figure 5) are two the most important operators in MM technique which derived from the basic operators - erosion and dilation [11]. Like previously mentioned operators they are normally applied to binary images, although there are also gray level versions. The basic effect of an opening is somewhat like erosion in that it tends to remove some of the foreground (bright) pixels from the edges of regions of foreground pixels. However, it is less destructive than erosion in general. As with other morphological operators, the exact operation is determined by a structuring element. The effect of the operator is to preserve foreground regions that have a similar shape to this structuring element, or that can completely contain the structuring element, while eliminating all other regions of foreground pixels [12].

The effect of closing is similar in some ways to dilation in that it tends to enlarge the boundaries of foreground (bright) regions in an image (and shrink background color holes in such regions), but it is less destructive of the original boundary shape. As with other morphological operators, the exact operation is determined by a structuring element. The effect of the operator is to preserve background regions that have a similar shape to this structuring element, or that can completely contain the structuring element, while eliminating all other regions of background pixels [11].

There are also other MM operators which are used more rarely, but are very important for digital image processing:

 Hit and miss transform: is a binary operation that is usually used to search for particular patterns of foreground and background pixels in an image (figure 6). The same as other operators it takes as input a binary image and a structuring element, and produces another binary image as output [11].

Operation is performed mostly the same way as other morphological operators, by translating the origin of the SE to all points in the image, and then comparing the SE with the underlying image pixels. If the foreground and background pixels in the SE exactly match foreground and background pixels in the image, then the pixel underneath the origin of the SE is set to the foreground color. If it doesn't match, then that pixel is set to the background color [11].

Fig. 6. Example of hit or miss operation for binary image (Adopted from [11])

 Thinning: Is used to remove selected foreground pixels from binary images, somewhat like erosion or opening. It can be used for several applications, but is particularly useful for skeletonization (figure 7). In this mode it is commonly used to tidy up the output of edge detectors by reducing all lines to single pixel thickness. Thinning is normally only applied to binary images, and produces another binary image as output.

Thinning operation is achieved translating the origin of the SE to all possible pixel position in the image, and at each such position is compared it with the underlying image pixels. If the foreground and background pixels in the SE exactly match foreground and background pixels in the image, then the image pixel underneath the origin of the SE is set to background (zero). Otherwise it is left unchanged. Note that the SE must always have a one or a blank at its origin if

Structuring element

it is to have any effect. The choice of SE determines under what situations a foreground pixel will be set to background, and hence it determines the application for the thinning operation.

Fig. 7. Example skeletonization by morphological thinning (Adopted from [11])

 Thickening: Is used to grow selected regions of foreground pixels in binary images, somewhat like dilation or closing. It has several applications, including determining the approximate convex hull of a shape, and determining the skeleton by zone of influence. Thickening is normally only applied to binary images, and it produces another binary image as output [11].

Thickening operation is calculated by translating the origin of the SE to each possible pixel position in the image (Figure 8), and at each such position comparing it with the underlying image pixels. If the foreground and background pixels in the SE exactly match foreground and background pixels in the image, then the image pixel underneath the origin of the SE is set to foreground (one). Otherwise it is left unchanged. Note that the SE must always have a zero or a blank at its origin if it have any effect [11].

Fig. 8. Example of thickening operation for binary image (Adopted from [11])

Structuring

elements

Structuring elements

 Skeletonization:

Is a process for reducing foreground regions in a binary image to a skeletal remnant that largely preserves the extent and connectivity of the original region while throwing away most of the original foreground pixels (Figure 9).

To see how this works, imagine that the foreground regions in the input binary image are made of some uniform slow-burning material. Light fires simultaneously at all points along the boundary of this region and watch the fire move into the interior. At points where the fire traveling from two different boundaries meets itself, the fire will extinguish itself and the points at which this happens form that called „quench line”- this line is the skeleton.

Another way to think about the skeleton is as the loci of centers of bi-tangent circles that fit entirely within the foreground region being considered.

Figure No. 9 illustrates this for a rectangular shape [11].

Fig. 9. Example of skeletonization operation for binary image (Adopted from [11])

7.4-Importance of cell motility analysis:

Quantitative cell motility studies are necessary for understanding biophysical processes, developing models for cell locomotion and for drug discovery.

While collective cell migration is crucial in development and tissue repair, it also mediates devastating diseases such as cancer. The traditional view of cancer metastasis is based on the notion that single cells death from primary tumors, crawl through the stroma, enter the blood and lymphatic vessels, and finally colonize in healthy tissues to form a secondary tumor [1].

However, increasing evidence indicates that tumor dissemination is driven not only by single cells but also by cohesive cell groups. Numerous techniques have been used to probe the relationship between

A

B

C

E

D

cellular deformability and metastatic ability including particle-tracking micro rheology, atomic force microscopy, and cell stretching devices [8]. Overall, studies of cell deformability indicate that more metastatic cells are more compliant than their healthy counterparts and that deformability is predictive of cancer staging.

Understanding these molecular mechanisms is critical in identifying therapeutic targets. As a result of development of intravital microscopy the understanding of collective invasion in cancer is currently undergoing rapid progress [8]. Intravital microscopy has demonstrated the coexistence of single and collective cell invasion in a variety of organotypic cancer models [1]. These modules may be useful for pharmacological testing and have clear implications for personalized medicine.

Accurate segmentation and tracking of cells in microscopic imaging is becoming to be an important step in cell-motility studies. Using image analysis techniques to extract velocity information from various origin images of cells and cell compounds, enables to quantify the migration. As migration studies become increasingly complex in an attempt to combat the wide variety of phenotypes seen in cancer and other diseases, various quantitative measures of motility will become increasingly important.

Lithuanian research scientists are also making their contribution to the field of cell motility analysis using MM methods. Laboratory of Biophysics and Bioinformatics of Neuroscience Institute of Lithuanian University of Health Sciences has a 10 years’ experience in creation various algorithms for cell motility analysis. Two of them are also included in this review [20, 27].

8. RESEARCH METHODOLOGY AND METHODS

A literature search was conducted in the following databases and search engines: Medline, Embase, Scopus, Google scholar (available via LSMU library data bases access).

No publication year limitations were set (due to master’s thesis regulations evaluation form requirements priority was set for articles not older than 10 years) in the databases or search engines settings.

The search was limited to English language and the following search strategy was selected:  “Cell motility” (for example articles found in Medline 222623).

 “Cell motility” and “images” (for example articles found in Medline 3660).

 “Cell motility” and “images” and “mathematical morphology” - (for example articles found in Medline 35).

Additional literature search was done in Lithuanian University of Health Sciences Medical Academy Neuroscience Institute’s Laboratory of Biophysics and Bioinformatics archives. Since this laboratory’s research topics are related to master thesis analysis subjects and personnel of the mentioned institution are attending conferences and participating in various projects a data base of articles is collected. The selection procedure was performed in two stages by two independent reviewers (RP and KA).

At first the selection was based upon the title and the abstract, considering the inclusion/exclusion criteria. The inclusion criteria were:

1. the study consists of cell or cell compound movement (motility) analysis;

2. one or more movement estimates are described and extracted from digital images; 3. full paper articles or studies are available.

Next stage of the study selection was full texts analysis and proposed method or algorithm investigation. If the analyzed technique in any of its steps consists of mathematical morphology operations it was added to studies database.

Disagreements during the first and second selection were resolved by discussion and consensus with reviewers and with the independent opinion of expert of image analysis from Neuroscience Institute’s laboratory of biophysics and bioinformatics.

The principles for the above-mentioned literature search was outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA, see Figure 10) guidelines.

After selection of the publications which were included in the systematic review data were extracted and pooled into table 2 (table 2 can be found in the results part of the thesis) collected data were:

Year; author; Title of the article; Proposed method employment; Mathematical Morphology (MM) appliance in developed algorithm.

Fig. 10. PRISMA flow diagram that’s show evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions. (Adopted from [28]).

PRISMA 2009 Flow Diagram

Records identified through database searching (n = 989) Scr e e n in g In cl u d e d El ig ib il ity Id e n ti fi cat

ion Additional records identified through other sources

(n = 23)

Records after duplicates removed (n = 657)

Records screened (n = 657)

Records excluded (n = 611)

Full-text articles assessed for eligibility

(n = 46)

Full-text articles excluded, with reasons (n = 31) Studies included in qualitative synthesis (n = 15) Studies included in quantitative synthesis (meta-analysis) (n =0)

9. RESULTS

The appearance, behavior and living environment of cells or their compounds can be quite different, therefore image analyze techniques developed to estimate their behavior are also quite different. The most common procedures of analyzed algorithms can be described in Figure 11.

Fig. 11. Main procedures of cell image analysis (Adopted from [29])

In many analyzed article cases we cannot segment cells or their compounds on the raw images directly due to inappropriate image quality. The most common problem is uneven illumination (Figure 12) between different images or in the image itself, image noise and other artefacts can impact and

Fig. 12. Example of uneven illumination (Adopted from (30))

Image pre-processing is done to eliminate the influence of distortions. In analyzed articles

[25,14,22,15,13] MM methods are used to correct this unwanted distortion in the raw images. Golden standard for this operation is Top-hat filtering. This procedure is performed using morphological opening by circle structural element.

It helps to determine background illumination profile of the raw image, which is subtracted from the initial array of the pixel values [20]. The circle radius is selected depending on the analyzed cell or cell compound sizes in the image. Example of the procedure is shown in Figure 13. The use of the Top-hat filtering removes the uneven illumination, reduces noise, and increases the dynamic range of the cell intensity for the setting of a global threshold [25].

Fig. 13. Example of corrected uneven illumination (Adopted from [25])

In majority of analyzed articles, after image pre-processing, images (light microscopy) of cells or their compounds had different intensities (higher) than their surroundings, therefore to segment objects of interest thresholding technique was used. It labels pixels above the intensity threshold as “needed” and the remainder of the image as “background”.

After this procedure image mask (binary image) is created which bright fields can be automatically classified as potential areas of interest and cell segmentation can be computed as we can see in Figure 14.

Fig. 14. Cell image (A) and its binary mask (B) (Adopted from [31])

This segmentation is usually the most important decision which needs to be done during the whole algorithm analysis. In fact, if the cells or their compounds can be properly segmented, the remaining of algorithm operations will be just the basic logic actions. The main problem is the precise detection of perimeter or limits of analyzed objects. We can always delineate an object by manually finding its limits, but this would be inaccurate and time consuming.

However, as much effort and care are put during the processes of preprocessing, thresholding and binary mask creation, the end result often retains imperfections. Threshold tends to output binary masks with particular amount of noise (background elements) in the shape of isolated pixels compounds. This unwanted effect is due to similar gray levels of noise to region of interest (ROI) pixels.

This will automatically bias the result of any kind of measurements done to binary mask region with noise artefacts. Moreover, if there is an overlapping of two objects which have blurred or out of focus edges in the binary mask they can be detected as a single object. We can’t reject a possibility that after all the actions the binary mask will not extract the whole object of interest and it may contain only a part of it or even have holes.

Finally, it is also likely to extract some parts of another object, especially in the corners of the binary masks image, these partial artifacts are surely to distort the end results of the analysis.

The most common MM operations that were used in the analyzed articles are Open, Close, Fill Holes, Skeletonize, and Watershed (a composition of MM operations). If the MM operations are applied in the right order and certain number of iterations, they can be used to eliminate undesired artifacts like undesired pixels, restore partial objects, or separate those which appear contiguous, but really are not in the image and would be interpreted as one by the analysis.

Fig. 15. Examples of Boolean operations (Adopted from [32])

Another set of extremely useful operations which are used together with MM operations mentioned before are called logic or Boolean operations (Figure 15).

When properly used with binary images, these logic operations can identify the pixels that were modified by the MM and if used properly allow to do step by step depuration to get the optimal binary image or mask wanted.

Our analyzed articles used these MM operations together with Boolean operations for cell segmentation and feature enchantment:

 Closing with a spherical SE of special radius followed by a hole filling operation that fills the interior of the cell, left unstained by the actin-specific staining that concentrates at the cortical regions of the cell and object skeletonization [26]. These procedures were crucial for cell and their filopodia segmentation.

 Various radius SE Opening operations [24]. Used for cell tracking, more precisely, for trajectories extraction.

 Dilation operation using various radius structuring elements [25]. Used for cell nuclei segmentation.

 Watershed, Dilation, Erosion, Opening and Closing operations with various structuring elements for characterization of spatial distribution of objects and how they are reorganized [14].

 Nuclei tracking in fluorescence 3D+t images of embryogenesis by using Opening operation with the reconstruction by Dilation operation, using various radius structuring elements, directly in the 4D image. The morphological reconstruction of a marker manually or automatically selected- in an initial spatio-temporal position generates a connected path over the time representing the cell migration [19].

 Adaptive morphological opening, using various radius structuring elements, for preliminary cell segmentation - artifact and noise removal [22].

 Dilation, Hole filling and Erode operations, using various radius structuring elements, for cell and their compound segmentation - artifact and noise removal [27].

 Dilation and Opening operation, using various radius structuring elements, for precise cell segmentation - artifact and noise removal [23].

 Erosion and Dilation, using various radius structuring elements, for precise cell segmentation – artifact and noise removal [16].

 Opening, Hole filling and Closing operations, using various radius structuring elements, for cell and their compound segmentation - artifact and noise removal [20].

 Fill holes and Erosion, using various radius structuring elements, operations for cell segmentation - artifact and noise removal [21].

 Skeletonization operation which helps to detects and characterizes pseudopodial behavior of cells [17].

 Dilation, using various radius structuring elements, operation for cell segmentation and classification [18].

 Erosion, Opening, Dilation, Closing, Hole filling, using various radius structuring elements, and Watershed for cell segmentation and classification [13].

After the identification and segmentation of the objects of interest, selection of the estimates of cell behavior are measured. In analyzed articles authors are measuring different estimates, but we can summarize them to:

 Morphology estimates (perimeter, roundness, size and so on).

 Motility estimates (position (center of mass, perimeter and so on) estimates changes in multiple images).

Table 2: Data from selected publications

Study ID

Study outcome

Year Author Title of the article Proposed method employment MM is used for

2012 A. Korzynska et al. [13]

Multistage morphological segmentation of bright-field and fluorescent microscopy images

Precise particular cell detection Cell segmentation algorithm 2012 Miguel A. Luengo-Oroz et al. [14] 3D + t Morphological Processing: Applications to Embryogenesis Image Analysis

Semi-automatic tracking and segmentation of cells in vivo (light microscopy 3d images)

Image pre-processing and cell segmentation algorithm

2013 T. Ketheesan et al [15].

A Novel Framework for Cellular Tracking and Mitosis Detection in Dense Phase Contrast Microscopy Images

Cell motility indication in large time-lapse phase-contrast image sequences

Cell segmentation algorithm

2018 M. Wei et al. [16]

A real-time detection and positioning method for small and weak targets using a 1D morphology-based

approach in 2D images

A small and weak target detection method in terms of real-time capability

Target detection algorithm

2010 Y. Xiong et al. [17]

Automated characterization of cell shape changes during amoeboid motility by skeletonization

Automatical detection and characterization of pseudopodial behavior of cells

Main technique for

realization of

skeletonization method

2011 F. Mech et al. [18]

Automated Image Analysis of the Host-Pathogen Interaction between Phagocytes and Aspergillus

Fumigatus

Automated image analysis of the Host-Pathogen Interaction between Phagocytes and Aspergillus fumigatus

Cell segmentation and classification algorithm

2009 D. Pastor et al. [19]

Cell tracking in fluorescence images of embryogenesis processes with

morphological reconstruction by 4D-tubular structuring elements

Simple and parameter-free nuclei tracking method for reconstructing cell dynamics in fluorescence 3D+t images of embryogenesis

Image pre-processing and cell detection algorithm

2016 R. Petrolis et al. [20]

Characterization of

gastrointestinal cancer cells invasiveness by estimation of their motility

Quantitative evaluation of cultured cell migration, based on long term continuous no stained gastrointestinal cancer cell imaging

Image pre-processing and cell segmentation algorithm

2015

Klas E. G. Magnusson et al. [21]

Global linking of cell tracks using the Viterbi algorithm

Global track linking algorithm, which links cell outlines generated by a segmentation algorithm into tracks

Cell segmentation algorithm 2014 V. Harma et al [22] Quantification of Dynamic Morphological Drug Responses in 3D Organotypic Cell Cultures by Automated Image Analysis

Streamlined stand-alone software solution that allows quantitative measurements of large numbers of images and structures, with a multitude of different spheroid shapes, sizes, and textures

Image pre-processing

2010 J. Huth et al [23]

Significantly improved precision of cell migration analysis in time-lapse video microscopy through use of a fully automated tracking system

Automatic multi-target tracking system for identifying cell objects and migration rate Cell segmentation algorithm 2012 D. Pastor-Escuredo et al. [24]

Spatio-temporal filtering with morphological operators for robust cell migration estimation in ”in-vivo” images

Tracking algorithm for correction and completion of the cell lineage for trajectories annotation

Main technique for realization of proposed method

2011 M. A. A. Dewan et al. [25]

Tracking Biological Cells in Time-Lapse Microscopy: An Adaptive Technique Motion and Topological Features

Method for automatic tracking of biological cells in time-lapse microscopy by combining the motion features with the topological features of the cells

Image pre-processing and cell detection algorithm

1

2018 C. Castilla et al. [26]

Three-Dimensional

Quantification of Filopodia in Motile Cancer Cells

Full 3D approach to quantitatively analyze single, actin-stained cells with filopodial Protrusions Cell segmentation algorithm 2019 R. Ramonaite et al. [27] Mathematical morphology-based imaging of

gastrointestinal cancer cell motility and 5-aminolevulinic acid-induced fluorescence

Quantitative evaluation of gastrointestinal cancer cell motility and 5-aminolevulinic acid (5-ALA)-induced fluorescence in vitro using mathematical morphology and structural analysis methods

Image pre-processing and cell segmentation algorithm

10. DISCUSSION OF THE RESULTS

In the last decades, a breakthrough in image analysis algorithms for cell behavior analysis was done. Researchers have a great variety of already created tools at their disposal, nevertheless they are worthless if not applied with careful planning. In our analyzed articles MM methods were used to achieve cell behavior quantitative evaluation. MM methodology in image analysis algorithms is used widely and even some combinations of MM operations are considered „golden standard“ for unwanted distortion elimination during the research till now, even though principles of it were published in 1980‘s.

Even though MM has a lot of operations and applications it can‘t identify or measure cell activity „on its own“, threshold filtering, image enhancement techniques and so on; Its should be used together to get the desirable precision. Example of cell identifications using MM operations and thresholding is given in Figure 16, example of identification of wanted radius snowflakes using MM operations and image enhancement techniques in Figure 17.

Fig. 16. Examples of cell segmentation using MM operations together with thresholding (Adopted from [25])

Although MM analysis with other methods shows good results in previously mentioned methods and in analyzed articles some problems still exist. Reliability is one of the biggest concerns in practical use of cell behavior analysis. A fully automatic and 100% reliable method is hardly imaginable due to:

 Existing noise and artifact problematic;  Data and measurement function agreement;

 Data and experiment repetition and parameter stability;  Inaccuracy due to statistical possibilities;

11. CONCLUSIONS

Literature analysis and systematic review of publications showed that the knowledge obtained from cell motility analyses are likely to yield an early abnormality detection and generate new insights into function and malfunction of biological processes, which will eventually lead to reducing deaths and morbidity, provision of prompt advice, and opportunities for innovative treatments.

 The study was focused on mathematical morphology (MM) based cell migration analysis techniques which proved to be a powerful mathematical tool for analyzing and solving problems that occurs during implementation of cell motility analysis algorithms.

 The principles for the literature search were outlined by the preferred reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). All found articles proposed algorithms used MM methods for image enhancement or cell segmentation which are the most important parts of articles proposed method.

 Nevertheless there were no methods found which were using only MM technique for cell motility analysis, without the MM methods none of the proposed algorithms would function properly.

 Although MM analysis together with other methods shows good results in previously mentioned methods and in analyzed articles some problems still exist. Reliability is one of the biggest concerns in practical use of cell behavior analysis. A fully automatic and 100% reliable method is hardly imaginable due to:

 Existing noise and artifact problematic.  Data and measurement function agreement.

 Data and experiment repetition and parameter stability.  Inaccuracy due to statistical possibilities.

12.

REFERENCES

[1] Trepat X, Chen Z, Jacobson K. Cell migration. Comprehensive Physiology. 2012; 2(4): p. 2369-92.

[2] Hiroaki T, Masayuki J, Toshio Y, Masahiro U. Functional Analysis of Spontaneous Cell Movement under Different Physiological Conditions. PLOS ONE. 2008; 3(7): p. 1371-2648.

[3] Krakhmal N, Zavyalova M, Denisov E, Vtorushin S, Perelmuter V. Cancer Invasion: Patterns and Mechanisms. Acta Nature. 2015; 7(2): p. 17-28.

[4] Chen Y, Allen S, Ingram P, Buckanovich R, Merajver S, Yoon E. Single-cell Migration Chip for Chemotaxis-based Microfluidic Selection of Heterogeneous Cell Populations. Scientific Reports. 2015; 5(9980): p. 10-38.

[5] Lee R, Kelley D, Nordstrom K, Ouellette N, Losert W. Quantifying stretching and rearrangement in epithelial sheet migration. New Journal of Physics. 2013; 2(15): p. 1367-2630.

[6] Lee RM, Stuelten CH, Parent CA, Losert W. Collective cell migration over long time scales reveals distinct phenotypes. Convergent Science Physical Oncology. 2016; 2(25001): p. 1739-2057.

[7] Kolchevskiy A, Petrolis R, Kriščiukaitis A. Cell motility evaluation methods in digital imaging: mathematical morphology based estimates of cultured cell behaviour. Virtual instruments in biomedicine 2018: international scientific-practical conference: Klaipėda.(conference was canceled, when article was submitted).

[8] Lintz M, Muñoz A, Reinhart-King C. The Mechanics of Single Cell and Collective Migration of Tumor Cells. Journal of Biomechanical Engineering. 2017; 139(2): p. 021005-1-021005-9.

[9] Horwitz R, Webb D. Cell migration. Current Biology. 2003; 13(19): p. 756-759. [10]

.

Friedl P, Wolf K. Plasticity of cell migration: a multiscale tuning model. Journal of Cell Biology. 2010; 188(1): p. 11-19.

[11] Fisher R, Perkins S, Walker A, Wolfart E. HIPR2-Erosion. [Online].; 2003. Available from: https://homepages.inf.ed.ac.uk/rbf/HIPR2/erode.htm.

[12] Gonzalez RC, Woods RE. PRENHALL. [Online].; 1992. Available from: http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/Digital_Image_Processing_2ndEd. pdf.

[13] Korzynska A, Iwanowski M. Multistage morphological segmentation of bright-field and fluorescent microscopy images. Opto-Electronics Review. 2012; 20: p. 174-186.

[14] Luengo-Oroz M, Pastor-Escuredo D, Castro-Gonzalez C, Faure E, Savy T, Lombardot B, et al. 3D+t morphological processing: applications to embryogenesis image analysis. IEEE Trans Image Process. 2012; 21(8): p. 3518-3530.

[15] Thirusittampalam , Hossain M, Ghita O, Whelan P. A novel framework for cellular tracking and mitosis detection in dense phase contrast microscopy images. IEEE J Biomed Health Inform. 2013; 17(3): p. 642-653.

[16] Wei M, Xing F, You Z. Light Science & Application. [Online].; 2018 [cited 2018 05 4.

Available from:

https://www-ncbi-nlm-nih-gov.ezproxy.dbazes.lsmuni.lt/pmc/articles/PMC6060073/.

[17] Xiong Y, Kabacoff C, Franca-Koh J, Devreotes P, Robinson D, Iglesias P. Automated characterization of cell shape changes during amoeboid motility by skeletonization. BMC Syst Biol. 2010; 33(4): p. 1752-509.

[18] Mech F, Thywissen A, Guthke R, Brakhage A, Figge M. Automated image analysis of the host-pathogen interaction between phagocytes and Aspergillus fumigatus. PLoS One. 2011; 6(5): p. e19591.

[19] Pastor D, Luengo-Oroz M, Lombardot B, Gonzalvez I, Duloquin L, Savy T, et al. Cell tracking in fluorescence images of embryogenesis processes with morphological reconstruction by 4D-tubular structuring elements. IEEE. 2009;: p. 970-973.

[20] Petrolis R, Ramonaitė R, Jocevičius D, Kiudelis G, Kupčinskas L, Kriščiukaitis A. Characterization of Gastrointestinal Cancer Cells Invasiveness by Estimation of Their Motility. In Biomedical Engineering; 2016; Kaunas.

[21] Magnusson K, Jalden J, Gilbert P, Blau H. Global linking of cell tracks using the Viterbi algorithm. IEEE Trans Med Imaging. 2015; 34(4): p. 911-925.

[22] Härmä V, Schukov H, Happonen A, Ahonen I, Virtanen J, Siitari H, et al. Quantification of dynamic morphological drug responses in 3D organotypic cell cultures by automated image analysis. PLoS One. 2014; 9(5): p. e96426.

[23] Huth J, Buchholz M, Kraus J, Schmucker M, von Wichert G, Krndija D, et al. ignificantly improved precision of cell migration analysis in time-lapse video microscopy through use of a fully automated tracking system. BMC Cell Biol. 2010; 24(11): p. 1471-2121.

[24] Pastor-Escuredo D, Luengo-Oroz , Duloquin , Lombardot , Ledesma-Carbayo M, Bourgine P, et al. Spatio-temporal filtering with morphological operators for robust cell migration estimation in ”in-vivo” images. In 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI); 2012; Barcelona. p. 1312-1315.

[25] Dewan M, Ahmad M, Swamy M. Tracking biological cells in time-lapse microscopy: an adaptive technique combining motion and topological features. IEEE. 2011; 58(6): p. 1637-1647.

[26] Castilla C, Maška M, V. Sorokin D, Meijering E, Ortiz-de-Solórzano C. 3-D Quantification of Filopodia in Motile Cancer Cells. IEEE. 2018; 38(3): p. 862 - 872.

[27] Ramonaite R, Petrolis R, Unay S, Kiudelis G, Skieceviciene J, Kupcinskas L, et al. Mathematical morphology-based imaging of gastrointestinal cancer cell motility and 5-aminolevulinic acid-induced fluorescence. Biomed Tech (Berl). 2019; 64(4): p. 711-720.

[28] Moher D, Liberati A, Tetzlaff J, Altman D. Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS med. 2009; 6(7).

[29] Chen S, Zhao M, Wu G, Yao C, Zhang J. Recent advances in morphological cell. Comput Math Methods Med. 2012; 2012(101536): p. 1-10.

[30] Petrolis P, Čižas P, Borutaitė V, Kriščiukaitis A. Method of fluorescence imaging for evaluation of membrane potential in cultured neurons using transmembrane voltage sensitive dye. In Biomedical engineering - 2011 : Proceedings of International Conference; 2011; Kaunas, LT.

[31] Detect Cell Using Edge Detection and Morphology. [Online].; 2020. Available from: https://www.mathworks.com/help/images/detecting-a-cell-using-image-segmentation.html. [32] Schubert Z. Logic gates as Venn diagrams. [Online].; 2010. Available from:

https://www.jeffreythompson.org/blog/2010/04/18/logic-gates-as-venn-diagrams/.

[33] Shannon RR, Ford BJ. Encyclopædia Britannica. [Online].; 2020 [cited 2019 09 12. Available from: https://www.britannica.com/technology/microscope/Confocal-microscopes.

[34] Parker N, Schneegurt M, Thi Tu A, Lister P, Forster BM. OpenStax. [Online].; 2016. Available from: https://openstax.org/books/microbiology/pages/2-1-the-properties-of-light. [35] Giri D. LaboratoryInfo. [Online].; 2020. Available from: https://laboratory

info.com/compound-microscope/.

[36] Hitachi High-Tech. Hitachi Global website. [Online].; 2020. Available from: https://www.hitachi-hightech.com/global/product_detail/?pn=em-hf3300.

[37] Aryal S, Mokobi F, Sharma Y, Neupane L. Online Microbiology Notes. [Online].; 2020. Available from: https://microbenotes.com/electron-microscopy-images-of-sars-cov-2/. [38] Sciences "I"C. "Digital Images." Computer Sciences. [Online].; 2020. Available from:

https://www.encyclopedia.com/computing/news-wires-white-papers-and-books/digital-images.

[39] Gearhart J, Mazeris M, Chen Y, Blazejewski J. Visual Resourources. [Online].; 2020. Available from: https://visualresources.princeton.edu/making-images/digital-image-basics/. [40] Cofield M. Digital Imaging Basics. Texas: Information Technology Lab School of

Information The University of Texas at Austin; 2005.

[41] Sachs J. Digital Light&Color. In Digital Image Basics.; 1996-2003. p. 2-7.

13. ANNEXES

Since this research is focused on reviewing mathematical morphology (MM) based cell motility analysis techniques and selected articles are analyzing digital images that are captured by microscopes, it is important to know basic principles of microscopy and digital images.

13.1- Microscopy:

Microscope is an instrument used to analyze objects that are too small to be seen by the naked eye. Since human cells and other microorganisms are generally quite small (diameter of a human red blood cell is 8 μm) in order to study and observe some abnormal phenomena, we must use microscopes. The main purpose of a microscope is to magnify objects which digital images can be later produced for detailed study. Almost all cell images are taken using microscope and these pictures can be called micrographs [33].

There are two types of microscopes:  Light microscopes.

 Electron microscopes.

Fig. 18. Examples of light (left side) and electron (right side) images (established according to [33])

 Light microscope:

Is the most commonly used and found microscope type in the world. In it visible light passes through the specimen (the biological sample you are analyzing) and is bent through the lens system (Figure 20), allowing the user to see a magnified image. In our study majority of microscopes were equipped with a digital camera allowing observation of a sample using specifically created image analyzing programs and algorithms. Microscopes can also be partly or wholly computer-controlled with various levels of automation [33].

The biggest benefit of light microscopy is that it can be used to spectate (ensuring vital conditions) living cells, so it’s possible to watch cells in their normal behavior (e.g., migrating or dividing).

Majority of microscopes are bright field microscopes, that means visible light is passed through the specimen and used to form an image directly, without any modifications. More sophisticated kinds of light microscopy use optical tricks (e.g. phase-contrast microscopy, ultra-microscope, etc.) to enhance contrast, making details of cells and tissues easier to analyze.

Fig. 19. Fluorescence microscopy image (established according to [34])

One more type of light microscopy which needs to be mentioned and is used often by researchers is fluorescence microscopy as we can see in Figure 19.

This technique is used for specimens that have the ability of fluoresce (absorb one wavelength of light and emit another wavelength).

Light of one wavelength is used to excite the fluorescent molecules of the specimen, and the light of a different wavelength that they emit is observed and often used to form an image. In most cases, the part of a cell or tissue that we want to analyze isn't naturally fluorescent, and instead must be labelled with a fluorescent dye or tag before it goes on the microscope [33].

Fig. 20. Light microscope structure (established according to [35] and [34])

 Electron microscope:

Is a microscope that uses a beam of accelerated electrons as a source of illumination and can produce high-resolution images of structures of very small objects which can‘t be observed with the help of standard light microscopes.

Since the image of a specimen that is being analyzed is produced from using beam of electrons which have a shorter wavelength than visible light electron microscopy can be used to examine not just whole cells, but also the subcellular structures and compartments within them. However, one of the biggest limitations of electron microscopy is that specimens must be placed in vacuum chamber and are typically prepared via an extensive fixation process, which means live cell imaging is impossible [33].

Fig. 21. Electron microscope (Adopted from [36])

You can find basically two main kinds of electron microscopy:

 Scanning electron microscopy (SEM) - a beam of electrons scans the exploration area’s surface by moving back and forth across all the surface of a specimen and creates a very detailed 3D image of the scanned area.

 Transmission electron microscopy (TEM) - the specimen is cut into especially thin slices (ultrathin sections that are less than 100 nm of thick) before scanning, and then electron beam is transmitted through the slice to form the image.

Fig. 22. Electron microscope (SEM and TEM) of Severe Acute Respiratory syndrome

Coronavirus Disease 2019 (COVID-19); apoptotic cell (red) heavily infected with SARS-COV-2 virus particles (yellow), isolated from a patient sample (Adopted from [37])

Electron microscopes are a massive piece of equipment and much more expensive than standardlight microscopes. This is not a surprise if you are not forgeting what particles they have to handle.

The most important parameters in microscope:

 Magnification is a characteristic of a microscope which shows how much larger an object appears in microscope‘s ocular then it’s actual size. For example, light microscopes which are typically used in high schools, colleges and universities magnify up to 400 times actual size. So, an object that was 10 mm wide in real life would be 4000 mm wide in the microscope image

.

 Resolution is a characteristic of a microscope or lens which indicates the smallest distance between two points till which they can be separated and still be validated as two separate objects. The smaller this value, the higher the resolving power of the microscope and the better the clarity and detail of the image. For instance, if two monitored cells are very close to each other on petri dish, they could look like a single, obscure spot on a microscope with low resolving power, but on a microscope with high resolving power they could be separated apart easily.

 Both of the parameters mentioned above (magnification and resolution) are important if you want a quality image of something very miniaturistic. For example, if a microscope has high level of magnification, but low resolution, you will view a bigger version of a blurry image. Different types of microscopes differ in their magnification and resolution. .

13.2- Digital image:

A digital image is a representation of a real image as a set of numbers that can be stored and processed by a computer [38].

Since humans has the ability of color vision (perceive differences between light composed of different wavelengths independently of light intensity) digital color image is usually stored in a way that the picture elements (pixels - smallest single component of a digital image) are sets of integer triplets. Each number of the triplet represents pixels intensity in a certain color channel (image layer). The most common color model of digital image representation and storing is RGB format [39].

The RGB color model is an additive color model in which red (R), green(G), and blue(B) light (in digital image pixel values) are added together in various ways to reproduce a broad array of colors. The name of the model comes from the initials of the three used primary colors, red, green, and blue [39].

The main purpose of the RGB color model is for the representation and display of images in electronic systems, such as smart phones, television and computers. Before the modern age, the RGB color model was already a solid theory, based on human perception of colors and used in conventional photography.

Fig. 23. Spatially sampled digital image (established according to [40])

In Figure 23 we can see LSMU MLK building image sampled to discretize two-dimensional NxM array of elements. The fundamental unit of a sampled image is a picture element and is

X Direction

Y D

irect

ion

typically referred to as a pixel. The value of each pixel (integer triplet) is equal to the average intensity of the continuous spatial image covered by that pixel in image layers. Image size (NxM) values are usually numbers that are powers of two or their combinations, depending on pictures later appliance. There are various digitized images size resolution (NxM) combinations and formats:

 1280×720 : D-VHS, HD DVD, Blu-ray, HDV (miniDV);  1440×1080 : HDV (miniDV);

 1920×1080 : HDV (miniDV), AVCHD, HD DVD, Blu-ray, HDCAM SR;  1998×1080 : 2K Flat (1.85:1);

 2048×1080 : 2K Digital Cinema;

 3840×2160 : 4K UHDTV, Ultra HD Blu-ray;  4096×2160 : 4K Digital Cinema;

 7680×4320 : 8K UHDTV;

 15360×8640 : 16K Digital Cinema;



Fig. 24. RGB image structure (established according to [41])

Since human eye is capable to sense red, green and blue color as we can see in Figure 24, it is theoretically possible to decompose each visible color into combinations of these three primary colors.

If all three color layers have a value of 0, it means that no light is emitted, and the resulting color is black.

If all three color channels are set to their maximum (255) values the resulting color is white. All other layer intensity value combinations available are yielding a color plane cube called RGB color space.