The labels have been obtained manually by looking for the prostate within the volumes on the axial, coronal and sagittal planes.
The convention that this software uses to define the 3D basis of the anatomical coor-dinate system is the RSA (Right-Superior-Anterior) defined as:
LP S =
from left towards right from posterior towards anterior from inferior towards superior
(3.2)
which is similar to the LPS, with the first two axes flipped.
For this reason, once the volumetric labels in .nrrd format have been saved, they are flipped to match with the convention used for the volumetric images.
An example of manual annotation is shown in figure 3.2 in green, superimposed on the original image in axial, sagittal and coronal views for a patient belonging to the training set.
Figure 3.2: Manual label superimposed on MRI image in A) axial, B) sagittal and C) coronal views for patient 55.
Figure 3.3: General pipeline of the method: the entire dataset is first preprocessed, then subdivided into training and test set. Training images and labels are used for the training of the 3D and 2D CNNs. Manual 3D annotations are also used to train the ASM model, tested on the output of the 3D CNN. After applying the ASM, 3D labels are restored.
The first step performed on the volumes is the pre-processing, illustrated in detail in figure 3.4.
Figure 3.4: Preprocessing steps: first N4ITK bias field correction, followed by intensity normalization.
Using the pre-processed volume, two tests are carried out in parallel, the first using a 3D CNN model and the second using a 2D model, which will serve as a comparison with the performance of our proposed method.
The input of the 3D model is the set of volumes, rearranged in order to match the dimen-sions required as the network input.
A format conversion has been performed on Python and it is necessary to be able to make use of a function, that will be later explained, to create the Image Data Generator.
For what concerns the 2D network, inputs are two-dimensional slices extracted from each volume and the 2D slices that are output in the test phase, are then stacked together to reobtain the 3D volume.
In order to apply the Active Shape Model, it is necessary to define a set of coincident vertices in all volumes used both for training and for testing the model.
The ASM training is carried out on the volumetric labels obtained manually and consists in the creation of a Mean Shape Model and in the definition of the gray profiles and their derivatives along normals exiting points on the prostate surface.
The model is applied on the volumes obtained as output from the predict phase of the CNN 3D, therefore volumes used as training set and test set.
After applying the model, the volumetric labels are restored from the updated vertices cloud using an algorithm for 3D surface reconstruction to reobtain the volumetric binary masks.
In the results section a comparison is made by evaluating the Dice similarity coefficient and the 95% Hausdorff distance for the train and test volumes in the following cases:
• 3D CNN
• 3D CNN + ASM
• 2D CNN
Pre-processing
Pre-processing is applied in order to improve the quality of the magnetic resonance im-ages by enhancing contrast and making the process of feature recognition easier for the algorithm.
Figure 3.4 shows our pre-processing steps.
• The first operation is called N4 Bias field Correction and it consists in removing the bias field signal that corrupts the MRI image.
This signal, as mentioned in section 2.3.1, can lead to incorrect results if algorithms based on image gray levels are used.
• The second operation consists in normalizing the image intensities according to a technique proposed by Ny`ul and Udupa, which works on the intensities histogram.
Further details on these two processes will be presented in separate sections in the follow-ing, together with a demonstration of their working on the dataset.
Images are finally saved in .mat format to be used by the network.
CNN 3D
A custom made network is used which works with three-dimensional volumes, based on the basic architecture of a UNet and trained from scratch.
The procedure that will be explained in detail in the chapter relative to the CNN 3D will follow these steps:
• Description of the network architecture and blocks used.
• Construction of the Image Data Generator with preparation of the images output from the pre-processing step in order to match the dimensions that the network requires in input.
• Explanation of callback functions and parameters used in the training process.
• Network training and testing.
Active Shape Model
The Active Shape Model is applied to the 3D network output to improve the segmentation along the prostate boundaries.
The proposed method consists of four steps:
• Compute a set of coherent vertices for:
– Manual volumetric labels of the training set
– 3D CNN output volumetric labels of the training set – 3D CNN output volumetric labels of the test set
• Model Training
– Computation of the Shape Model to find the mean shape within the training set and the parameters of the transformation object.
– Computation of the Appearance Data, or rather the gray levels and their derivatives along profiles normal to the prostate surface.
• Model Application, carried out on training and test images.
• Volumetric labels reconstruction
Transform the vertices obtained after applying the model into two-dimensional bi-nary masks that will be stacked together to create the volumetric label.
CNN 2D
To make a comparison with the performances of the 3D network, a basic 2D UNet model has been used, initializing the weights derived from a pre-training on the Ima-geNet dataset.
In the in-depth chapter on CNN 2D the following steps will be further explained:
• Data preparation and construction of the Image Data Generator with data augmen-tation.
• Network architecture with modification to adapt the network dimensions to our study.
• Description of callbacks and parameters used during training.
• Network training and testing.