Survey of Deep Learning in Medical Image Analysis

1. A Survey on Deep Learning in Medical Image Analysis

• low-level pixel processing(edge and line detector filters, region growing , etc...) and analogy with expert systems - the 1970s to the 1990s

• supervised techniques, training data is used to develop a system, feature extraction and the use of statistical classifiers, it was received a title "pattern recognition" , "feature engineering" which is the current name in modern era.

  • a shift from system that are completely designed by humans to systems that are trained by computers using sample data which features vectors are extracted
  • extraction of discriminant features from images - crucial step, handpicked or handcrafted features

• next step, Neural Network, Deep Learning which leads to automation of feature extraction

  • typically, CNN which began from last century 1995 by Lo et al(LeNet LeCun et al 1998)
  • CNN did not gather momentum until various new techniques were developed - watershed 2012, deeper architectures 2014
  • before AlexNet, many feature detection techniques were popular
  • Medical Image Analysis became popular since 2015

  ---

• Overview of Deep Learning methods

  • 

    image.png

• learning algorithms

  • supervised and un-supervised

• Neural Networks

  
  
  

  image.png
  

  image.png

  softmax
  

  

  image.png

  log loss
  

  

  image.png

  • Diff CNN and MLP
    1. The differences between MLP and CNN, CNN wegihts in network are shared which perform convolution operations on images, this way, model does not need to learn separate detectors for the same object, it also drastically reduces the amount of parameters - the number of weights no longer depends on the size of the input images

    2.Second difference is the incorporation of pooling layer, this induces a certain amount of translation invariance and reduces the number of parameters
    3. Adding fully connected layer and softmax function along side the maximum likelihood log loss

• Deep CNN architectures

  • LeNet (LeCun et al. 1998), AlexNet(Krizhevsky et al. 2012)
    both networks were relatively shallow, consisting of 2 and 5 convolutional layers, respectively and employed kernels with large receptive field in early layers which close to the input data and smaller kernel closer to the output, AlexNet incorporates relu instead of tanh
    after year 2012, the novel architecture exploration were took off, deeper models took place. The newer models generally have a lower memory footprint during inference such as VGG19 whichs contains fixed and smaller sized kernels

  • Deeper networks for improving efficiency and reducing amount of parameters.

    1. Szegedy et al. 2014        22 layers GoogleLeNet
    2. Lin et al. 2013                 Inception Network
    3. He et al. 2015                  ResNet
    4. Page 5 quotes

       Since 2014, the performance on the ImageNet benchmark has saturated and it is difficult to assess whether the small increases in performance can really be attributed to BETTER and more sophisticated architectures, the advantage of the lower memory footprint these models provide is typically not as important for Medical Applications
       Consequently, AlexNet or other simple model such as VGG still popular for medical data

  • Multi-stream architectures

    1. Kamnitsas et al. 2017 dual pathway architecture
       • multi-scale image analysis
       • 2.5D classification which are both for medical image processing tasks
    2. to detect the abnormalities, the context is often an important cue, we can feed larger patches to the network, this could significantly increase the amount of parameter, i.e the memory requirements of the network, consequently, the architectures have been investigated where context is added in a down-scaled representation in addition to high resolution local information ...

  • Segmentation architectures

    1. Segmentation is common task for both nature and medical images.

       • sliding-window, input patches from neighboring pixels have huge overlap and same convolutions are computed many times.
       • because the convolution ops similar to dot product therefor we can write the inner products as convolutions and vice versa(consider pixel-wise convs and fully connected layer operation).
       • by rewriting the fully connected layers as convolutions, the CNN can take input images larger than it was trained on and produce a likelihood map, the resulting fully convolutional network - fCNN can then be applied to an entire input image or volume in an efficient fashion...

    2. Ronneberger et al.2015 took the idea of fCNN, and proposed the U-Net architecture. Comprising a regular fCNN followed by an upsampling part where up-convolutions are used to increase the image size, coined contractive and expansive paths.
       Long et al. 2015 skip-connection to directly connect opposing contracting and expanding convolutional layers; Cicek et al. 2016 3D data;
       Milletari et al. 2016b proposed an extension to U-Net which incorporates ResNet-like residual blocks and a Dice loss layer, rather than convolutional cross-entropy which directly minimizes this commonly used segmentation error measure(Consider class imbalance cases)

• Recurrent Neural Networks...(remain unknown which will do in the later future)

• Unsupervised models...(remain unknown which will do in the later future)

• Hardware and software

  • one of the main contributors to steep rise of Deep Learning has been the widespread availability of GPU and GPU-computing libraries(CUDA, OpenCL). GPUs are highly parallel computing engines which have anorder of magnitude more execution threads than central processing unit(CPUs). with GPUs, Deep Learning model is typically 10 to 30 times faster than on CPUs.

  • Caffe, Tensorflow, Theano, Torch
    third-party packages written on top of above tools - keras, Lasagne ...

• Deep Learning uses in Medical Imaging...