Research Highlights

September, 2017


Deep learning has slowly pervaded every aspect of medical imaging. Recently, DIAG published an extensive review, titled ‘A Survey on Deep Learning in Medical Image Analysis’ in the journal Medical Image Analysis. It covers a significant part of the medical imaging field, ranging from radiology to pathology and ophthalmology. The review is subdivided into four main parts. The first part briefly introduces some general concepts in deep learning and some basic neural network architectures. Subsequently, we discuss several novel and interesting applications of deep learning with respect to specific tasks in medical image analysis, for example detection, segmentation and image generation and enhancement. The third part focuses on different application areas. We give a thorough overview of all papers published for every specific area like brain imaging, digital pathology or color fundus images. Last, we provide some insight on current challenges and opportunities for deep learning in medical image analysis and shed some light on the potential application of novel architectures like generative adversarial networks and variational auto-encoders. We hope the paper can function as a primer for both medical image analysis researchers interested in applying deep learning algorithms to their work and computer scientists who want to ventures into medical imaging. You can download the survey from the following sites: arXiv and MEDIA

March, 2017


Jeroen van der Laak has contributed to a news article of the NOS about the success of a deep learning-based algorithm to automatically detect malignant lymph tissue in pathology slides. The algorithm, trained on data provided by the DIAG group, outperforms pathologists on this task. More info can be found on the Google research Blog and in the resulting paper.

November, 2016

Nov16 Highlight.jpg

Click here to view a short AudioSlides presentation of Thomas van den Heuvel about the automatic detection of cerebral microbleeds in patients with traumatic brain injury. The presence of Cerebral Microbleeds (CMBs) may have prognostic value in Traumatic Brain Injury (TBI) patients. However, manually annotating CMBs in TBI patients is a time consuming task that is prone to errors, because CMBs are easily overlooked and are difficult to distinguish from blood vessels. A Computer Aided Detection system was developed that automatically detects CMBs in TBI patients. This work has been published in NeuroImage: Clinical.

October, 2016


Ajay Patel, Midas Meijs and Sil van de Leemput (4DCT group) have combined their latest results into one rendering (see above). The rendering is based on cranial cavity segmentation, vessel segmentation and white matter/gray matter/cerebrospinal fluid segmentation. This image has been submitted to this year's RSNA image contest in the category 'best medical image'. Voting will take place until October 31st and can be done via this link.

August, 2016

Highlight August.png

Reliable breast density measurement is needed to personalize screening. Katharina Holland investigated the consistency of BI-RADS density categories (1 to 4) in serial screening mammograms and compared the results to the automated breast density measurements. Less density category changes occurred with automated assessment than with human assessment. The image shows a prior/current mammogram and the density scores given by the readers and the software. Her work has been published in The Breast.

June, 2016

Highlight June.jpg

MR Lymphography (MRL) is the most accurate imaging modality for the assessment of lymph node metastases in prostate cancer patients, but the interpretation of MRL images is a very time-consuming task for the radiologist. Oscar Debats improved a computer-aided detection system using anatomical information from multi-atlas registration. The new system finds small lymph nodes automatically. The figure shows: an example MRL slice (left), the old and new lymph node likelihood map (middle and right). The new map is much more accurate. His work appeared in the June 2016 issue of Medical Physics.

May, 2016


Convolutional neural networks (CNNs) are network architectures that are becoming increasingly popular in medical image analysis, but are computationally expensive to train. Mark van Grinsven has developed a method to improve and speed up the CNN training. The method was applied to the automatic detection of hemorrhages in color fundus images. The Figures show an example case (left), the annotations made by a human expert (middle) and the output of the automatic system (right). His work has been published in the Special Issue on Deep Learning of Transactions on Medical Imaging.

March, 2016

Example current priornegative 1x2 v2.png

Breast cancer lesions might be overlooked or misinterpreted in breast screening programs with MRI. Albert Gubern-Mérida developed an automated system which is able to detect breast cancer lesions in MRI scans that were thought to be negative. The figure shows a cancer on the left breast that was detected during MRI screening in the current scan, but was missed in the previous screening round. The automated system was able to detect the cancer (red box) in both current and prior examinations. This work has been published in the European Journal of Radiology and was the topic of an article in AuntMinnie.

February, 2016


Detection of change between consecutive low-dose CT images is crucial in lung cancer screening. Visual comparison of CT scans is tedious and hence, automatic detection of change may aid human readers. Colin Jacobs developed an automatic system for detecting change between low-dose CT images using subtraction images. The figure shows a growing part-solid nodule with the current scan on the left, the prior scan on the right, and the subtraction image in the middle. This work was presented at the RSNA conference in 2015 and was the topic of an article on AuntMinnie.

September, 2015


Histopathology involves microscopic examination of stained histological slides to study presence and characteristics of disease. Tissue sections are stained with multiple contrasting dyes to highlight different tissue structures and cellular features. This staining provides invaluable information to the pathologists for diagnosing and characterizing various pathological conditions. Computer-aided diagnosis (CAD) can potentially alleviate shortcomings of human interpretation. However, variations in the color and intensity of hematoxylin and eosin (H&E) stained histological slides can potentially hamper the effectiveness of quantitative image analysis. Babak Ehteshami Bejnordi proposed an algorithm for standardizing whole-slide histopathological images. The proposed method is based on transformation of the chromatic and density distributions for each individual stain class in the hue-saturation-density (HSD) color model. The results of the standardization performed by the proposed algorithm is shown in the figure. The image shown in the top left was used as the template image. The images on the second row are example images that were stained in different laboratories. The standardized versions of these images are presented in the third row. More...

June, 2015


CT scans are three-dimensional images, reconstructed from many different projection images. Manufacturers of CT scanner have different software for reconstructing the images from the projection data, with many different settings, so the resulting images can look smooth (top left) or a bit sharper (top right). If you want to quantify disease with simple procedures like thresholding, as is common practice to get a measure of the severity of emphysema, the results (the amount of colored pixels) are very dependent on the reconstruction settings. Leticia Gallardo Estrella proposed a simple procedure to standardize the sharpness of the reconstructed scans. The bottom row shows the standardized images, computed from the image directly above. Now the amount of dark pixels is very comparable. We use this procedure to obtain more objective measurements of emphysema in studies with data from different scanners and different reconstruction settings. More...

January, 2015


Francesco Ciompi developed a general scale-invariant and rotation-invariant descriptor for objects in 3D images called Bag of Frequencies. He showed that this descriptor can be used to distinguish between true pulmonary nodules and false positives detected by a computer system to find nodules automatically, and that the descriptor can be used to classify which nodules are spiculated, an indication of malignancy. More...

February, 2014

3D renderingsDIAG680.png

The human lungs consist of five parts, the lobes (the right lung has three lobes, the left lung two). Segmenting these lobes in CT scans of the lungs is not a simple task. Bianca Lassen developed an automatic method to precisely delineate the lobes. She evaluated the method on 55 scans for a publicly available data set LOLA11. The renderings above illustrate the results on eight of these scans. More...

March, 2013


Computer-aided detection is expected to play an important role to facilitate the reading of automated 3D breast ultrasound images that are increasingly used in breast cancer screening. To reduce the number of false positive detections outside of the breast, Tao Tan published a method to automatically locate the chest wall in Medical Image Analysis. The left images show manually annotated points on the surface of the ribs in the coronal and sagittal plane. The crosses represent the manually annotated points on the rib surface, projected on the current slice. Annotated points on the current slice are represented by pink crosses, and projected crosses are red. The right images show a scan with dark shadow enhancement overlay on the coronal and sagittal plane. More...

February, 2013


Lung cancer, by far the most deadly cancer worldwide, usually becomes symptomatic only when it is already advanced. With low dose CT scanning, lung cancer can be detected in an early stage, when it can still be treated successfully. Bram van Ginneken has been awarded a 1.5 million Euro VICI grant, in the NWO Vernieuwingsimpuls programme, for his proposal Lung CT Screening: More for Less. The goal of this project is to automate the reading of lung screening CT scans as much as possible, using computer detection algorithms and automatic volumetric segmentation of lung nodules, as illustrated above for one lung nodule that grows over a period of three years. From this analysis the probability that a suspicious lesions represents lung cancer can be accurately estimated and appropriate work up for the patient can be determined. We will also develop an automatic computer algorithm to estimate risk for cardiovascular and chronic obstructive lung disease from lung CT screening scans. All this information can be combined by an expert system to make a personal recommendation for the screening interval: not everybody needs a yearly CT. In this way we hope it will be possible to make screening both more effective and less costly.

January, 2013


Prostate cancer is the second most common cause of cancer death in men. Image registration tools are commonly used for image-guided interventions in prostate cancer. Wendy van de Ven extended a non-rigid surface-based registration method with biomechanical modeling usable for e.g. MR guided TRUS biopsies. By using biomechanical modeling the internal prostatic deformation can be better controlled than with a regular surface-based registration method. The left image shows a T2-weighted MR image of the prostate before deformation with internal anatomical prostate landmarks in blue. The middle image shows the prostate after deformation with the real positions of the corresponding landmarks indicated in green and the registered landmarks in red after a regular non-rigid surface-based registration. The right image shows the result obtained after applying a non-rigid surface-based registration with biomechanical regularization. The registration error was significantly smaller when extending a surface-based prostate registration method with a biomechanical model. More...

December, 2012


The human lungs are divided into lobes, which are separated by a double layer of visceral pleura called the lobar fissures. These fissures are often incomplete, and it has been found that certain new treatments are less effective when this is the case. Measuring fissural completeness is therefore important, but visual assessment is time-consuming and tedious. Eva van Rikxoort developed a method to automatically detect the fissures and quantify their completeness from chest CT scans. The left image shows a coronal slice of a chest CT scan, the image on the right shows a visualization of the fissure completeness, where the detected fissure is indicated in yellow and the lobar boundary that is not delineated by a fissure is indicated in red. The automatic fissure completeness was tested on subjects with COPD and shown to perform as good as experienced radiologists. More...

August, 2012


If you take a picture and what you expect to see is not there, it could mean that something is wrong. Or... you took the picture at the wrong moment! In CT angiography, where contrast is injected in the blood vessels to see if a blood vessel is occluded, poor timing of the moment the 3D scan is acquired can lead to the wrong diagnosis. In the CTA scan on the left, the missing vessel at the location of the arrow may be completely occluded. Ewoud Smit developed a technique to derive a timing-invariant CTA (TI-CTA) from a 4D scan that makes a movie while the contrast enters and leaves the brain. On the TI-CTA, shown on the right, we can see that the vessel is not occluded, but apparently the timing of the CTA was wrong and the contrast arrived a little later. Smit's technique and the advantages it brings are presented in his recent Radiology paper. More...

May, 2012


Prostate cancer is the second most common cause of cancer death in men. At DIAG we are developing a CAD system that can detect prostate cancer in MRI studies. The left image shows an image from a typical T2-weighted MR series where a cancer is circled. The center image shows initial cancer likelihood on a per voxel basis. The right image shows the final output of the CAD system, where the cancer is segmented and a probability is given. This work was presented by Geert Litjens at the SPIE Medical Imaging Conference in February, 2012. More...

January, 2012


Tuberculosis is still a large healthcare problem in the world. The CAD4TB group in the Diagnostic Image Analysis Group is developing a CAD system for tuberculosis on chest radiographs. The left image shows a small lesion in the upper right lobe, the right image its corresponding detection by the CAD system. Laurens Hogeweg evaluated this system on a database of radiographs of homeless people from London, UK. This work was presented at the RSNA conference in 2011 and covered by AuntMinnie. More...

December, 2011


Mammographic breast density is a strong risk factor for breast cancer. Most studies measure breast density subjectively with a semi-automatic threshold method through a software package named Cumulus (middle image). Michiel Kallenberg developed a completely automatic method (right image) to assess breast density that corresponds excellently with Cumulus. More...

November, 2011

MRI LymphNodeSegmentation.png

Magnetic Resonance Lymphography (MRL) is a promising new imaging technique for the detection of lymph node metastases. Oscar Debats developed two new methods for lymph node segmentation in MRL images. Two example lymph nodes are shown above, in coronal (cor), sagittal (sag), and transversal (tra) view. The two new methods, called ECC and PSAM, closely resemble the manual segmentations while existing methods tend to 'leak' out of the nodes, as shown here for the CCRG and GCS methods. More...

July, 2011

Retinal featured image.jpg
Color fundus images are widely used for screening and diagnosis of diabetic retinopathy. This task involves the detection and quantification of retinal lesions, such as hemorrhages and hard exudates. On the left a color fundus image with such lesions is displayed. Clarisa Sánchez developed a computer-aided diagnosis scheme that automatically detects retinal lesions on color fundus images, shown in the color overlay on the right, and determines if the patient should be referred to a specialist. More...

February, 2011

Automated 3D breast ultrasound (ABUS) is a new imaging technique that can help to detect early breast cancer. On the left a malignant lesion imaged with ABUS is displayed in coronal (top) and transversal view (bottom). Especially in the coronal view, spiculation can be observed. Tao Tan developed a computer-aided diagnosis scheme that computes a spiculation feature map, shown in color overlays on the right, and from this map determines the probability that a lesion is malignant. The system obtained very promising results in a dataset of 40 lesions including 20 cancers. More...

December, 2010

Geert Litjens has developed a method to simulate nodules on chest radiographs. Such nodules can be lung cancer and should not be missed. Computer-aided detection schemes may be improved if they can be trained with high quality simulated nodules. Two of the four cases shown above are simulated. Click here to find out which ones. More...

September, 2010

On the left a normal chest radiograph. On the right the same radiograph, but with part of the fifth through the nineth posterior rib suppressed using a technique developed by Laurens Hogeweg. Suppressing the ribs makes it easier to analyze the texture of the lung parenchyma. More...