DeepReg: a deep learning toolkit for medical image registration

DeepReg (https://github.com/DeepRegNet/DeepReg) is a community-supported open-source toolkit for research and education in medical image registration using deep learning.

A submission to the MICCAI Educational Challenge has utilised the DeepReg code and demos to explore the link between classical algorithms and deep-learning-based methods [5], while a recently published research work investigated temporal changes in prostate cancer imaging, by using a longitudinal registration adapted from the DeepReg code [6].

Statement of need
Currently, popular packages focusing on deep learning methods for medical imaging, such as NiftyNet [7] and MONAI (https://monai.io/), do not support image registration. The existing open-sourced registration projects either implement specific published algorithms without automated testing, such as the VoxelMorph [8], or focus on classical methods, such as NiftiReg [9], SimpleElastix [10] and AirLab [11]. Therefore an open-sourced project focusing on image registration with deep learning is much needed for general research and education purposes.

Implementation
DeepReg implements a framework for unsupervised learning [12,8], weakly-supervised learning [13,14] and their combinations and variants, e.g. [15]. Many options are included for major components of these approaches, such as different image-and label dissimilarity functions, transformation models [16,17,1], deformation regularisation [18] and different neural network architectures [14,19,20]. Details of the implemented methods are described in the documentation. The provided dataset loaders adopt staged random sampling strategy to ensure unbiased learning from groups, images and labels [14,6]. These algorithmic components together with the flexible dataset loaders are building blocks of many other registration tasks, such as group-wise registration and morphological template construction [21,22,23].

DeepReg Demos
In addition to the tutorials and documentation, DeepReg provides a collection of demonstrations, DeepReg Demos, using open-accessible data with real-world clinical applications.

Paired images
Many clinical applications for tracking organ motion and other temporal changes require intra-subject single-modality image registration. Registering lung CT images for the same patient, acquired at expiratory and inspiratory phases [24], is such an example of both unsupervised (without labels) and combined supervision (trained with additional label dissimilarity based on anatomical segmentation). Furthermore, registering prostate MR, acquired before surgery, and intra-operative ultrasound images is an example of weakly-supervised learning for multimodal image registration [14]. Another DeepReg Demo illustrates MR-to-ultrasound image registration is to track tissue deformation and brain tumour resection during neurosurgery [25].

Unpaired images
Unpaired images are found in applications such as single-modality inter-subject registration. One demo registers different brain MR images from different subjects [26], fundamental to population studies. Two other applications align unpaired inter-subject CT images for lung [24] and abdominal organs [27]. Additionally, the support for crossvalidation in DeepReg has been included in a demo, which registers 3D ultrasound images from different prostate cancer patients.

Grouped images
Unpaired images may also be grouped in applications such as single-modality intra-subject registration. In this case, each subject has multiple images acquired, for instance, at two or more time points. For demonstration, multi-sequence cardiac MR images, acquired from myocardial infarction patients [28], are registered, where multiple images within each subject are considered as grouped images. Prostate longitudinal MR registration is proposed to track the cancer progression during active surveillance programme [6]. Using segmentation from this application, another demo application illustrates aligning intra-patient prostate gland masks -also an example of feature-based registration based on deep learning.

Conclusion
DeepReg provides a collection of deep learning algorithms and dataset loaders to train image registration networks, which provides a reference of basic functionalities. In its permissible open-source format, DeepReg not only provides a tool for scientific research and higher education, but also welcomes contributions from wider communities.
is supported by the Natural Sciences and Engineering Research Council of Canada Postgraduate Scholarships-Doctoral Program and the University College London Overseas and Graduate Research Scholarships.