Deep Learning Publications

2022
Yu Y, Safdar S, Bourantas G, Zwick B, Joldes G, Kapur T, Frisken S, Kikinis R, Nabavi A, Golby A, et al. Automatic Framework for Patient-Specific Modelling of Tumour Resection-Induced Brain Shift. Comput Biol Med. 2022;143 :105271.Abstract
Our motivation is to enable non-biomechanical engineering specialists to use sophisticated biomechanical models in the clinic to predict tumour resection-induced brain shift, and subsequently know the location of the residual tumour and its boundary. To achieve this goal, we developed a framework for automatically generating and solving patient-specific biomechanical models of the brain. This framework automatically determines patient-specific brain geometry from MRI data, generates patient-specific computational grid, assigns material properties, defines boundary conditions, applies external loads to the anatomical structures, and solves differential equations of nonlinear elasticity using Meshless Total Lagrangian Explicit Dynamics (MTLED) algorithm. We demonstrated the effectiveness and appropriateness of our framework on real clinical cases of tumour resection-induced brain shift.
Zhang F, Wells WM, O'Donnell LJ. Deep Diffusion MRI Registration (DDMReg): A Deep Learning Method for Diffusion MRI Registration. IEEE Trans Med Imaging. 2022;41 (6) :1454-67.Abstract
In this paper, we present a deep learning method, DDMReg, for accurate registration between diffusion MRI (dMRI) datasets. In dMRI registration, the goal is to spatially align brain anatomical structures while ensuring that local fiber orientations remain consistent with the underlying white matter fiber tract anatomy. DDMReg is a novel method that uses joint whole-brain and tract-specific information for dMRI registration. Based on the successful VoxelMorph framework for image registration, we propose a novel registration architecture that leverages not only whole brain information but also tract-specific fiber orientation information. DDMReg is an unsupervised method for deformable registration between pairs of dMRI datasets: it does not require nonlinearly pre-registered training data or the corresponding deformation fields as ground truth. We perform comparisons with four state-of-the-art registration methods on multiple independently acquired datasets from different populations (including teenagers, young and elderly adults) and different imaging protocols and scanners. We evaluate the registration performance by assessing the ability to align anatomically corresponding brain structures and ensure fiber spatial agreement between different subjects after registration. Experimental results show that DDMReg obtains significantly improved registration performance compared to the state-of-the-art methods. Importantly, we demonstrate successful generalization of DDMReg to dMRI data from different populations with varying ages and acquired using different acquisition protocols and different scanners.
Wang S, Zhang F, Huang P, Hong H, Jiaerken Y, Yu X, Zhang R, Zeng Q, Zhang Y, Kikinis R, et al. Superficial White Matter Microstructure Affects Processing Speed in Cerebral Small Vessel Disease. Hum Brain Mapp. 2022.Abstract
White matter hyperintensities (WMH) are a typical feature of cerebral small vessel disease (CSVD), which contributes to about 50% of dementias worldwide. Microstructural alterations in deep white matter (DWM) have been widely examined in CSVD. However, little is known about abnormalities in superficial white matter (SWM) and their relevance for processing speed, the main cognitive deficit in CSVD. In 141 CSVD patients, processing speed was assessed using Trail Making Test Part A. White matter abnormalities were assessed by WMH burden (volume on T2-FLAIR) and diffusion MRI measures. SWM imaging measures had a large contribution to processing speed, despite a relatively low SWM WMH burden. Across all imaging measures, SWM free water (FW) had the strongest association with processing speed, followed by SWM mean diffusivity (MD). SWM FW was the only marker to significantly increase between two subgroups with the lowest WMH burdens. When comparing two subgroups with the highest WMH burdens, the involvement of WMH in the SWM was accompanied by significant differences in processing speed and white matter microstructure. Mediation analysis revealed that SWM FW fully mediated the association between WMH volume and processing speed, while no mediation effect of MD or DWM FW was observed. Overall, results suggest that the SWM has an important contribution to processing speed, while SWM FW is a sensitive imaging marker associated with cognition in CSVD. This study extends the current understanding of CSVD-related dysfunction and suggests that the SWM, as an understudied region, can be a potential target for monitoring pathophysiological processes.
Cole AP, Langbein BJ, Giganti F, Fennessy FM, Tempany CM, Emberton M. Is Perfect the Enemy of Good? Weighing the Evidence for Biparametric MRI in Prostate Cancer. Br J Radiol. 2022;95 (1131) :20210840.Abstract
The role of multiparametric MRI in diagnosis, staging and treatment planning for prostate cancer is well established. However, there remain several challenges to widespread adoption. One such challenge is the duration and cost of the examination. Abbreviated exams omitting contrast-enhanced sequences may help address this challenge. In this review, we will discuss the rationale for biparametric MRI for detection and characterization of clinically significant prostate cancer prior to biopsy and synthesize the published literature. We will weigh up the advantages and disadvantages to this approach and lay out a conceptual cost/benefit analysis regarding adoption of biparametric MRI.
Zheng J, Yang Q, Makris N, Huang K, Liang J, Ye C, Yu X, Tian M, Ma T, Mou T, et al. Three-Dimensional Digital Reconstruction of the Cerebellar Cortex: Lobule Thickness, Surface Area Measurements, and Layer Architecture. Cerebellum. 2022.Abstract
The cerebellum is ontogenetically one of the first structures to develop in the central nervous system; nevertheless, it has been only recently reconsidered for its significant neurobiological, functional, and clinical relevance in humans. Thus, it has been a relatively under-studied compared to the cerebrum. Currently, non-invasive imaging modalities can barely reach the necessary resolution to unfold its entire, convoluted surface, while only histological analyses can reveal local information at the micrometer scale. Herein, we used the BigBrain dataset to generate area and point-wise thickness measurements for all layers of the cerebellar cortex and for each lobule in particular. We found that the overall surface area of the cerebellar granular layer (including Purkinje cells) was 1,732 cm2 and the molecular layer was 1,945 cm2. The average thickness of the granular layer is 0.88 mm (± 0.83) and that of the molecular layer is 0.32 mm (± 0.08). The cerebellum (both granular and molecular layers) is thicker at the depth of the sulci and thinner at the crowns of the gyri. Globally, the granular layer is thicker in the lateral-posterior-inferior region than the medial-superior regions. The characterization of individual layers in the cerebellum achieved herein represents a stepping-stone for investigations interrelating structural and functional connectivity with cerebellar architectonics using neuroimaging, which is a matter of considerable relevance in basic and clinical neuroscience. Furthermore, these data provide templates for the construction of cerebellar topographic maps and the precise localization of structural and functional alterations in diseases affecting the cerebellum.
Zhou H, Jayender J. EMDQ: Removal of Image Feature Mismatches in Real-Time. IEEE Trans Image Process. 2022;31 :706-20.Abstract
This paper proposes a novel method for removing image feature mismatches in real-time that can handle both rigid and smooth deforming environments. Image distortion, parallax and object deformation may cause the pixel coordinates of feature matches to have non-rigid deformations, which cannot be represented using a single analytical rigid transformation. To solve this problem, we propose an algorithm based on the re-weighting and 1-point RANSAC strategy (R1P-RNSC), which operates under the assumption that a non-rigid deformation can be approximately represented by multiple rigid transformations. R1P-RNSC is fast but suffers from the drawback that local smoothing information cannot be considered, thus limiting its accuracy. To solve this problem, we propose a non-parametric algorithm based on the expectation-maximization algorithm and the dual quaternion-based representation (EMDQ). EMDQ generates dense and smooth deformation fields by interpolating among the feature matches, simultaneously removing mismatches that are inconsistent with the deformation field. It relies on the rigid transformations obtained by R1P-RNSC to improve its accuracy. The experimental results demonstrate that EMDQ has superior accuracy compared to other state-of-the-art mismatch removal methods. The ability to build correspondences for all image pixels using the dense deformation field is another contribution of this paper.
Robles DJ, Dharani A, Rostowsky KA, Chaudhari NN, Ngo V, Zhang F, O'Donnell LJ, Green L, Sheikh-Bahaei N, Chui HC, et al. Older Age, Male Sex, and Cerebral Microbleeds Predict White Matter Loss After Traumatic Brain Injury. Geroscience. 2022;44 (1) :83-102.Abstract
Little is known on how mild traumatic brain injury affects white matter based on age at injury, sex, cerebral microbleeds, and time since injury. Here, we study the fractional anisotropy of white matter to study these effects in 109 participants aged 18-77 (46 females, age μ ± σ = 40 ± 17 years) imaged within [Formula: see text] 1 week and [Formula: see text] 6 months post-injury. Age is found to be linearly associated with white matter degradation, likely due not only to injury but also to cumulative effects of other pathologies and to their interactions with injury. Age is associated with mean anisotropy decreases in the corpus callosum, middle longitudinal fasciculi, inferior longitudinal and occipitofrontal fasciculi, and superficial frontal and temporal fasciculi. Over [Formula: see text] 6 months, the mean anisotropies of the corpus callosum, left superficial frontal fasciculi, and left corticospinal tract decrease significantly. Independently of other predictors, age and cerebral microbleeds contribute to anisotropy decrease in the callosal genu. Chronically, the white matter of commissural tracts, left superficial frontal fasciculi, and left corticospinal tract degrade appreciably, independently of other predictors. Our findings suggest that large commissural and intra-hemispheric structures are at high risk for post-traumatic degradation. This study identifies detailed neuroanatomic substrates consistent with brain injury patients' age-dependent deficits in information processing speed, interhemispheric communication, motor coordination, visual acuity, sensory integration, reading speed/comprehension, executive function, personality, and memory. We also identify neuroanatomic features underlying white matter degradation whose severity is associated with the male sex. Future studies should compare our findings to functional measures and other neurodegenerative processes.
Yu Y, Bourantas G, Zwick B, Joldes G, Kapur T, Frisken S, Kikinis R, Nabavi A, Golby A, Wittek A, et al. Computer Simulation of Tumour Resection-Induced Brain Deformation by a Meshless Approach. Int J Numer Method Biomed Eng. 2022;38 (1) :e3539.Abstract
Tumour resection requires precise planning and navigation to maximise tumour removal while simultaneously protecting nearby healthy tissues. Neurosurgeons need to know the location of the remaining tumour after partial tumour removal before continuing with the resection. Our approach to the problem uses biomechanical modelling and computer simulation to compute the brain deformations after the tumour is resected. In this study, we use Meshless Total Lagrangian Explicit Dynamics (MTLED) as the solver. The problem geometry is extracted from the patient-specific MRI data and includes the parenchyma, tumour, cerebrospinal fluid (CSF) and skull. Appropriate nonlinear material formulation is used. Loading is performed by imposing intra-operative conditions of gravity and reaction forces between the tumour and surrounding healthy parenchyma tissues. A finite frictionless sliding contact is enforced between the skull (rigid) and parenchyma. The meshless simulation results are compared to intra-operative MRI sections. We also calculate Hausdorff distances between the computed deformed surfaces (ventricles and tumour cavities) and surfaces observed intra-operatively. Over 80% of points on ventricle surface and 95% of points on tumour cavity surface were successfully registered (results within the limits of two times the original in-plane resolution of the intra-operative image). Computed results demonstrate the potential for our method in estimating the tissue deformation and tumour boundary during the resection. This article is protected by copyright. All rights reserved.
2021
Schilling KG, Rheault F, Petit L, Hansen CB, Nath V, Yeh F-C, Girard G, Barakovic M, Rafael-Patino J, Yu T, et al. Tractography Dissection Variability: What Happens When 42 Groups Dissect 14 White Matter Bundles on the Same Dataset?. Neuroimage. 2021;243 :118502.Abstract
White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human whole-brain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols for each fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection process.
Abdelmoula WM, Lopez BG-C, Randall EC, Kapur T, Sarkaria JN, White FM, Agar JN, Wells WM, Agar NYR. Peak Learning of Mass Spectrometry Imaging Data Using Artificial Neural Networks. Nat Commun. 2021;12 (1) :5544.Abstract
Mass spectrometry imaging (MSI) is an emerging technology that holds potential for improving, biomarker discovery, metabolomics research, pharmaceutical applications and clinical diagnosis. Despite many solutions being developed, the large data size and high dimensional nature of MSI, especially 3D datasets, still pose computational and memory complexities that hinder accurate identification of biologically relevant molecular patterns. Moreover, the subjectivity in the selection of parameters for conventional pre-processing approaches can lead to bias. Therefore, we assess if a probabilistic generative model based on a fully connected variational autoencoder can be used for unsupervised analysis and peak learning of MSI data to uncover hidden structures. The resulting msiPL method learns and visualizes the underlying non-linear spectral manifold, revealing biologically relevant clusters of tissue anatomy in a mouse kidney and tumor heterogeneity in human prostatectomy tissue, colorectal carcinoma, and glioblastoma mouse model, with identification of underlying m/z peaks. The method is applied for the analysis of MSI datasets ranging from 3.3 to 78.9 GB, without prior pre-processing and peak picking, and acquired using different mass spectrometers at different centers.
Fichtinger G, Mousavi P, Ungi T, Fenster A, Abolmaesumi P, Kronreif G, Ruiz-Alzola J, Ndoye A, Diao B, Kikinis R. Design of an Ultrasound-Navigated Prostate Cancer Biopsy System for Nationwide Implementation in Senegal. J Imaging. 2021;7 (8) :154.Abstract
This paper presents the design of NaviPBx, an ultrasound-navigated prostate cancer biopsy system. NaviPBx is designed to support an affordable and sustainable national healthcare program in Senegal. It uses spatiotemporal navigation and multiparametric transrectal ultrasound to guide biopsies. NaviPBx integrates concepts and methods that have been independently validated previously in clinical feasibility studies and deploys them together in a practical prostate cancer biopsy system. NaviPBx is based entirely on free open-source software and will be shared as a free open-source program with no restriction on its use. NaviPBx is set to be deployed and sustained nationwide through the Senegalese Military Health Service. This paper reports on the results of the design process of NaviPBx. Our approach concentrates on "frugal technology", intended to be affordable for low-middle income (LMIC) countries. Our project promises the wide-scale application of prostate biopsy and will foster time-efficient development and programmatic implementation of ultrasound-guided diagnostic and therapeutic interventions in Senegal and beyond.
Fedorov A, Longabaugh WJR, Pot D, Clunie DA, Pieper S, Aerts HJWL, Homeyer A, Lewis R, Akbarzadeh A, Bontempi D, et al. NCI Imaging Data Commons. Cancer Res. 2021;81 (16) :4188-93.Abstract
The National Cancer Institute (NCI) Cancer Research Data Commons (CRDC) aims to establish a national cloud-based data science infrastructure. Imaging Data Commons (IDC) is a new component of CRDC supported by the Cancer Moonshot{trade mark, serif}. The goal of IDC is to enable a broad spectrum of cancer researchers, with and without imaging expertise, to easily access and explore the value of de-identified imaging data and to support integrated analyses with non-imaging data. We achieve this goal by co-locating versatile imaging collections with cloud-based computing resources and data exploration, visualization, and analysis tools. The IDC pilot was released in October 2020 and is being continuously populated with radiology and histopathology collections. IDC provides access to curated imaging collections, accompanied by documentation, a user forum, and a growing number of analysis use cases that aim to demonstrate the value of a data commons framework applied to cancer imaging research.
Bastos DCDA, Juvekar P, Tie Y, Jowkar N, Pieper S, Wells WM, Bi WL, Golby A, Frisken S, Kapur T. Challenges and Opportunities of Intraoperative 3D Ultrasound With Neuronavigation in Relation to Intraoperative MRI. Front Oncol. 2021;11 :656519.Abstract
Introduction: Neuronavigation greatly improves the surgeons ability to approach, assess and operate on brain tumors, but tends to lose its accuracy as the surgery progresses and substantial brain shift and deformation occurs. Intraoperative MRI (iMRI) can partially address this problem but is resource intensive and workflow disruptive. Intraoperative ultrasound (iUS) provides real-time information that can be used to update neuronavigation and provide real-time information regarding the resection progress. We describe the intraoperative use of 3D iUS in relation to iMRI, and discuss the challenges and opportunities in its use in neurosurgical practice. Methods: We performed a retrospective evaluation of patients who underwent image-guided brain tumor resection in which both 3D iUS and iMRI were used. The study was conducted between June 2020 and December 2020 when an extension of a commercially available navigation software was introduced in our practice enabling 3D iUS volumes to be reconstructed from tracked 2D iUS images. For each patient, three or more 3D iUS images were acquired during the procedure, and one iMRI was acquired towards the end. The iUS images included an extradural ultrasound sweep acquired before dural incision (iUS-1), a post-dural opening iUS (iUS-2), and a third iUS acquired immediately before the iMRI acquisition (iUS-3). iUS-1 and preoperative MRI were compared to evaluate the ability of iUS to visualize tumor boundaries and critical anatomic landmarks; iUS-3 and iMRI were compared to evaluate the ability of iUS for predicting residual tumor. Results: Twenty-three patients were included in this study. Fifteen patients had tumors located in eloquent or near eloquent brain regions, the majority of patients had low grade gliomas (11), gross total resection was achieved in 12 patients, postoperative temporary deficits were observed in five patients. In twenty-two iUS was able to define tumor location, tumor margins, and was able to indicate relevant landmarks for orientation and guidance. In sixteen cases, white matter fiber tracts computed from preoperative dMRI were overlaid on the iUS images. In nineteen patients, the EOR (GTR or STR) was predicted by iUS and confirmed by iMRI. The remaining four patients where iUS was not able to evaluate the presence or absence of residual tumor were recurrent cases with a previous surgical cavity that hindered good contact between the US probe and the brainsurface. Conclusion: This recent experience at our institution illustrates the practical benefits, challenges, and opportunities of 3D iUS in relation to iMRI.
Zhang F, Breger A, Cho KIK, Ning L, Westin C-F, O'Donnell LJ, Pasternak O. Deep Learning Based Segmentation of Brain Tissue from Diffusion MRI. Neuroimage. 2021;233 :117934.Abstract
Segmentation of brain tissue types from diffusion MRI (dMRI) is an important task, required for quantification of brain microstructure and for improving tractography. Current dMRI segmentation is mostly based on anatomical MRI (e.g., T1- and T2-weighted) segmentation that is registered to the dMRI space. However, such inter-modality registration is challenging due to more image distortions and lower image resolution in dMRI as compared with anatomical MRI. In this study, we present a deep learning method for diffusion MRI segmentation, which we refer to as DDSeg. Our proposed method learns tissue segmentation from high-quality imaging data from the Human Connectome Project (HCP), where registration of anatomical MRI to dMRI is more precise. The method is then able to predict a tissue segmentation directly from new dMRI data, including data collected with different acquisition protocols, without requiring anatomical data and inter-modality registration. We train a convolutional neural network (CNN) to learn a tissue segmentation model using a novel augmented target loss function designed to improve accuracy in regions of tissue boundary. To further improve accuracy, our method adds diffusion kurtosis imaging (DKI) parameters that characterize non-Gaussian water molecule diffusion to the conventional diffusion tensor imaging parameters. The DKI parameters are calculated from the recently proposed mean-kurtosis-curve method that corrects implausible DKI parameter values and provides additional features that discriminate between tissue types. We demonstrate high tissue segmentation accuracy on HCP data, and also when applying the HCP-trained model on dMRI data from other acquisitions with lower resolution and fewer gradient directions.
Sedghi A, O'Donnell LJ, Kapur T, Learned-Miller E, Mousavi P, Wells WM. Image Registration: Maximum Likelihood, Minimum Entropy and Deep Learning. Med Image Anal. 2021;69 :101939.Abstract
In this work, we propose a theoretical framework based on maximum profile likelihood for pairwise and groupwise registration. By an asymptotic analysis, we demonstrate that maximum profile likelihood registration minimizes an upper bound on the joint entropy of the distribution that generates the joint image data. Further, we derive the congealing method for groupwise registration by optimizing the profile likelihood in closed form, and using coordinate ascent, or iterative model refinement. We also describe a method for feature based registration in the same framework and demonstrate it on groupwise tractographic registration. In the second part of the article, we propose an approach to deep metric registration that implements maximum likelihood registration using deep discriminative classifiers. We show further that this approach can be used for maximum profile likelihood registration to discharge the need for well-registered training data, using iterative model refinement. We demonstrate that the method succeeds on a challenging registration problem where the standard mutual information approach does not perform well.