HOME / RESEARCH /

Computation Core

William Wells Bruno Madore
William Wells, PhD
Core Lead 		Bruno Madore, PhD
Project Lead

The computation project is leveraging recent progress in ultrasound-ultrasound (US) registration and in hybrid US-MRI technology to develop synergistic software and hardware technology that is aimed at improving surgical and interventional guidance in the presence of tissue deformation or motion, issues that complicate treatment monitoring or comparisons to pre-operative images and treatment plans.  Our approach to addressing deformation problems in image guided therapy (IGT) leverages our recent work in feature-based US-US registration, where image content is modeled in terms of local scale-invariant image features, i.e., distinctive patterns of echogenic anatomical tissue that can be automatically extracted from images and used as the basis for registration. Our solution for motion in IGT is built upon our recent developments in hybrid US-MRI technology that acquires MRI and ultrasound simultaneously to exploit the relative strengths of MRI (high spatial resolution and excellent soft tissue contrast), and US (high frame rate). Much of the proposed research deals with providing solutions to registration problems for IGT applications, such as tissue deformation fields, and we believe that in this context it is important to characterize the potential uncertainties in these solutions, similarly to providing error bars in other estimation problems.To this end we are developing registration-with-uncertainty algorithms that incorporate random process models of spatial uncertainty. The technology is evaluated in the context of our testbed clinical projects, image-guided neurosurgery and abdominal cryotherapy, in the AMIGO suite, our advanced interventional suite that includes intra-operative 3T MRI, ultrasound and PET/CT. The hybrid US-MRI approach enables rapid updates to MRI images to accommodate, e.g., breathing motions during cryoablation procedures.In addition, US-US registration algorithms facilitate improvements in US-updated neurosurgical guidance, and have potential IGT applications in our program or elsewhere, for example in prostate biopsies. In order to facilitate dissemination of these algorithms to the broader IGT community, we distribute software components in the open-source SlicerIGT platform. Our projects are:

Registration algorithms for MRI and US with emphasis on uncertainty and algorithm performance. We continue algorithm developments aimed at characterizing uncertainty and accuracy in image registration,and tissue deformation estimation from implanted trackers,that are based on Gaussian Random Fields (GRF). We are also developing algorithms that estimate surgical tissue deformations from our feature-based ultrasound / ultrasound registration technology. Finally, we translate the developed algorithms into AMIGO using the SlicerIGT platform by providing extensions that visualize deformed MRI based on intraoperative US, associated registration uncertainty, and integrated laser surface scanning for neurosurgery. (Contact: William Wells)

Technology for simultaneous US-MRI acquisition for monitoring procedures. We are developing machine learning techniques that use high bandwidth US data to estimate motion and deformation in MRI images. We are also further generalizing the hybrid US-MRI approach by exploiting information from 256 independent channels, from a custom-built MR-compatible 256-element 2D US transducer array provided by an industrial partner. We are developing a pre-scan calibration (“learning”) phase that employs simultaneously-acquired MRI and USdata. We will deploy on-line deformation-corrected updates of MR as they become available from the scanner, for monitoring cryoablations. (Contact: Bruno Madore)

Software and Documentation

3D Slicer, a comprehensive open source platform for medical image analysis, contains several modules and functions that have been contributed by us for Computation. These include: Source Code for the Paper Titled: Efficient and Robust Model-to-Image Alignment using 3D Scale-Invariant Features (Med Image Anal. 2013 Apr;17(3):271-82.)

Data

MRI acquired to guide Gynecologic Brachytherapy Catheter Placement

Links

3D Slicer

Full Publication List

In NIH/NLMdatabase and in our Abstracts Database.

Select Recent Publications

Wang J, Wells WM, Golland P, Zhang M. Registration Uncertainty Quantification via Low-dimensional Characterization of Geometric Deformations. Magn Reson Imaging. 2019;64 :122-31.Abstract
This paper presents an efficient approach to quantifying image registration uncertainty based on a low-dimensional representation of geometric deformations. In contrast to previous methods, we develop a Bayesian diffeomorphic registration framework in a bandlimited space, rather than a high-dimensional image space. We show that a dense posterior distribution on deformation fields can be fully characterized by much fewer parameters, which dramatically reduces the computational complexity of model inferences. To further avoid heavy computation loads introduced by random sampling algorithms, we approximate a marginal posterior by using Laplace's method at the optimal solution of log-posterior distribution. Experimental results on both 2D synthetic data and real 3D brain magnetic resonance imaging (MRI) scans demonstrate that our method is significantly faster than the state-of-the-art diffeomorphic registration uncertainty quantification algorithms, while producing comparable results.
Cheng C-C, Preiswerk F, Madore B. Multi-pathway Multi-echo Acquisition and Neural Contrast Translation to Generate a Variety of Quantitative and Qualitative Image Contrasts. Magn Reson Med. 2020;83 (6) :2310-21.Abstract
PURPOSE: Clinical exams typically involve acquiring many different image contrasts to help discriminate healthy from diseased states. Ideally, 3D quantitative maps of all of the main MR parameters would be obtained for improved tissue characterization. Using data from a 7-min whole-brain multi-pathway multi-echo (MPME) scan, we aimed to synthesize several 3D quantitative maps (T and T ) and qualitative contrasts (MPRAGE, FLAIR, T -weighted, T -weighted, and proton density [PD]-weighted). The ability of MPME acquisitions to capture large amounts of information in a relatively short amount of time suggests it may help reduce the duration of neuro MR exams. METHODS: Eight healthy volunteers were imaged at 3.0T using a 3D isotropic (1.2 mm) MPME sequence. Spin-echo, MPRAGE, and FLAIR scans were performed for training and validation. MPME signals were interpreted through neural networks for predictions of different quantitative and qualitative contrasts. Predictions were compared to reference values at voxel and region-of-interest levels. RESULTS: Mean absolute errors (MAEs) for T and T maps were 216 ms and 11 ms, respectively. In ROIs containing white matter (WM) and thalamus tissues, the mean T /T predicted values were 899/62 ms and 1139/58 ms, consistent with reference values of 850/66 ms and 1126/58 ms, respectively. For qualitative contrasts, signals were normalized to those of WM, and MAEs for MPRAGE, FLAIR, T -weighted, T -weighted, and PD-weighted contrasts were 0.14, 0.15, 0.13, 0.16, and 0.05, respectively. CONCLUSIONS: Using an MPME sequence and neural-network contrast translation, whole-brain results were obtained with a variety of quantitative and qualitative contrast in ~6.8 min.
Luo J, Sedghi A, Popuri K, Cobzas D, Zhang M, Preiswerk F, Toews M, Golby A, Sugiyama M, Wells WIIIM, et al. On the Applicability of Registration Uncertainty, in MICCAI 2019. Vol LNCS 11765. Shenzhen, China: Springer ; 2019 :410-9.Abstract
Luo MICCAI 2019
Estimating the uncertainty in (probabilistic) image registration enables, e.g., surgeons to assess the operative risk based on the trustworthiness of the registered image data. If surgeons receive inaccurately calculated registration uncertainty and misplace unwarranted confidence in the alignment solutions, severe consequences may result. For probabilistic image registration (PIR), the predominant way to quantify the registration uncertainty is using summary statistics of the distribution of transformation parameters. The majority of existing research focuses on trying out different summary statistics as well as means to exploit them. Distinctively, in this paper, we study two rarely examined topics: (1) whether those summary statistics of the transformation distribution most informatively represent the registration uncertainty; (2) Does utilizing the registration uncertainty always be beneficial. We show that there are two types of uncertainties: the transformation uncertainty, Ut, and label uncertainty Ul. The conventional way of using Ut to quantify Ul is inappropriate and can be misleading. By a real data experiment, we also share a potentially critical finding that making use of the registration uncertainty may not always be an improvement.
Luo J, Toews M, Machado I, Frisken S, Zhang M, Preiswerk F, Sedghi A, Ding H, Pieper S, Golland P, et al. A Feature-Driven Active Framework for Ultrasound-Based Brain Shift Compensation, in MICCAI 2018. Vol LNCS 11073. Springer ; 2018 :30-38.Abstract
Luo MICCAI 2018
A reliable Ultrasound (US)-to-US registration method to compensate for brain shift would substantially improve Image-Guided Neurological Surgery. Developing such a registration method is very challenging, due to factors such as the tumor resection, the complexity of brain pathology and the demand for fast computation. We propose a novel feature-driven active registration framework. Here, landmarks and their displacement are first estimated from a pair of US images using corresponding local image features. Subsequently, a Gaussian Process (GP) model is used to interpolate a dense deformation field from the sparse landmarks. Kernels of the GP are estimated by using variograms and a discrete grid search method. If necessary, the user can actively add new landmarks based on the image context and visualization of the uncertainty measure provided by the GP to further improve the result. We retrospectively demonstrate our registration framework as a robust and accurate brain shift compensation solution on clinical data.
Machado I, Toews M, George E, Unadkat P, Essayed W, Luo J, Teodoro P, Carvalho H, Martins J, Golland P, et al. Deformable MRI-Ultrasound Registration using Correlation-based Attribute Matching for Brain Shift Correction: Accuracy and Generality in Multi-site Data. Neuroimage. 2019;202 :116094.Abstract
Intraoperative tissue deformation, known as brain shift, decreases the benefit of using preoperative images to guide neurosurgery. Non-rigid registration of preoperative magnetic resonance (MR) to intraoperative ultrasound (US) has been proposed as a means to compensate for brain shift. We focus on the initial registration from MR to predurotomy US. We present a method that builds on previous work to address the need for accuracy and generality of MR-iUS registration algorithms in multi-site clinical data. To improve accuracy of registration, we use high-dimensional texture attributes instead of image intensities and propose to replace the standard difference-based attribute matching with correlation-based attribute matching. We also present a strategy that deals explicitly with the large field-of-view mismatch between MR and iUS images. We optimize key parameters across independent MR-iUS brain tumor datasets acquired at three different institutions, with a total of 43 tumor patients and 758 corresponding landmarks to validate the registration algorithm. Despite differences in imaging protocols, patient demographics and landmark distributions, our algorithm was able to reduce landmark errors prior to registration in three data sets (5.37 ± 4.27, 4.18 ± 1.97 and 6.18 ± 3.38 mm, respectively) to a consistently low level (2.28 ± 0.71, 2.08 ± 0.37 and 2.24 ± 0.78 mm, respectively). Our algorithm is compared to 15 other algorithms that have been previously tested on MR-iUS registration and it is competitive with the state-of-the-art on multiple datasets. We show that our algorithm has one of the lowest errors in all datasets (accuracy), and this is achieved while sticking to a fixed set of parameters for multi-site data (generality). In contrast, other algorithms/tools of similar performance need per-dataset parameter tuning (high accuracy but lower generality), and those that stick to fixed parameters have larger errors or inconsistent performance (generality but not the top accuracy). We further characterized landmark errors according to brain regions and tumor types, a topic so far missing in the literature. We found that landmark errors were higher in high-grade than low-grade glioma patients, and higher in tumor regions than in other brain regions.
Frisken S, Luo M, Machado I, Unadkat P, Juvekar P, Bunevicius A, Toews M, Wells WM, Miga MI, Golby AJ. Preliminary Results Comparing Thin Plate Splines with Finite Element Methods for Modeling Brain Deformation during Neurosurgery using Intraoperative Ultrasound. Proc SPIE Int Soc Opt Eng. 2019;10951.Abstract
Brain shift compensation attempts to model the deformation of the brain which occurs during the surgical removal of brain tumors to enable mapping of presurgical image data into patient coordinates during surgery and thus improve the accuracy and utility of neuro-navigation. We present preliminary results from clinical tumor resections that compare two methods for modeling brain deformation, a simple thin plate spline method that interpolates displacements and a more complex finite element method (FEM) that models physical and geometric constraints of the brain and its material properties. Both methods are driven by the same set of displacements at locations surrounding the tumor. These displacements were derived from sets of corresponding matched features that were automatically detected using the SIFT-Rank algorithm. The deformation accuracy was tested using a set of manually identified landmarks. The FEM method requires significantly more preprocessing than the spline method but both methods can be used to model deformations in the operating room in reasonable time frames. Our preliminary results indicate that the FEM deformation model significantly out-performs the spline-based approach for predicting the deformation of manual landmarks. While both methods compensate for brain shift, this work suggests that models that incorporate biophysics and geometric constraints may be more accurate.
Kocev B, Hahn HK, Linsen L, Wells WM, Kikinis R. Uncertainty-aware Asynchronous Scattered Motion Interpolation using Gaussian Process Regression. Comput Med Imaging Graph. 2019;72 :1-12.Abstract
We address the problem of interpolating randomly non-uniformly spatiotemporally scattered uncertain motion measurements, which arises in the context of soft tissue motion estimation. Soft tissue motion estimation is of great interest in the field of image-guided soft-tissue intervention and surgery navigation, because it enables the registration of pre-interventional/pre-operative navigation information on deformable soft-tissue organs. To formally define the measurements as spatiotemporally scattered motion signal samples, we propose a novel motion field representation. To perform the interpolation of the motion measurements in an uncertainty-aware optimal unbiased fashion, we devise a novel Gaussian process (GP) regression model with a non-constant-mean prior and an anisotropic covariance function and show through an extensive evaluation that it outperforms the state-of-the-art GP models that have been deployed previously for similar tasks. The employment of GP regression enables the quantification of uncertainty in the interpolation result, which would allow the amount of uncertainty present in the registered navigation information governing the decisions of the surgeon or intervention specialist to be conveyed.
Ciris PA, Chiou J-yuan G, Glazer DI, Chao T-C, Tempany-Afdhal CM, Madore B, Maier SE. Accelerated Segmented Diffusion-Weighted Prostate Imaging for Higher Resolution, Higher Geometric Fidelity, and Multi-b Perfusion Estimation. Invest Radiol. 2019;54 (4) :238-46.Abstract
PURPOSE: The aim of this study was to improve the geometric fidelity and spatial resolution of multi-b diffusion-weighted magnetic resonance imaging of the prostate. MATERIALS AND METHODS: An accelerated segmented diffusion imaging sequence was developed and evaluated in 25 patients undergoing multiparametric magnetic resonance imaging examinations of the prostate. A reduced field of view was acquired using an endorectal coil. The number of sampled diffusion weightings, or b-factors, was increased to allow estimation of tissue perfusion based on the intravoxel incoherent motion (IVIM) model. Apparent diffusion coefficients measured with the proposed segmented method were compared with those obtained with conventional single-shot echo-planar imaging (EPI). RESULTS: Compared with single-shot EPI, the segmented method resulted in faster acquisition with 2-fold improvement in spatial resolution and a greater than 3-fold improvement in geometric fidelity. Apparent diffusion coefficient values measured with the novel sequence demonstrated excellent agreement with those obtained from the conventional scan (R = 0.91 for bmax = 500 s/mm and R = 0.89 for bmax = 1400 s/mm). The IVIM perfusion fraction was 4.0% ± 2.7% for normal peripheral zone, 6.6% ± 3.6% for normal transition zone, and 4.4% ± 2.9% for suspected tumor lesions. CONCLUSIONS: The proposed accelerated segmented prostate diffusion imaging sequence achieved improvements in both spatial resolution and geometric fidelity, along with concurrent quantification of IVIM perfusion.
Yengul SS, Barbone PE, Madore B. Dispersion in Tissue-Mimicking Gels Measured with Shear Wave Elastography and Torsional Vibration Rheometry. Ultrasound Med Biol. 2019;45 (2) :586-604.Abstract
Dispersion, or the frequency dependence of mechanical parameters, is a primary confounding factor in elastography comparisons. We present a study of dispersion in tissue-mimicking gels over a wide frequency band using a combination of ultrasound shear wave elastography (SWE), and a novel torsional vibration rheometry which allows independent mechanical measurement of SWE samples. Frequency-dependent complex shear modulus was measured in homogeneous gelatin hydrogels of two different bloom strengths while controlling for confounding factors such as temperature, water content and material aging. Furthermore, both techniques measured the same physical samples, thereby eliminating possible variation caused by batch-to-batch gel variation, sample geometry differences and boundary artifacts. The wide-band measurement, from 1 to 1800 Hz, captured a 30%-50% increase in the storage modulus and a nearly linear increase with frequency of the loss modulus. The magnitude of the variation suggests that accounting for dispersion is essential for meaningful comparisons between SWE implementations.
Mehrtash A, Ghafoorian M, Pernelle G, Ziaei A, Heslinga FG, Tuncali K, Fedorov A, Kikinis R, Tempany CM, Wells WM, et al. Automatic Needle Segmentation and Localization in MRI with 3D Convolutional Neural Networks: Application to MRI-targeted Prostate Biopsy. IEEE Trans Med Imaging. 2019;38 (4) :1026-36.Abstract
Image-guidance improves tissue sampling during biopsy by allowing the physician to visualize the tip and trajectory of the biopsy needle relative to the target in MRI, CT, ultrasound, or other relevant imagery. This paper reports a system for fast automatic needle tip and trajectory localization and visualization in MRI that has been developed and tested in the context of an active clinical research program in prostate biopsy. To the best of our knowledge, this is the first reported system for this clinical application, and also the first reported system that leverages deep neural networks for segmentation and localization of needles in MRI across biomedical applications. Needle tip and trajectory were annotated on 583 T2-weighted intra-procedural MRI scans acquired after needle insertion for 71 patients who underwent transperenial MRI-targeted biopsy procedure at our institution. The images were divided into two independent training-validation and test sets at the patient level. A deep 3-dimensional fully convolutional neural network model was developed, trained and deployed on these samples. The accuracy of the proposed method, as tested on previously unseen data, was 2.80 mm average in needle tip detection, and 0.98° in needle trajectory angle. An observer study was designed in which independent annotations by a second observer, blinded to the original observer, were compared to the output of the proposed method. The resultant error was comparable to the measured inter-observer concordance, reinforcing the clinical acceptability of the proposed method. The proposed system has the potential for deployment in clinical routine.