HOME / RESEARCH /

Computation Core

William Wells Bruno Madore
William Wells, PhD
Core Lead 		Bruno Madore, PhD
Project Lead

The computation project is leveraging recent progress in ultrasound-ultrasound (US) registration and in hybrid US-MRI technology to develop synergistic software and hardware technology that is aimed at improving surgical and interventional guidance in the presence of tissue deformation or motion, issues that complicate treatment monitoring or comparisons to pre-operative images and treatment plans.  Our approach to addressing deformation problems in image guided therapy (IGT) leverages our recent work in feature-based US-US registration, where image content is modeled in terms of local scale-invariant image features, i.e., distinctive patterns of echogenic anatomical tissue that can be automatically extracted from images and used as the basis for registration. Our solution for motion in IGT is built upon our recent developments in hybrid US-MRI technology that acquires MRI and ultrasound simultaneously to exploit the relative strengths of MRI (high spatial resolution and excellent soft tissue contrast), and US (high frame rate). Much of the proposed research deals with providing solutions to registration problems for IGT applications, such as tissue deformation fields, and we believe that in this context it is important to characterize the potential uncertainties in these solutions, similarly to providing error bars in other estimation problems.To this end we are developing registration-with-uncertainty algorithms that incorporate random process models of spatial uncertainty. The technology is evaluated in the context of our testbed clinical projects, image-guided neurosurgery and abdominal cryotherapy, in the AMIGO suite, our advanced interventional suite that includes intra-operative 3T MRI, ultrasound and PET/CT. The hybrid US-MRI approach enables rapid updates to MRI images to accommodate, e.g., breathing motions during cryoablation procedures.In addition, US-US registration algorithms facilitate improvements in US-updated neurosurgical guidance, and have potential IGT applications in our program or elsewhere, for example in prostate biopsies. In order to facilitate dissemination of these algorithms to the broader IGT community, we distribute software components in the open-source SlicerIGT platform. Our projects are:

Registration algorithms for MRI and US with emphasis on uncertainty and algorithm performance. We continue algorithm developments aimed at characterizing uncertainty and accuracy in image registration,and tissue deformation estimation from implanted trackers,that are based on Gaussian Random Fields (GRF). We are also developing algorithms that estimate surgical tissue deformations from our feature-based ultrasound / ultrasound registration technology. Finally, we translate the developed algorithms into AMIGO using the SlicerIGT platform by providing extensions that visualize deformed MRI based on intraoperative US, associated registration uncertainty, and integrated laser surface scanning for neurosurgery. (Contact: William Wells)

Technology for simultaneous US-MRI acquisition for monitoring procedures. We are developing machine learning techniques that use high bandwidth US data to estimate motion and deformation in MRI images. We are also further generalizing the hybrid US-MRI approach by exploiting information from 256 independent channels, from a custom-built MR-compatible 256-element 2D US transducer array provided by an industrial partner. We are developing a pre-scan calibration (“learning”) phase that employs simultaneously-acquired MRI and USdata. We will deploy on-line deformation-corrected updates of MR as they become available from the scanner, for monitoring cryoablations. (Contact: Bruno Madore)

Software and Documentation

3D Slicer, a comprehensive open source platform for medical image analysis, contains several modules and functions that have been contributed by us for Computation. These include: Source Code for the Paper Titled: Efficient and Robust Model-to-Image Alignment using 3D Scale-Invariant Features (Med Image Anal. 2013 Apr;17(3):271-82.)

Data

MRI acquired to guide Gynecologic Brachytherapy Catheter Placement

Links

3D Slicer

Full Publication List

In NIH/NLMdatabase and in our Abstracts Database.

Select Recent Publications

Luo J, Frisken S, Machado I, Zhang M, Pieper S, Golland P, Toews M, Unadkat P, Sedghi A, Zhou H, et al. Using the Variogram for Vector Outlier Screening: Application to Feature-based Image Registration. Int J Comput Assist Radiol Surg. 2018.Abstract
PURPOSE: Matching points that are derived from features or landmarks in image data is a key step in some medical imaging applications. Since most robust point matching algorithms claim to be able to deal with outliers, users may place high confidence in the matching result and use it without further examination. However, for tasks such as feature-based registration in image-guided neurosurgery, even a few mismatches, in the form of invalid displacement vectors, could cause serious consequences. As a result, having an effective tool by which operators can manually screen all matches for outliers could substantially benefit the outcome of those applications. METHODS: We introduce a novel variogram-based outlier screening method for vectors. The variogram is a powerful geostatistical tool for characterizing the spatial dependence of stochastic processes. Since the spatial correlation of invalid displacement vectors, which are considered as vector outliers, tends to behave differently than normal displacement vectors, they can be efficiently identified on the variogram. RESULTS: We validate the proposed method on 9 sets of clinically acquired ultrasound data. In the experiment, potential outliers are flagged on the variogram by one operator and further evaluated by 8 experienced medical imaging researchers. The matching quality of those potential outliers is approximately 1.5 lower, on a scale from 1 (bad) to 5 (good), than valid displacement vectors. CONCLUSION: The variogram is a simple yet informative tool. While being used extensively in geostatistical analysis, it has not received enough attention in the medical imaging field. We believe there is a good deal of potential for clinically applying the proposed outlier screening method. By way of this paper, we also expect researchers to find variogram useful in other medical applications that involve motion vectors analyses.
Wachinger C, Toews M, Langs G, Wells W, Golland P. Keypoint Transfer for Fast Whole-Body Segmentation. IEEE Trans Med Imaging. 2018.Abstract
We introduce an approach for image segmentation based on sparse correspondences between keypoints in testing and training images. Keypoints represent automatically identified distinctive image locations, where each keypoint correspondence suggests a transformation between images. We use these correspondences to transfer label maps of entire organs from the training images to the test image. The keypoint transfer algorithm includes three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ segmentations. We report segmentation results for abdominal organs in whole-body CT and MRI, as well as in contrast-enhanced CT and MRI. Our method offers a speed-up of about three orders of magnitude in comparison to common multi-atlas segmentation, while achieving an accuracy that compares favorably. Moreover, keypoint transfer does not require the registration to an atlas or a training phase. Finally, the method allows for the segmentation of scans with highly variable field-of-view.
Duryea J, Cheng C, Schaefer LF, Smith S, Madore B. Integration of Accelerated MRI and Post-Processing Software: A Promising Method for Studies of Knee Osteoarthritis. Osteoarthritis Cartilage. 2016;24 (11) :1905-1909.Abstract
OBJECTIVE: Magnetic resonance imaging (MRI) is a widely used imaging modality for studies of knee osteoarthritis (OA). Compared to radiography, MRI offers exceptional soft tissue imaging and true three-dimensional (3D) visualization. However, MRI is expensive both due to the cost of acquisition and evaluation of the images. The goal of our study is to develop a new method to address the cost of MRI by combining innovative acquisition methods and automated post-processing software. METHODS: Ten healthy volunteers were scanned with three different MRI protocols: A standard 3D dual-echo steady state (DESS) pulse sequence, an accelerated DESS (DESS), acquired at approximately half the time compared to DESS, and a multi-echo time DESS (DESS), which is capable of producing measurements of T2 relaxation time. A software tool was used to measure cartilage volume. Accuracy was quantified by comparing DESS to DESS and DESS and precision was measured using repeat readings and acquisitions. T2 precision was determined using duplicate DESS acquisitions. Intra-class correlation coefficients (ICCs), root-mean square standard deviation (RMSSD), and the coefficient of variation (CoV) were used to quantify accuracy and precision. RESULTS: The accuracies of DESS and DESS were CoV = 3.7% and CoV = 6.6% respectively, while precision was 3.8%, 3.0%, and 3.1% for DESS, DESS and DESS. T2 repositioning precision was 5.8%. CONCLUSION: The results demonstrate that accurate and precise quantification of cartilage volume is possible using a combination of substantially faster MRI acquisition and post-processing software. Precise measurements of cartilage T2 and volume can be made using the same acquisition.
Yengul SS, Barbone PE, Madore B. Application of a Forward Model of Axisymmetric Shear Wave Propagation in Viscoelastic Media to Shear Wave Elastography. J Acoust Soc Am. 2018;143 (6) :3266.Abstract
A simple but general solution of Navier's equation for axisymmetric shear wave propagation in a homogeneous isotropic viscoelastic medium is presented. It is well-suited for use as a forward model for some acoustic radiation force impulse based shear wave elastography applications because it does not require precise knowledge of the strength of the source, nor its spatial or temporal distribution. Instead, it depends on two assumptions: (1) the source distribution is axisymmetric and confined to a small region near the axis of symmetry, and (2) the propagation medium is isotropic and homogeneous. The model accounts for the vector polarization of shear waves and exactly represents geometric spreading of the shear wavefield, whether spherical, cylindrical, or neither. It makes no assumption about the frequency dependence of material parameters, i.e., it is material-model independent. Validation using measured shear wavefields excited by acoustic radiation force in a homogeneous gelatin sample show that the model accounts for well over 90% of the measured wavefield "energy." An optimal fit of the model to simulated shear wavefields with noise in a homogeneous viscoelastic medium enables estimation of both the shear storage modulus and shear wave attenuation to within 1%.
Toews M, Wells WM. Phantomless Auto-Calibration and Online Calibration Assessment for a Tracked Freehand 2-D Ultrasound Probe. IEEE Trans Med Imaging. 2018;37 (1) :262-72.Abstract
This paper presents a method for automatically calibrating and assessing the calibration quality of an externally tracked 2-D ultrasound (US) probe by scanning arbitrary, natural tissues, as opposed a specialized calibration phantom as is the typical practice. A generative topic model quantifies the posterior probability of calibration parameters conditioned on local 2-D image features arising from a generic underlying substrate. Auto-calibration is achieved by identifying the maximum a-posteriori image-to-probe transform, and calibration quality is assessed online in terms of the posterior probability of the current image-to-probe transform. Both are closely linked to the 3-D point reconstruction error (PRE) in aligning feature observations arising from the same underlying physical structure in different US images. The method is of practical importance in that it operates simply by scanning arbitrary textured echogenic structures, e.g., in-vivo tissues in the context of the US-guided procedures, without requiring specialized calibration procedures or equipment. Observed data take the form of local scale-invariant features that can be extracted and fit to the model in near real-time. Experiments demonstrate the method on a public data set of in vivo human brain scans of 14 unique subjects acquired in the context of neurosurgery. Online calibration assessment can be performed at approximately 3 Hz for the US images of pixels. Auto-calibration achieves an internal mean PRE of 1.2 mm and a discrepancy of [2 mm, 6 mm] in comparison to the calibration via a standard phantom-based method.
Niethammer M, Pohl KM, Janoos F, Wells WM. Active Mean Fields for Probabilistic Image Segmentation: Connections with Chan-Vese and Rudin-Osher-Fatemi Models. SIAM J. Imaging Sci. 2017;10 (3) :1069–1103.Abstract
Segmentation is a fundamental task for extracting semantically meaningful regions from an image. The goal of segmentation algorithms is to accurately assign object labels to each image location. However, image noise, shortcomings of algorithms, and image ambiguities cause uncertainty in label assignment. Estimating this uncertainty is important in multiple application domains, such as segmenting tumors from medical images for radiation treatment planning. One way to estimate these uncertainties is through the computation of posteriors of Bayesian models, which is computationally prohibitive for many practical applications. However, most computationally efficient methods fail to estimate label uncertainty. We therefore propose in this paper the active mean fields (AMF) approach, a technique based on Bayesian modeling that uses a mean-field approximation to efficiently compute a segmentation and its corresponding uncertainty. Based on a variational formulation, the resulting convex model combines any label-likelihood measure with a prior on the length of the segmentation boundary. A specific implementation of that model is the Chan-Vese segmentation model, in which the binary segmentation task is defined by a Gaussian likelihood and a prior regularizing the length of the segmentation boundary. Furthermore, the Euler-Lagrange equations derived from the AMF model are equivalent to those of the popular Rudin-Osher-Fatemi (ROF) model for image denoising. Solutions to the AMF model can thus be implemented by directly utilizing highly efficient ROF solvers on log-likelihood ratio fields. We qualitatively assess the approach on synthetic data as well as on real natural and medical images. For a quantitative evaluation, we apply our approach to the tt icgbench dataset.
Ghafoorian M, Mehrtash A, Kapur T, Karssemeijer N, Marchiori E, Pesteie M, Guttmann CRG, de Leeuw F-E, Tempany CMC, van Ginneken B, et al. Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation. Int Conf Med Image Comput Comput Assist Interv. 2017;20 (Pt3) :516-24.Abstract
Ghafoorian MICCAI 2017
Magnetic Resonance Imaging (MRI) is widely used in routine clinical diagnosis and treatment. However, variations in MRI acquisition protocols result in different appearances of normal and diseased tissue in the images. Convolutional neural networks (CNNs), which have shown to be successful in many medical image analysis tasks, are typically sensitive to the variations in imaging protocols. Therefore, in many cases, networks trained on data acquired with one MRI protocol, do not perform satisfactorily on data acquired with different protocols. This limits the use of models trained with large annotated legacy datasets on a new dataset with a different domain which is often a recurring situation in clinical settings. In this study, we aim to answer the following central questions regarding domain adaptation in medical image analysis: Given a fitted legacy model, (1) How much data from the new domain is required for a decent adaptation of the original network?; and, (2) What portion of the pre-trained model parameters should be retrained given a certain number of the new domain training samples? To address these questions, we conducted extensive experiments in white matter hyperintensity segmentation task. We trained a CNN on legacy MR images of brain and evaluated the performance of the domain-adapted network on the same task with images from a different domain. We then compared the performance of the model to the surrogate scenarios where either the same trained network is used or a new network is trained from scratch on the new dataset. The domain-adapted network tuned only by two training examples achieved a Dice score of 0.63 substantially outperforming a similar network trained on the same set of examples from scratch.
Mastmeyer A, Pernelle G, Ma R, Barber L, Kapur T. Accurate Model-based Segmentation of Gynecologic Brachytherapy Catheter Collections in MRI-images. Med Image Anal. 2017;42 :173-88.Abstract
The gynecological cancer mortality rate, including cervical, ovarian, vaginal and vulvar cancers, is more than 20,000 annually in the US alone. In many countries, including the US, external-beam radiotherapy followed by high dose rate brachytherapy is the standard-of-care. The superior ability of MR to visualize soft tissue has led to an increase in its usage in planning and delivering brachytherapy treatment. A technical challenge associated with the use of MRI imaging for brachytherapy, in contrast to that of CT imaging, is the visualization of catheters that are used to place radiation sources into cancerous tissue. We describe here a precise, accurate method for achieving catheter segmentation and visualization. The algorithm, with the assistance of manually provided tip locations, performs segmentation using image-features, and is guided by a catheter-specific, estimated mechanical model. A final quality control step removes outliers or conflicting catheter trajectories. The mean Hausdorff error on a 54 patient, 760 catheter reference database was 1.49  mm; 51 of the outliers deviated more than two catheter widths (3.4  mm) from the gold standard, corresponding to catheter identification accuracy of 93% in a Syed-Neblett template. In a multi-user simulation experiment for evaluating RMS precision by simulating varying manually-provided superior tip positions, 3σ maximum errors were 2.44  mm. The average segmentation time for a single catheter was 3 s on a standard PC. The segmentation time, accuracy and precision, are promising indicators of the value of this method for clinical translation of MR-guidance in gynecologic brachytherapy and other catheter-based interventional procedures.
Zhang M, Wells WM, Golland P. Probabilistic Modeling of Anatomical Variability using a Low Dimensional Parameterization of Diffeomorphisms. Med Image Anal. 2017;41 :55-62.Abstract
We present an efficient probabilistic model of anatomical variability in a linear space of initial velocities of diffeomorphic transformations and demonstrate its benefits in clinical studies of brain anatomy. To overcome the computational challenges of the high dimensional deformation-based descriptors, we develop a latent variable model for principal geodesic analysis (PGA) based on a low dimensional shape descriptor that effectively captures the intrinsic variability in a population. We define a novel shape prior that explicitly represents principal modes as a multivariate complex Gaussian distribution on the initial velocities in a bandlimited space. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than the state-of-the-art method such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA) that operate in the high dimensional image space.
Preiswerk F, Toews M, Cheng C-C, Chiou J-yuan G, Mei C-S, Schaefer LF, Hoge WS, Schwartz BM, Panych LP, Madore B. Hybrid MRI Ultrasound Acquisitions, and Scannerless Real-time Imaging. Magn Reson Med. 2016;78 (3) :897-908.Abstract

PURPOSE: To combine MRI, ultrasound, and computer science methodologies toward generating MRI contrast at the high frame rates of ultrasound, inside and even outside the MRI bore. METHODS: A small transducer, held onto the abdomen with an adhesive bandage, collected ultrasound signals during MRI. Based on these ultrasound signals and their correlations with MRI, a machine-learning algorithm created synthetic MR images at frame rates up to 100 per second. In one particular implementation, volunteers were taken out of the MRI bore with the ultrasound sensor still in place, and MR images were generated on the basis of ultrasound signal and learned correlations alone in a "scannerless" manner. RESULTS: Hybrid ultrasound-MRI data were acquired in eight separate imaging sessions. Locations of liver features, in synthetic images, were compared with those from acquired images: The mean error was 1.0 pixel (2.1 mm), with best case 0.4 and worst case 4.1 pixels (in the presence of heavy coughing). For results from outside the bore, qualitative validation involved optically tracked ultrasound imaging with/without coughing. CONCLUSION: The proposed setup can generate an accurate stream of high-speed MR images, up to 100 frames per second, inside or even outside the MR bore. Magn Reson Med, 2016. © 2016 International Society for Magnetic Resonance in Medicine.