Segmentation is a fundamental task for extracting semantically meaningful regions from an image. The goal of segmentation algorithms is to accurately assign object labels to each image location. However, image noise, shortcomings of algorithms, and image ambiguities cause uncertainty in label assignment. Estimating this uncertainty is important in multiple application domains, such as segmenting tumors from medical images for radiation treatment planning. One way to estimate these uncertainties is through the computation of posteriors of Bayesian models, which is computationally prohibitive for many practical applications. However, most computationally efficient methods fail to estimate label uncertainty. We therefore propose in this paper the active mean fields (AMF) approach, a technique based on Bayesian modeling that uses a mean-field approximation to efficiently compute a segmentation and its corresponding uncertainty. Based on a variational formulation, the resulting convex model combines any label-likelihood measure with a prior on the length of the segmentation boundary. A specific implementation of that model is the Chan-Vese segmentation model, in which the binary segmentation task is defined by a Gaussian likelihood and a prior regularizing the length of the segmentation boundary. Furthermore, the Euler-Lagrange equations derived from the AMF model are equivalent to those of the popular Rudin-Osher-Fatemi (ROF) model for image denoising. Solutions to the AMF model can thus be implemented by directly utilizing highly efficient ROF solvers on log-likelihood ratio fields. We qualitatively assess the approach on synthetic data as well as on real natural and medical images. For a quantitative evaluation, we apply our approach to the tt icgbench dataset.
Neurosurgery makes use of preoperative imaging to visualize pathology, inform surgical planning, and evaluate the safety of selected approaches. The utility of preoperative imaging for neuronavigation, however, is diminished by the well-characterized phenomenon of brain shift, in which the brain deforms intraoperatively as a result of craniotomy, swelling, gravity, tumor resection, cerebrospinal fluid (CSF) drainage, and many other factors. As such, there is a need for updated intraoperative information that accurately reflects intraoperative conditions. Since 1982, intraoperative ultrasound has allowed neurosurgeons to craft and update operative plans without ionizing radiation exposure or major workflow interruption. Continued evolution of ultrasound technology since its introduction has resulted in superior imaging quality, smaller probes, and more seamless integration with neuronavigation systems. Furthermore, the introduction of related imaging modalities, such as 3-dimensional ultrasound, contrast-enhanced ultrasound, high-frequency ultrasound, and ultrasound elastography, has dramatically expanded the options available to the neurosurgeon intraoperatively. In the context of these advances, we review the current state, potential, and challenges of intraoperative ultrasound for brain tumor resection. We begin by evaluating these ultrasound technologies and their relative advantages and disadvantages. We then review three specific applications of these ultrasound technologies to brain tumor resection: (1) intraoperative navigation, (2) assessment of extent of resection, and (3) brain shift monitoring and compensation. We conclude by identifying opportunities for future directions in the development of ultrasound technologies.
We present an efficient probabilistic model of anatomical variability in a linear space of initial velocities of diffeomorphic transformations and demonstrate its benefits in clinical studies of brain anatomy. To overcome the computational challenges of the high dimensional deformation-based descriptors, we develop a latent variable model for principal geodesic analysis (PGA) based on a low dimensional shape descriptor that effectively captures the intrinsic variability in a population. We define a novel shape prior that explicitly represents principal modes as a multivariate complex Gaussian distribution on the initial velocities in a bandlimited space. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than the state-of-the-art method such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA) that operate in the high dimensional image space.
Magnetic Resonance Imaging (MRI) is widely used in routine clinical diagnosis and treatment. However, variations in MRI acquisition protocols result in different appearances of normal and diseased tissue in the images. Convolutional neural networks (CNNs), which have shown to be successful in many medical image analysis tasks, are typically sensitive to the variations in imaging protocols. Therefore, in many cases, networks trained on data acquired with one MRI protocol, do not perform satisfactorily on data acquired with different protocols. This limits the use of models trained with large annotated legacy datasets on a new dataset with a different domain which is often a recurring situation in clinical settings. In this study, we aim to answer the following central questions regarding domain adaptation in medical image analysis: Given a fitted legacy model, (1) How much data from the new domain is required for a decent adaptation of the original network?; and, (2) What portion of the pre-trained model parameters should be retrained given a certain number of the new domain training samples? To address these questions, we conducted extensive experiments in white matter hyperintensity segmentation task. We trained a CNN on legacy MR images of brain and evaluated the performance of the domain-adapted network on the same task with images from a different domain. We then compared the performance of the model to the surrogate scenarios where either the same trained network is used or a new network is trained from scratch on the new dataset. The domain-adapted network tuned only by two training examples achieved a Dice score of 0.63 substantially outperforming a similar network trained on the same set of examples from scratch.
OBJECTIVE: Magnetic resonance imaging (MRI) is a widely used imaging modality for studies of knee osteoarthritis (OA). Compared to radiography, MRI offers exceptional soft tissue imaging and true three-dimensional (3D) visualization. However, MRI is expensive both due to the cost of acquisition and evaluation of the images. The goal of our study is to develop a new method to address the cost of MRI by combining innovative acquisition methods and automated post-processing software. METHODS: Ten healthy volunteers were scanned with three different MRI protocols: A standard 3D dual-echo steady state (DESS) pulse sequence, an accelerated DESS (DESS), acquired at approximately half the time compared to DESS, and a multi-echo time DESS (DESS), which is capable of producing measurements of T2 relaxation time. A software tool was used to measure cartilage volume. Accuracy was quantified by comparing DESS to DESS and DESS and precision was measured using repeat readings and acquisitions. T2 precision was determined using duplicate DESS acquisitions. Intra-class correlation coefficients (ICCs), root-mean square standard deviation (RMSSD), and the coefficient of variation (CoV) were used to quantify accuracy and precision. RESULTS: The accuracies of DESS and DESS were CoV = 3.7% and CoV = 6.6% respectively, while precision was 3.8%, 3.0%, and 3.1% for DESS, DESS and DESS. T2 repositioning precision was 5.8%. CONCLUSION: The results demonstrate that accurate and precise quantification of cartilage volume is possible using a combination of substantially faster MRI acquisition and post-processing software. Precise measurements of cartilage T2 and volume can be made using the same acquisition.
Registration of multiple 3D ultrasound sectors in order to provide an extended field of view is important for the appreciation of larger anatomical structures at high spatial and temporal resolution. In this paper, we present a method for fully automatic spatio-temporal registration between two partially overlapping 3D ultrasound sequences. The temporal alignment is solved by aligning the normalized cross correlation-over-time curves of the sequences. For the spatial alignment, corresponding 3D Scale Invariant Feature Transform (SIFT) features are extracted from all frames of both sequences independently of the temporal alignment. A rigid transform is then calculated by least squares minimization in combination with random sample consensus. The method is applied to 16 echocardiographic sequences of the left and right ventricles and evaluated against manually annotated temporal events and spatial anatomical landmarks. The mean distances between manually identified landmarks in the left and right ventricles after automatic registration were (mean ± SD) 4.3 ± 1.2 mm compared to a reference error of 2.8 ± 0.6 mm with manual registration. For the temporal alignment, the absolute errors in valvular event times were 14.4 ± 11.6 ms for Aortic Valve (AV) opening, 18.6 ± 16.0 ms for AV closing, and 34.6 ± 26.4 ms for mitral valve opening, compared to a mean inter-frame time of 29 ms.
PURPOSE: To describe how B0 inhomogeneities can cause errors in proton resonance frequency (PRF) shift thermometry, and to correct for these errors. METHODS: With PRF thermometry, measured phase shifts are converted into temperature measurements through the use of a scaling factor proportional to the echo time, TE. However, B0 inhomogeneities can deform, spread, and translate MR echoes, potentially making the "true" echo time vary spatially within the imaged object and take on values that differ from the prescribed TE value. Acquisition and reconstruction methods able to avoid or correct for such errors are presented. RESULTS: Tests were performed in a gel phantom during sonication, and temperature measurements were made with proper shimming as well as with intentionally introduced B0 inhomogeneities. Errors caused by B0 inhomogeneities were observed, described, and corrected by the proposed methods. No statistical difference was found between the corrected results and the reference results obtained with proper shimming, while errors by more than 10% in temperature elevation were corrected for. The approach was also applied to an abdominal in vivo dataset. CONCLUSION: Field variations induce errors in measured field values, which can be detected and corrected. The approach was validated for a PRF thermometry application.
Magnetic Resonance (MR) imaging provides excellent image quality at a high cost and low frame rate. Ultrasound (US) provides poor image quality at a low cost and high frame rate. We propose an instance-based learning system to obtain the best of both worlds: high quality MR images at high frame rates from a low cost single-element US sensor. Concurrent US and MRI pairs are acquired during a relatively brief offine learning phase involving the US transducer and MR scanner. High frame rate, high quality MR imaging of respiratory organ motion is then predicted from US measurements, even after stopping MRI acquisition, using a probabilistic kernel regression framework. Experimental results show predicted MR images to be highly representative of actual MR images.
We present an image segmentation method that transfers label maps of entire organs from the training images to the novel image to be segmented. The transfer is based on sparse correspondences between keypoints that represent automatically identified distinctive image locations. Our segmentation algorithm consists of three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ label maps. We introduce generative models for the inference of keypoint labels and for image segmentation, where keypoint matches are treated as a latent random variable and are marginalized out as part of the algorithm. We report segmentation results for abdominal organs in whole-body CT and in contrast-enhanced CT images. The accuracy of our method compares favorably to common multi-atlas segmentation while offering a speed-up of about three orders of magnitude. Furthermore, keypoint transfer requires no training phase or registration to an atlas. The algorithm's robustness enables the segmentation of scans with highly variable field-of-view.
OBJECTIVE: Accurate biopsy sampling of the suspected lesions is critical for the diagnosis and clinical management of prostate cancer. Transperineal in-bore MRI-guided prostate biopsy (tpMRgBx) is a targeted biopsy technique that was shown to be safe, efficient, and accurate. Our goal was to develop an open source software platform to support evaluation, refinement, and translation of this biopsy approach. METHODS: We developed SliceTracker, a 3D Slicer extension to support tpMRgBx. We followed modular design of the implementation to enable customization of the interface and interchange of image segmentation and registration components to assess their effect on the processing time, precision, and accuracy of the biopsy needle placement. The platform and supporting documentation were developed to enable the use of software by an operator with minimal technical training to facilitate translation. Retrospective evaluation studied registration accuracy, effect of the prostate segmentation approach, and re-identification time of biopsy targets. Prospective evaluation focused on the total procedure time and biopsy targeting error (BTE). RESULTS: Evaluation utilized data from 73 retrospective and ten prospective tpMRgBx cases. Mean landmark registration error for retrospective evaluation was 1.88 ± 2.63 mm, and was not sensitive to the approach used for prostate gland segmentation. Prospectively, we observed target re-identification time of 4.60 ± 2.40 min and BTE of 2.40 ± 0.98 mm. CONCLUSION: SliceTracker is modular and extensible open source platform for supporting image processing aspects of the tpMRgBx procedure. It has been successfully utilized to support clinical research procedures at our site.
Patient-mounted needle guide devices for percutaneous ablation are vulnerable to patient motion. The objective of this study is to develop and evaluate a software system for an MRI-compatible patient-mounted needle guide device that can adaptively compensate for displacement of the device due to patient motion using a novel image-based automatic device-to-image registration technique. We have developed a software system for an MRI-compatible patient-mounted needle guide device for percutaneous ablation. It features fully-automated image-based device-to-image registration to track the device position, and a device controller to adjust the needle trajectory to compensate for the displacement of the device. We performed: (a) a phantom study using a clinical MR scanner to evaluate registration performance; (b) simulations using intraoperative time-series MR data acquired in 20 clinical cases of MRI-guided renal cryoablations to assess its impact on motion compensation; and (c) a pilot clinical study in three patients to test its feasibility during the clinical procedure. FRE, TRE, and success rate of device-to-image registration were [Formula: see text] mm, [Formula: see text] mm, and 98.3% for the phantom images. The simulation study showed that the motion compensation reduced the targeting error for needle placement from 8.2 mm to 5.4 mm (p < 0.0005) in patients under general anesthesia (GA), and from 14.4 mm to 10.0 mm ([Formula: see text]) in patients under monitored anesthesia care (MAC). The pilot study showed that the software registered the device successfully in a clinical setting. Our simulation study demonstrated that the software system could significantly improve targeting accuracy in patients treated under both MAC and GA. Intraprocedural image-based device-to-image registration was feasible.
PURPOSE: To develop and evaluate an approach to estimate the respiratory-induced motion of lesions in the chest and abdomen. MATERIALS AND METHODS: The proposed approach uses the motion of an initial reference needle inserted into a moving organ to estimate the lesion (target) displacement that is caused by respiration. The needles position is measured using an inertial measurement unit (IMU) sensor externally attached to the hub of an initially placed reference needle. Data obtained from the IMU sensor and the target motion are used to train a learning-based approach to estimate the position of the moving target. An experimental platform was designed to mimic respiratory motion of the liver. Liver motion profiles of human subjects provided inputs to the experimental platform. Variables including the insertion angle, target depth, target motion velocity and target proximity to the reference needle were evaluated by measuring the error of the estimated target position and processing time. RESULTS: The mean error of estimation of the target position ranged between 0.86 and 1.29 mm. The processing maximum training and testing time was 5 ms which is suitable for real-time target motion estimation using the needle position sensor. CONCLUSION: The external motion of an initially placed reference needle inserted into a moving organ can be used as a surrogate, measurable and accessible signal to estimate in real-time the position of a moving target caused by respiration; this technique could then be used to guide the placement of subsequently inserted needles directly into the target.
Brain shift during tumor resection compromises the spatial validity of registered preoperative imaging data that is critical to image-guided procedures. One current clinical solution to mitigate the effects is to reimage using intraoperative magnetic resonance (iMR) imaging. Although iMR has demonstrated benefits in accounting for preoperative-to-intraoperative tissue changes, its cost and encumbrance have limited its widespread adoption. While iMR will likely continue to be employed for challenging cases, a cost-effective model-based brain shift compensation strategy is desirable as a complementary technology for standard resections. We performed a retrospective study of [Formula: see text] tumor resection cases, comparing iMR measurements with intraoperative brain shift compensation predicted by our model-based strategy, driven by sparse intraoperative cortical surface data. For quantitative assessment, homologous subsurface targets near the tumors were selected on preoperative MR and iMR images. Once rigidly registered, intraoperative shift measurements were determined and subsequently compared to model-predicted counterparts as estimated by the brain shift correction framework. When considering moderate and high shift ([Formula: see text], [Formula: see text] measurements per case), the alignment error due to brain shift reduced from [Formula: see text] to [Formula: see text], representing [Formula: see text] correction. These first steps toward validation are promising for model-based strategies.
OBJECTIVE: The purpose of this article is to report our intermediate to long-term outcomes with image-guided percutaneous hepatic tumor cryoablation and to evaluate its technical success, technique efficacy, local tumor progression, and adverse event rate. MATERIALS AND METHODS: Between 1998 and 2014, 299 hepatic tumors (243 metastases and 56 primary tumors; mean diameter, 2.5 cm; median diameter, 2.2 cm; range, 0.3-7.8 cm) in 186 patients (95 women; mean age, 60.9 years; range, 29-88 years) underwent cryoablation during 236 procedures using CT (n = 126), MRI (n = 100), or PET/CT (n = 10) guidance. Technical success, technique efficacy at 3 months, local tumor progression (mean follow-up, 2.5 years; range, 2 months to 14.6 years), and adverse event rates were calculated. RESULTS: The technical success rate was 94.6% (279/295). The technique efficacy rate was 89.5% (231/258) and was greater for tumors smaller than 4 cm (93.4%; 213/228) than for larger tumors (60.0%; 18/30) (p < 0.0001). Local tumor progression occurred in 23.3% (60/258) of tumors and was significantly more common after the treatment of tumors 4 cm or larger (63.3%; 19/30) compared with smaller tumors (18.0%; 41/228) (p < 0.0001). Adverse events followed 33.8% (80/236) of procedures and were grade 3-5 in 10.6% (25/236) of cases. Grade 3 or greater adverse events more commonly followed the treatment of larger tumors (19.5%; 8/41) compared with smaller tumors (8.7%; 17/195) (p = 0.04). CONCLUSION: Image-guided percutaneous cryoablation of hepatic tumors is efficacious; however, tumors smaller than 4 cm are more likely to be treated successfully and without an adverse event.