Publications by Year: 2021

Meyer A, Mehrtash A, Rak M, Bashkanov O, Langbein B, Ziaei A, Kibel AS, Tempany CM, Hansen C, Tokuda J. Domain Adaptation for Segmentation of Critical Structures for Prostate Cancer Therapy. Sci Rep. 2021;11 (1) :11480.Abstract
Preoperative assessment of the proximity of critical structures to the tumors is crucial in avoiding unnecessary damage during prostate cancer treatment. A patient-specific 3D anatomical model of those structures, namely the neurovascular bundles (NVB) and the external urethral sphincters (EUS), can enable physicians to perform such assessments intuitively. As a crucial step to generate a patient-specific anatomical model from preoperative MRI in a clinical routine, we propose a multi-class automatic segmentation based on an anisotropic convolutional network. Our specific challenge is to train the network model on a unique source dataset only available at a single clinical site and deploy it to another target site without sharing the original images or labels. As network models trained on data from a single source suffer from quality loss due to the domain shift, we propose a semi-supervised domain adaptation (DA) method to refine the model's performance in the target domain. Our DA method combines transfer learning and uncertainty guided self-learning based on deep ensembles. Experiments on the segmentation of the prostate, NVB, and EUS, show significant performance gain with the combination of those techniques compared to pure TL and the combination of TL with simple self-learning ([Formula: see text] for all structures using a Wilcoxon's signed-rank test). Results on a different task and data (Pancreas CT segmentation) demonstrate our method's generic application capabilities. Our method has the advantage that it does not require any further data from the source domain, unlike the majority of recent domain adaptation strategies. This makes our method suitable for clinical applications, where the sharing of patient data is restricted.
Zhou H, Jayender J. Real-Time Nonrigid Mosaicking of Laparoscopy Images. IEEE Trans Med Imaging. 2021;40 (6) :1726-36.Abstract
The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D non-rigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on in vivo and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.
Madore B, Preiswerk F, Bredfeldt JS, Zong S, Cheng C-C. Ultrasound-based Sensors to Monitor Physiological Motion. Med Phys. 2021.Abstract
PURPOSE: Medical procedures can be difficult to perform on anatomy that is constantly moving. Respiration displaces internal organs by up to several centimeters with respect to the surface of the body, and patients often have limited ability to hold their breath. Strategies to compensate for motion during diagnostic and therapeutic procedures require reliable information to be available. However, current devices often monitor respiration indirectly, through changes on the outline of the body, and they may be fixed to floors or ceilings, and thus unable to follow a given patient through different locations. Here we show that small ultrasound-based sensors referred to as "organ configuration motion" (OCM) sensors can be fixed to the abdomen and/or chest and provide information-rich, breathing-related signals. METHODS: By design, the proposed sensors are relatively inexpensive. Breathing waveforms were obtained from tissues at varying depths and/or using different sensor placements. Validation was performed against breathing waveforms derived from magnetic resonance imaging (MRI) and optical tracking signals in five and eight volunteers, respectively. RESULTS: Breathing waveforms from different modalities were scaled so they could be directly compared. Differences between waveforms were expressed in the form of a percentage, as compared to the amplitude of a typical breath. Expressed in this manner, for shallow tissues, OCM-derived waveforms on average differed from MRI and optical tracking results by 13.1% and 15.5%, respectively. CONCLUSION: The present results suggest that the proposed sensors provide measurements that properly characterize breathing states. While OCM-based waveforms from shallow tissues proved similar in terms of information content to those derived from MRI or optical tracking, OCM further captured depth-dependent and position-dependent (i.e., chest and abdomen) information. In time, the richer information content of OCM-based waveforms may enable better respiratory gating to be performed, to allow diagnostic and therapeutic equipment to perform at their best.
Chaudhuri S, Fowler MJ, Baker C, Stopka SA, Regan MS, Sablatura L, Broughton CW, Knight BE, Stabenfeldt SE, Agar NYR, et al. β-Cyclodextrin-poly (β-Amino Ester) Nanoparticles Are a Generalizable Strategy for High Loading and Sustained Release of HDAC Inhibitors. ACS Appl Mater Interfaces. 2021;13 (18) :20960-73.Abstract
Therapeutic development of histone deacetylase inhibitors (HDACi) has been hampered by a number of barriers to drug delivery, including poor solubility and inadequate tissue penetration. Nanoparticle encapsulation could be one approach to improve the delivery of HDACi to target tissues; however, effective and generalizable loading of HDACi within nanoparticle systems remains a long-term challenge. We hypothesized that the common terminally ionizable moiety on many HDACi molecules could be capitalized upon for loading in polymeric nanoparticles. Here, we describe the simple, efficient formulation of a novel library of β-cyclodextrin-poly (β-amino ester) networks (CDN) to achieve this goal. We observed that network architecture was a critical determinant of CDN encapsulation of candidate molecules, with a more hydrophobic core enabling effective self-assembly and a PEGylated surface enabling high loading (up to ∼30% w/w), effective self-assembly of the nanoparticle, and slow release of drug into aqueous media (up to 24 days) for the model HDACi panobinostat. We next constructed a library of CDNs to encapsulate various small, hydrophobic, terminally ionizable molecules (panobinostat, quisinostat, dacinostat, givinostat, bortezomib, camptothecin, nile red, and cytarabine), which yielded important insights into the structural requirements for effective drug loading and CDN self-assembly. Optimized CDN nanoparticles were taken up by GL261 cells in culture and a released panobinostat was confirmed to be bioactive. Panobinostat-loaded CDNs were next administered by convection-enhanced delivery (CED) to mice bearing intracranial GL261 tumors. These studies confirm that CDN encapsulation enables a higher deliverable dose of drug to effectively slow tumor growth. Matrix-assisted laser desorption/ionization (MALDI) analysis on tissue sections confirms higher exposure of tumor to drug, which likely accounts for the therapeutic effects. Taken in sum, these studies present a novel nanocarrier platform for encapsulation of HDACi via both ionic and hydrophobic interactions, which is an important step toward better treatment of disease via HDACi therapy.
Wang D, Zhang T, Li M, Bueno R, Jayender J. 3D Deep Learning Based Classification of Pulmonary Ground Glass Opacity Nodules With Automatic Segmentation. Comput Med Imaging Graph. 2021;88 :101814.Abstract
Classifying ground-glass lung nodules (GGNs) into atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA), and invasive adenocarcinoma (IAC) on diagnostic CT images is important to evaluate the therapy options for lung cancer patients. In this paper, we propose a joint deep learning model where the segmentation can better facilitate the classification of pulmonary GGNs. Based on our observation that masking the nodule to train the model results in better lesion classification, we propose to build a cascade architecture with both segmentation and classification networks. The segmentation model works as a trainable preprocessing module to provide the classification-guided 'attention' weight map to the raw CT data to achieve better diagnosis performance. We evaluate our proposed model and compare with other baseline models for 4 clinically significant nodule classification tasks, defined by a combination of pathology types, using 4 classification metrics: Accuracy, Average F1 Score, Matthews Correlation Coefficient (MCC), and Area Under the Receiver Operating Characteristic Curve (AUC). Experimental results show that the proposed method outperforms other baseline models on all the diagnostic classification tasks.
Wang D, Li M, Ben-Shlomo N, Corrales EC, Cheng Y, Zhang T, Jayender J. A Novel Dual-Network Architecture for Mixed-Supervised Medical Image Segmentation. Comput Med Imaging Graph. 2021;89 :101841.Abstract
In medical image segmentation tasks, deep learning-based models usually require densely and precisely annotated datasets to train, which are time-consuming and expensive to prepare. One possible solution is to train with the mixed-supervised dataset, where only a part of data is densely annotated with segmentation map and the rest is annotated with some weak form, such as bounding box. In this paper, we propose a novel network architecture called Mixed-Supervised Dual-Network (MSDN), which consists of two separate networks for the segmentation and detection tasks respectively, and a series of connection modules between the layers of the two networks. These connection modules are used to extract and transfer useful information from the detection task to help the segmentation task. We exploit a variant of a recently designed technique called 'Squeeze and Excitation' in the connection module to boost the information transfer between the two tasks. Compared with existing model with shared backbone and multiple branches, our model has flexible and trainable feature sharing fashion and thus is more effective and stable. We conduct experiments on 4 medical image segmentation datasets, and experiment results show that the proposed MSDN model outperforms multiple baselines.
He J, Zhang F, Xie G, Yao S, Feng Y, Bastos DCA, Rathi Y, Makris N, Kikinis R, Golby AJ, et al. Comparison of Multiple Tractography Methods for Reconstruction of the Retinogeniculate Visual Pathway Using Diffusion MRI. Hum Brain Mapp. 2021.Abstract
The retinogeniculate visual pathway (RGVP) conveys visual information from the retina to the lateral geniculate nucleus. The RGVP has four subdivisions, including two decussating and two nondecussating pathways that cannot be identified on conventional structural magnetic resonance imaging (MRI). Diffusion MRI tractography has the potential to trace these subdivisions and is increasingly used to study the RGVP. However, it is not yet known which fiber tracking strategy is most suitable for RGVP reconstruction. In this study, four tractography methods are compared, including constrained spherical deconvolution (CSD) based probabilistic (iFOD1) and deterministic (SD-Stream) methods, and multi-fiber (UKF-2T) and single-fiber (UKF-1T) unscented Kalman filter (UKF) methods. Experiments use diffusion MRI data from 57 subjects in the Human Connectome Project. The RGVP is identified using regions of interest created by two clinical experts. Quantitative anatomical measurements and expert anatomical judgment are used to assess the advantages and limitations of the four tractography methods. Overall, we conclude that UKF-2T and iFOD1 produce the best RGVP reconstruction results. The iFOD1 method can better quantitatively estimate the percentage of decussating fibers, while the UKF-2T method produces reconstructed RGVPs that are judged to better correspond to the known anatomy and have the highest spatial overlap across subjects. Overall, we find that it is challenging for current tractography methods to both accurately track RGVP fibers that correspond to known anatomy and produce an approximately correct percentage of decussating fibers. We suggest that future algorithm development for RGVP tractography should take consideration of both of these two points.
Zhang F, Breger A, Cho KIK, Ning L, Westin C-F, O'Donnell LJ, Pasternak O. Deep Learning Based Segmentation of Brain Tissue from Diffusion MRI. Neuroimage. 2021;233 :117934.Abstract
Segmentation of brain tissue types from diffusion MRI (dMRI) is an important task, required for quantification of brain microstructure and for improving tractography. Current dMRI segmentation is mostly based on anatomical MRI (e.g., T1- and T2-weighted) segmentation that is registered to the dMRI space. However, such inter-modality registration is challenging due to more image distortions and lower image resolution in dMRI as compared with anatomical MRI. In this study, we present a deep learning method for diffusion MRI segmentation, which we refer to as DDSeg. Our proposed method learns tissue segmentation from high-quality imaging data from the Human Connectome Project (HCP), where registration of anatomical MRI to dMRI is more precise. The method is then able to predict a tissue segmentation directly from new dMRI data, including data collected with different acquisition protocols, without requiring anatomical data and inter-modality registration. We train a convolutional neural network (CNN) to learn a tissue segmentation model using a novel augmented target loss function designed to improve accuracy in regions of tissue boundary. To further improve accuracy, our method adds diffusion kurtosis imaging (DKI) parameters that characterize non-Gaussian water molecule diffusion to the conventional diffusion tensor imaging parameters. The DKI parameters are calculated from the recently proposed mean-kurtosis-curve method that corrects implausible DKI parameter values and provides additional features that discriminate between tissue types. We demonstrate high tissue segmentation accuracy on HCP data, and also when applying the HCP-trained model on dMRI data from other acquisitions with lower resolution and fewer gradient directions.
Xu Z, Luo J, Yan J, Li X, Jayender J. F3RNet: Full-resolution Residual Registration Network for Deformable Image Registration. Int J Comput Assist Radiol Surg. 2021;16 (6) :923-32.Abstract
PURPOSE: Deformable image registration (DIR) is essential for many image-guided therapies. Recently, deep learning approaches have gained substantial popularity and success in DIR. Most deep learning approaches use the so-called mono-stream high-to-low, low-to-high network structure and can achieve satisfactory overall registration results. However, accurate alignments for some severely deformed local regions, which are crucial for pinpointing surgical targets, are often overlooked. Consequently, these approaches are not sensitive to some hard-to-align regions, e.g., intra-patient registration of deformed liver lobes. METHODS: We propose a novel unsupervised registration network, namely full-resolution residual registration network (F3RNet), for deformable registration of severely deformed organs. The proposed method combines two parallel processing streams in a residual learning fashion. One stream takes advantage of the full-resolution information that facilitates accurate voxel-level registration. The other stream learns the deep multi-scale residual representations to obtain robust recognition. We also factorize the 3D convolution to reduce the training parameters and enhance network efficiency. RESULTS: We validate the proposed method on a clinically acquired intra-patient abdominal CT-MRI dataset and a public inspiratory and expiratory thorax CT dataset. Experiments on both multimodal and unimodal registration demonstrate promising results compared to state-of-the-art approaches. CONCLUSION: By combining the high-resolution information and multi-scale representations in a highly interactive residual learning fashion, the proposed F3RNet can achieve accurate overall and local registration. The run time for registering a pair of images is less than 3 s using a GPU. In future works, we will investigate how to cost-effectively process high-resolution information and fuse multi-scale representations.
Moreira P, Grimble J, Iftimia N, Bay CP, Tuncali K, Park J, Tokuda J. In Vivo Evaluation of Angulated Needle-Guide Template for MRI-Guided Transperineal Prostate Biopsy. Med Phys. 2021;48 (5) :2553-65.Abstract
PURPOSE: Magnetic resonance imaging (MRI)-guided transperineal prostate biopsy has been practiced since the early 2000s. The technique often suffers from targeting error due to deviation of the needle as a result of physical interaction between the needle and inhomogeneous tissues. Existing needle guide devices, such as a grid template, do not allow choosing an alternative insertion path to mitigate the deviation because of their limited degree-of-freedom (DoF). This study evaluates how an angulated needle insertion path can reduce needle deviation and improve needle placement accuracy. METHODS: We extended a robotic needle-guidance device (Smart Template) for in-bore MRI-guided transperineal prostate biopsy. The new Smart Template has a 4-DoF needle-guiding mechanism allowing a translational range of motion of 65 and 58 mm along the vertical and horizontal axis, and a needle rotational motion around the vertical and horizontal axis and a vertical rotational range of , respectively. We defined a path planning strategy, which chooses between straight and angulated insertion paths depending on the anatomical structures on the potential insertion path. We performed (a) a set of experiments to evaluate the device positioning accuracy outside the MR-bore, and (b) an in vivo experiment to evaluate the improvement of targeting accuracy combining straight and angulated insertions in animal models (swine, ). RESULTS: We analyzed 46 in vivo insertions using either straight or angulated insertions paths. The experiment showed that the proposed strategy of selecting straight or angulated insertions based on the subject's anatomy outperformed the conventional approach of just straight insertions in terms of targeting accuracy (2.4 mm [1.3-3.7] vs 3.9 mm [2.4-5.0] {Median ); p = 0.041 after the bias correction). CONCLUSION: The in vivo experiment successfully demonstrated that an angulated needle insertion path could improve needle placement accuracy with a path planning strategy that takes account of the subject-specific anatomical structures.
Sedghi A, O'Donnell LJ, Kapur T, Learned-Miller E, Mousavi P, Wells WM. Image Registration: Maximum Likelihood, Minimum Entropy and Deep Learning. Med Image Anal. 2021;69 :101939.Abstract
In this work, we propose a theoretical framework based on maximum profile likelihood for pairwise and groupwise registration. By an asymptotic analysis, we demonstrate that maximum profile likelihood registration minimizes an upper bound on the joint entropy of the distribution that generates the joint image data. Further, we derive the congealing method for groupwise registration by optimizing the profile likelihood in closed form, and using coordinate ascent, or iterative model refinement. We also describe a method for feature based registration in the same framework and demonstrate it on groupwise tractographic registration. In the second part of the article, we propose an approach to deep metric registration that implements maximum likelihood registration using deep discriminative classifiers. We show further that this approach can be used for maximum profile likelihood registration to discharge the need for well-registered training data, using iterative model refinement. We demonstrate that the method succeeds on a challenging registration problem where the standard mutual information approach does not perform well.
Moreira P, Tuncali K, Tempany CM, Tokuda J. The Impact of Placement Errors on the Tumor Coverage in MRI-Guided Focal Cryoablation of Prostate Cancer. Acad Radiol. 2021;28 (6) :841-8.Abstract
RATIONALE AND OBJECTIVES: There have been multiple investigations defining and reporting the effectiveness of focal cryoablation as a treatment option for organ-confined prostate cancer. However, the impact of cryo-needle/probe placement accuracy within the tumor and gland has not been extensively studied. We analyzed how variations in the placement of the cryo-needles, specifically errors leading to incomplete ablation, may affect prostate cancer's resulting cryoablation. MATERIALS AND METHODS: We performed a study based on isothermal models using Monte Carlo simulations to analyze the impact of needle placement errors on tumor coverage and the probability of positive ablation margin. We modeled the placement error as a Gaussian noise on the cryo-needle position. The analysis used retrospective MRI data of 15 patients with biopsy-proven, unifocal, and MRI visible prostate cancer to calculate the impact of placement error on the volume of the tumor encompassed by the -40°C and -20°C isotherms using one to four cryo-needles. RESULTS: When the standard deviation of the placement error reached 3 mm, the tumor coverage was still above 97% with the -20°C isotherm, and above 81% with the -40°C isotherm using two cryo-needles or more. The probability of positive margin was significantly lower considering the -20°C isotherm (0.04 for three needles) than using the -40°C isotherm (0.66 for three needles). CONCLUSION: The results indicated that accurate cryo-needle placement is essential for the success of focal cryoablation of prostate cancer. The analysis shows that an admissible targeting error depends on the lethal temperature considered and the number of cryo-needles used.
Nitsch J, Sack J, Halle MW, Moltz JH, Wall A, Rutherford AE, Kikinis R, Meine H. MRI-Based Radiomic Feature Analysis of End-Stage Liver Disease for Severity Stratification. Int J Comput Assist Radiol Surg. 2021;16 (3) :457-66.Abstract
PURPOSE: We aimed to develop a predictive model of disease severity for cirrhosis using MRI-derived radiomic features of the liver and spleen and compared it to the existing disease severity metrics of MELD score and clinical decompensation. The MELD score is compiled solely by blood parameters, and so far, it was not investigated if extracted image-based features have the potential to reflect severity to potentially complement the calculated score. METHODS: This was a retrospective study of eligible patients with cirrhosis ([Formula: see text]) who underwent a contrast-enhanced MR screening protocol for hepatocellular carcinoma (HCC) screening at a tertiary academic center from 2015 to 2018. Radiomic feature analyses were used to train four prediction models for assessing the patient's condition at time of scan: MELD score, MELD score [Formula: see text] 9 (median score of the cohort), MELD score [Formula: see text] 15 (the inflection between the risk and benefit of transplant), and clinical decompensation. Liver and spleen segmentations were used for feature extraction, followed by cross-validated random forest classification. RESULTS: Radiomic features of the liver and spleen were most predictive of clinical decompensation (AUC 0.84), which the MELD score could predict with an AUC of 0.78. Using liver or spleen features alone had slightly lower discrimination ability (AUC of 0.82 for liver and AUC of 0.78 for spleen features only), although this was not statistically significant on our cohort. When radiomic prediction models were trained to predict continuous MELD scores, there was poor correlation. When stratifying risk by splitting our cohort at the median MELD 9 or at MELD 15, our models achieved AUCs of 0.78 or 0.66, respectively. CONCLUSIONS: We demonstrated that MRI-based radiomic features of the liver and spleen have the potential to predict the severity of liver cirrhosis, using decompensation or MELD status as imperfect surrogate measures for disease severity.
Tempany-Afdhal CMC. Focal Treatment of Prostate Cancer: MRI Helps Guide the Way Forward. Editorial. Radiology. 2021;298 (3) :704-6.
Noh T, Mustroph M, Golby AJ. Intraoperative Imaging for High-Grade Glioma Surgery. Neurosurg Clin N Am. 2021;32 (1) :47-54.Abstract
This article discusses intraoperative imaging techniques used during high-grade glioma surgery. Gliomas can be difficult to differentiate from surrounding tissue during surgery. Intraoperative imaging helps to alleviate problems encountered during glioma surgery, such as brain shift and residual tumor. There are a variety of modalities available all of which aim to give the surgeon more information, address brain shift, identify residual tumor, and increase the extent of surgical resection. The article starts with a brief introduction followed by a review of with the latest advances in intraoperative ultrasound, intraoperative MRI, and intraoperative computed tomography.