TMI 202107论文速递(IEEE Transactions on Medical Imaging)
1. Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization
用于不成对图像协调的分割重归一化深度特征调制
Mengwei Ren. Neel Dey. James Fishbaugh. Guido Gerig.
原文链接
“Deep networks are now ubiquitous in large-scale multi-center imaging studies. However, the direct aggregation of images across sites is contraindicated for downstream statistical and deep learning-based image analysis due to inconsistent contrast, resolution, and noise. To this end, in the absence of paired data, variations of Cycle-consistent Generative Adversarial Networks have been used to harmonize image sets between a source and target domain. Importantly, these methods are prone to instability, contrast inversion, intractable manipulation of pathology, and steganographic mappings which limit their reliable adoption in real-world medical imaging. In this work, based on an underlying assumption that morphological shape is consistent across imaging sites, we propose a segmentation-renormalized image translation framework to reduce inter-scanner heterogeneity while preserving anatomical layout. We replace the affine transformations used in the normalization layers within generative networks with trainable scale and shift parameters conditioned on jointly learned anatomical segmentation embeddings to modulate features at every level of translation. We evaluate our methodologies against recent baselines across several imaging modalities (T1w MRI, FLAIR MRI, and OCT) on datasets with and without lesions. Segmentation-renormalization for translation GANs yields superior image harmonization as quantified by Inception distances, demonstrates improved downstream utility via post-hoc segmentation accuracy, and improved robustness to translation perturbation and self-adversarial attacks.
2. A Cervical Histopathology Dataset for Computer Aided Diagnosis of Precancerous Lesions
用于计算机辅助诊断癌前病变的宫颈组织病理学数据集
Zhu Meng. Zhicheng Zhao. Bingyang Li. Fei Su. Limei Guo.
原文链接
“Cervical cancer, as one of the most frequently diagnosed cancers worldwide, is curable when detected early. Histopathology images play an important role in precision medicine of the cervical lesions. However, few computer aided algorithms have been explored on cervical histopathology images due to the lack of public datasets. In this article, we release a new cervical histopathology image dataset for automated precancerous diagnosis. Specifically, 100 slides from 71 patients are annotated by three independent pathologists. To show the difficulty of the task, benchmarks are obtained through both fully and weakly supervised learning. Extensive experiments based on typical classification and semantic segmentation networks are carried out to provide strong baselines. In particular, a strategy of assembling classification, segmentation, and pseudo-labeling is proposed to further improve the performance. The Dice coefficient reaches 0.7833, indicating the feasibility of computer aided diagnosis and the effectiveness of our weakly supervised ensemble algorithm. The dataset and evaluation codes are publicly available. To the best of our knowledge, it is the first public cervical histopathology dataset for automated precancerous segmentation. We believe that this work will attract researchers to explore novel algorithms on cervical automated diagnosis, thereby assisting doctors and patients clinically.
3. Learning With Context Feedback Loop for Robust Medical Image Segmentation
使用上下文反馈循环进行稳健医学图像分割的学习
Kibrom Berihu Girum. Gilles Créhange. Alain Lalande.
原文链接
“Deep learning has successfully been leveraged for medical image segmentation. It employs convolutional neural networks (CNN) to learn distinctive image features from a defined pixel-wise objective function. However, this approach can lead to less output pixel interdependence producing incomplete and unrealistic segmentation results. In this paper, we present a fully automatic deep learning method for robust medical image segmentation by formulating the segmentation problem as a recurrent framework using two systems. The first one is a forward system of an encoder-decoder CNN that predicts the segmentation result from the input image. The predicted probabilistic output of the forward system is then encoded by a fully convolutional network (FCN)-based context feedback system. The encoded feature space of the FCN is then integrated back into the forward system’s feed-forward learning process. Using the FCN-based context feedback loop allows the forward system to learn and extract more high-level image features and fix previous mistakes, thereby improving prediction accuracy over time. Experimental results, performed on four different clinical datasets, demonstrate our method’s potential application for single and multi-structure medical image segmentation by outperforming the state of the art methods. With the feedback loop, deep learning methods can now produce results that are both anatomically plausible and robust to low contrast images. Therefore, formulating image segmentation as a recurrent framework of two interconnected networks via context feedback loop can be a potential method for robust and efficient medical image analysis.
4. Cascaded Regression Neural Nets for Kidney Localization and Segmentation-free Volume Estimation
用于肾脏定位和无分割体积估计的级联回归神经网络
Mohammad Arafat Hussain. Ghassan Hamarneh. Rafeef Garbi.
原文链接
“Kidney volume is an essential biomarker for a number of kidney disease diagnoses, for example, chronic kidney disease. Existing total kidney volume estimation methods often rely on an intermediate kidney segmentation step. On the other hand, automatic kidney localization in volumetric medical images is a critical step that often precedes subsequent data processing and analysis. Most current approaches perform kidney localization via an intermediate classification or regression step. This paper proposes an integrated deep learning approach for (i) kidney localization in computed tomography scans and (ii) segmentation-free renal volume estimation. Our localization method uses a selection -convolutional neural network that approximates the kidney inferior-superior span along the axial direction. Cross-sectional (2D) slices from the estimated span are subsequently used in a combined sagittal-axial Mask-RCNN that detects the organ bounding boxes on the axial and sagittal slices, the combination of which produces a final 3D organ bounding box. Furthermore, we use a fully convolutional network to estimate the kidney volume that skips the segmentation procedure. We also present a mathematical expression to approximate the ‘volume error’ metric from the ‘Sørensen–Dice coefficient.’ We accessed 100 patients’ CT scans from the Vancouver General Hospital records and obtained 210 patients’ CT scans from the 2019 Kidney Tumor Segmentation Challenge database to validate our method. Our method produces a kidney boundary wall localization error of ~2.4mm and a mean volume estimation error of ~5%.
5. Direct Differentiation of Pathological Changes in the Human Lung Parenchyma With Grating-Based Spectral X-ray Dark-Field Radiography
用基于光栅的光谱 X 射线暗场射线照相直接鉴别人肺实质的病理变化
Kirsten Taphorn. Korbinian Mechlem. Thorsten Sellerer. Fabio De Marco. Manuel Viermetz. Franz Pfeiffer. Daniela Pfeiffer. Julia Herzen.
原文链接
“Diagnostic lung imaging is often associated with high radiation dose and lacks sensitivity, especially for diagnosing early stages of structural lung diseases. Therefore, diagnostic imaging methods are required which provide sound diagnosis of lung diseases with a high sensitivity as well as low patient dose. In small animal experiments, the sensitivity of grating-based X-ray dark-field imaging to structural changes in the lung tissue was demonstrated. The energy-dependence of the X-ray dark-field signal of lung tissue is a function of its microstructure and not yet known. Furthermore, conventional X-ray dark-field imaging is not capable of differentiating different types of pathological changes, such as fibrosis and emphysema. Here we demonstrate the potential diagnostic power of grating-based X-ray dark-field in combination with spectral imaging in human chest radiography for the direct differentiation of lung diseases. We investigated the energy-dependent linear diffusion coefficient of simulated lung tissue with different diseases in wave-propagation simulations and validated the results with analytical calculations. Additionally, we modeled spectral X-ray dark-field chest radiography scans to exploit these differences in energy-dependency. The results demonstrate the potential to directly differentiate structural changes in the human lung. Consequently, grating-based spectral X-ray dark-field imaging potentially contributes to the differential diagnosis of structural lung diseases at a clinically relevant dose level.
6. Development and Initial Results of a Brain PET Insert for Simultaneous 7-Tesla PET/MRI Using an FPGA-Only Signal Digitization Method
使用仅 FPGA 的信号数字化方法开发用于同时进行 7-Tesla PET/MRI 的脑 PET 插入物的开发和初步结果
Jun Yeon Won. Haewook Park. Seungeun Lee. Jeong-Whan Son. Yina Chung. Guen Bae Ko. Kyeong Yun Kim. Junghyun Song. Seongho Seo. Yeunchul Ryu. Jun-Young Chung. Jae Sung Lee.
原文链接
“In study, we developed a positron emission tomography (PET) insert for simultaneous brain imaging within 7-Tesla (7T) magnetic resonance (MR) imaging scanners. The PET insert has 18 sectors, and each sector is assembled with two-layer depth-of-interaction (DOI)-capable high-resolution block detectors. The PET scanner features a 16.7-cm-long axial field-of-view (FOV) to provide entire human brain images without bed movement. The PET scanner early digitizes a large number of block detector signals at a front-end data acquisition (DAQ) board using a novel field-programmable gate array (FPGA)-only signal digitization method. All the digitized PET data from the front-end DAQ boards are transferred using gigabit transceivers via non-magnetic high-definition multimedia interface (HDMI) cables. A back-end DAQ system provides a common clock and synchronization signal for FPGAs over the HDMI cables. An active cooling system using copper heat pipes is applied for thermal regulation. All the 2.17-mm-pitch crystals with two-layer DOI information were clearly identified in the block detectors, exhibiting a system-level energy resolution of 12.6%. The PET scanner yielded clear hot-rod and Hoffman brain phantom images and demonstrated 3D PET imaging capability without bed movement. We also performed a pilot simultaneous PET/MR imaging study of a brain phantom. The PET scanner achieved a spatial resolution of 2.5 mm at the center FOV (NU 4) and a sensitivity of 18.9 kcps/MBq (NU 2) and 6.19% (NU 4) in accordance with the National Electrical Manufacturers Association (NEMA) standards.
7. Multi-Modal Retinal Image Classification With Modality-Specific Attention Network
具有特定模态注意网络的多模态视网膜图像分类
Xingxin He. Ying Deng. Leyuan Fang. Qinghua Peng.
原文链接
“Recently, automatic diagnostic approaches have been widely used to classify ocular diseases. Most of these approaches are based on a single imaging modality (e.g., fundus photography or optical coherence tomography (OCT)), which usually only reflect the oculopathy to a certain extent, and neglect the modality-specific information among different imaging modalities. This paper proposes a novel modality-specific attention network (MSAN) for multi-modal retinal image classification, which can effectively utilize the modality-specific diagnostic features from fundus and OCT images. The MSAN comprises two attention modules to extract the modality-specific features from fundus and OCT images, respectively. Specifically, for the fundus image, ophthalmologists need to observe local and global pathologies at multiple scales (e.g., from microaneurysms at the micrometer level, optic disc at millimeter level to blood vessels through the whole eye). Therefore, we propose a multi-scale attention module to extract both the local and global features from fundus images. Moreover, large background regions exist in the OCT image, which is meaningless for diagnosis. Thus, a region-guided attention module is proposed to encode the retinal layer-related features and ignore the background in OCT images. Finally, we fuse the modality-specific features to form a multi-modal feature and train the multi-modal retinal image classification network. The fusion of modality-specific features allows the model to combine the advantages of fundus and OCT modality for a more accurate diagnosis. Experimental results on a clinically acquired multi-modal retinal image (fundus and OCT) dataset demonstrate that our MSAN outperforms other well-known single-modal and multi-modal retinal image classification methods.
8. Learning Tubule-Sensitive CNNs for Pulmonary Airway and Artery-Vein Segmentation in CT
在 CT 中学习用于肺气道和动脉静脉分割的小管敏感 CNN
Yulei Qin. Hao Zheng. Yun Gu. Xiaolin Huang. Jie Yang. Lihui Wang. Feng Yao. Yue-Min Zhu. Guang-Zhong Yang.
原文链接
“Training convolutional neural networks (CNNs) for segmentation of pulmonary airway, artery, and vein is challenging due to sparse supervisory signals caused by the severe class imbalance between tubular targets and background. We present a CNNs-based method for accurate airway and artery-vein segmentation in non-contrast computed tomography. It enjoys superior sensitivity to tenuous peripheral bronchioles, arterioles, and venules. The method first uses a feature recalibration module to make the best use of features learned from the neural networks. Spatial information of features is properly integrated to retain relative priority of activated regions, which benefits the subsequent channel-wise recalibration. Then, attention distillation module is introduced to reinforce representation learning of tubular objects. Fine-grained details in high-resolution attention maps are passing down from one layer to its previous layer recursively to enrich context. Anatomy prior of lung context map and distance transform map is designed and incorporated for better artery-vein differentiation capacity. Extensive experiments demonstrated considerable performance gains brought by these components. Compared with state-of-the-art methods, our method extracted much more branches while maintaining competitive overall segmentation performance. Codes and models are available at http://www.pami.sjtu.edu.cn/News/56 .
9. 3D Multi-Attention Guided Multi-Task Learning Network for Automatic Gastric Tumor Segmentation and Lymph Node Classification
用于自动胃肿瘤分割和淋巴结分类的 3D 多注意力引导多任务学习网络
Yongtao Zhang. Haimei Li. Jie Du. Jing Qin. Tianfu Wang. Yue Chen. Bing Liu. Wenwen Gao. Guolin Ma. Baiying Lei.
原文链接
“Automatic gastric tumor segmentation and lymph node (LN) classification not only can assist radiologists in reading images, but also provide image-guided clinical diagnosis and improve diagnosis accuracy. However, due to the inhomogeneous intensity distribution of gastric tumor and LN in CT scans, the ambiguous/missing boundaries, and highly variable shapes of gastric tumor, it is quite challenging to develop an automatic solution. To comprehensively address these challenges, we propose a novel 3D multi-attention guided multi-task learning network for simultaneous gastric tumor segmentation and LN classification, which makes full use of the complementary information extracted from different dimensions, scales, and tasks. Specifically, we tackle task correlation and heterogeneity with the convolutional neural network consisting of scale-aware attention-guided shared feature learning for refined and universal multi-scale features, and task-aware attention-guided feature learning for task-specific discriminative features. This shared feature learning is equipped with two types of scale-aware attention (visual attention and adaptive spatial attention) and two stage-wise deep supervision paths. The task-aware attention-guided feature learning comprises a segmentation-aware attention module and a classification-aware attention module. The proposed 3D multi-task learning network can balance all tasks by combining segmentation and classification loss functions with weight uncertainty. We evaluate our model on an in-house CT images dataset collected from three medical centers. Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, and obtains promising performance for tumor segmentation and LN classification. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge. Our implementation is released at https://github.com/infinite-tao/MA-MTLN .
10. Relation-Induced Multi-Modal Shared Representation Learning for Alzheimer’s Disease Diagnosis
用于阿尔茨海默病诊断的关系诱导的多模态共享表示学习
Zhenyuan Ning. Qing Xiao. Qianjin Feng. Wufan Chen. Yu Zhang.
原文链接
“The fusion of multi-modal data (e.g., magnetic resonance imaging (MRI) and positron emission tomography (PET)) has been prevalent for accurate identification of Alzheimer’s disease (AD) by providing complementary structural and functional information. However, most of the existing methods simply concatenate multi-modal features in the original space and ignore their underlying associations which may provide more discriminative characteristics for AD identification. Meanwhile, how to overcome the overfitting issue caused by high-dimensional multi-modal data remains appealing. To this end, we propose a relation-induced multi-modal shared representation learning method for AD diagnosis. The proposed method integrates representation learning, dimension reduction, and classifier modeling into a unified framework. Specifically, the framework first obtains multi-modal shared representations by learning a bi-directional mapping between original space and shared space. Within this shared space, we utilize several relational regularizers (including feature-feature, feature-label, and sample-sample regularizers) and auxiliary regularizers to encourage learning underlying associations inherent in multi-modal data and alleviate overfitting, respectively. Next, we project the shared representations into the target space for AD diagnosis. To validate the effectiveness of our proposed approach, we conduct extensive experiments on two independent datasets (i.e., ADNI-1 and ADNI-2), and the experimental results demonstrate that our proposed method outperforms several state-of-the-art methods.
11. Hierarchical Temporal Attention Network for Thyroid Nodule Recognition Using Dynamic CEUS Imaging
使用动态 CEUS 成像进行甲状腺结节识别的分层时间注意网络
Peng Wan. Fang Chen. Chunrui Liu. Wentao Kong. Daoqiang Zhang.
原文链接
“Contrast-enhanced ultrasound (CEUS) has emerged as a popular imaging modality in thyroid nodule diagnosis due to its ability to visualize vascular distribution in real time. Recently, a number of learning-based methods are dedicated to mine pathological-related enhancement dynamics and make prediction at one step, ignoring a native diagnostic dependency. In clinics, the differentiation of benign or malignant nodules always precedes the recognition of pathological types. In this paper, we propose a novel hierarchical temporal attention network (HiTAN) for thyroid nodule diagnosis using dynamic CEUS imaging, which unifies dynamic enhancement feature learning and hierarchical nodules classification into a deep framework. Specifically, this method decomposes the diagnosis of nodules into an ordered two-stage classification task, where diagnostic dependency is modeled by Gated Recurrent Units (GRUs). Besides, we design a local-to-global temporal aggregation (LGTA) operator to perform a comprehensive temporal fusion along the hierarchical prediction path. Particularly, local temporal information is defined as typical enhancement patterns identified with the guidance of perfusion representation learned from the differentiation level. Then, we leverage an attention mechanism to embed global enhancement dynamics into each identified salient pattern. In this study, we evaluate the proposed HiTAN method on the collected CEUS dataset of thyroid nodules. Extensive experimental results validate the efficacy of dynamic patterns learning, fusion and hierarchical diagnosis mechanism.
12. Measurement of the Lag Correction Factor in Low-Dose Fluoroscopic Imaging
低剂量透视成像中滞后校正因子的测量
Dong Sik Kim. Eunae Lee.
原文链接
“Lag signals occur at images sequentially acquired from a flat-panel (FP) dynamic detector in fluoroscopic imaging due to charge trapping in photodiodes and incomplete readouts. This lag signal produces various lag artifacts and prevents analyzing detector performances because the measured noise power spectrum (NPS) values are reduced. In order to design dynamic detectors, which produce low lag artifacts, accurately evaluating the detector lag through its quantitative measurement is required. A lag correction factor can be used to both examine the detector lag and correct measured NPS. To measure the lag correction factor, the standard of IEC62220-1-3 suggests a temporal power spectral density under a constant potential generator for the x-rays. However, this approach is sensitive to disturbing noise and thus becomes a problem in obtaining accurate estimates especially at low doses. The Granfors-Aufrichtig (GA) method is appropriate for noisy environments with a synchronized pulse x-ray source. However, for the x-ray source of a constant potential generator, gate-line scanning to read out charges produces a nonuniform lag signal within each image frame and thus the conventional GA method yields wrong estimates. In this paper, we first analyze the GA method and show that the method is an asymptotically unbiased estimate. Based on the GA method, we then propose three algorithms considering the scanning process and exposure leak, in which line estimates along the gate line are exploited. We extensively conducted experiments for FP dynamic detectors and compared the results with conventional algorithms.
13. Identify Complex Imaging Genetic Patterns via Fusion Self-Expressive Network Analysis
通过融合自我表达网络分析识别复杂的成像遗传模式
Meiling Wang. Wei Shao. Xiaoke Hao. Daoqiang Zhang.
原文链接
“In the brain imaging genetic studies, it is a challenging task to estimate the association between quantitative traits (QTs) extracted from neuroimaging data and genetic markers such as single-nucleotide polymorphisms (SNPs). Most of the existing association studies are based on the extensions of sparse canonical correlation analysis (SCCA) for the identification of complex bi-multivariate associations, which can take the specific structure and group information into consideration. However, they often take the original data as input without considering its underlying complex multi-subspace structure, which will deteriorate the performance of the following integrative analysis. Accordingly, in this paper, the self-expressive property is exploited for the reconstruction of the original data before the association analysis, which can well describe the similarity structure. Specifically, we first apply the within-class similarity information to construct self-expressive networks by sparse representation. Then, we use the fusion method to iteratively fuse the self-expressive networks from multi-modality brain phenotypes into one network. Finally, we calculate the imaging genetic association based on the fused self-expressive network. We conduct the experiments on both single-modality and multi-modality phenotype data. Related experimental results validate that our method can not only better estimate the potential association between genetic markers and quantitative traits but also identify consistent multi-modality imaging genetic biomarkers to guide the interpretation of Alzheimer’s disease.
14. Modeling and Synthesis of Breast Cancer Optical Property Signatures With Generative Models
使用生成模型对乳腺癌光学特性特征进行建模和合成
Arturo Pardo. Samuel S. Streeter. Benjamin W. Maloney. José A. Gutiérrez-Gutiérrez. David M. McClatchy. Wendy A. Wells. Keith D. Paulsen. José M. López-Higuera. Brian W. Pogue. Olga M. Conde.
原文链接
“Is it possible to find deterministic relationships between optical measurements and pathophysiology in an unsupervised manner and based on data alone? Optical property quantification is a rapidly growing biomedical imaging technique for characterizing biological tissues that shows promise in a range of clinical applications, such as intraoperative breast-conserving surgery margin assessment. However, translating tissue optical properties to clinical pathology information is still a cumbersome problem due to, amongst other things, inter- and intrapatient variability, calibration, and ultimately the nonlinear behavior of light in turbid media. These challenges limit the ability of standard statistical methods to generate a simple model of pathology, requiring more advanced algorithms. We present a data-driven, nonlinear model of breast cancer pathology for real-time margin assessment of resected samples using optical properties derived from spatial frequency domain imaging data. A series of deep neural network models are employed to obtain sets of latent embeddings that relate optical data signatures to the underlying tissue pathology in a tractable manner. These self-explanatory models can translate absorption and scattering properties measured from pathology, while also being able to synthesize new data. The method was tested on a total of 70 resected breast tissue samples containing 137 regions of interest, achieving rapid optical property modeling with errors only limited by current semi-empirical models, allowing for mass sample synthesis and providing a systematic understanding of dataset properties, paving the way for deep automated margin assessment algorithms using structured light imaging or, in principle, any other optical imaging technique seeking modeling. Code is available.
15. Multi-Atlas Image Soft Segmentation via Computation of the Expected Label Value
通过计算预期标签值的多图集图像软分割
Iman Aganj. Bruce Fischl.
原文链接
“The use of multiple atlases is common in medical image segmentation. This typically requires deformable registration of the atlases (or the average atlas) to the new image, which is computationally expensive and susceptible to entrapment in local optima. We propose to instead consider the probability of all possible atlas-to-image transformations and compute the expected label value (ELV) , thereby not relying merely on the transformation deemed “optimal” by the registration method. Moreover, we do so without actually performing deformable registration, thus avoiding the associated computational costs. We evaluate our ELV computation approach by applying it to brain, liver, and pancreas segmentation on datasets of magnetic resonance and computed tomography images.