mr成像
MR imaging is a powerful and diverse imaging technique employed to investigate and diagnose a range of diseases in different body areas. MRI scans are acquired by employing specific parameters in a “sequence” to encode in data in arbitrary space known as “k-space”. Image is reconstructed by applying mathematical transforms (mainly Fourier) to the k-space data. To obtain images of particular contrast (T1w, T2w, T2* etc) optimal sequence settings must be employed. Briefly, image contrast arise from magnetic property of the hydrogen atoms, that can varied setting such as echo time and Tr in sequence setting. These are obtained by solving bloch equations, and the settings needs to be predetermined before starting the acquisition process. To acquire image each line of k-space needs to be acquired using the sequence with these predetermined settings. This leads to slow encoding processing, and also leads to longer scans duration. Therefore can we apply AI to automate this process to obtain optimal images in short scan duration. We start by briefly discussing AI methods currently employed in medical imaging.
MR成像是一种强大而多样的成像技术,可用于研究和诊断不同身体部位的一系列疾病。 通过在“序列”中采用特定参数以在称为“ k空间”的任意空间中的数据中进行编码来获取MRI扫描。 通过对k空间数据应用数学变换(主要是傅里叶)来重建图像。 为了获得具有特殊对比度(T1w,T2w,T2 *等)的图像,必须采用最佳顺序设置。 简而言之,图像对比度是由氢原子的磁性引起的,该磁性可以改变诸如回波时间和序列设置中的Tr之类的设置。 这些可以通过求解bloch方程获得,并且需要在开始采集过程之前预先确定设置。 为了获取图像,需要使用具有这些预定设置的序列来获取k空间的每一行。 这导致编码处理变慢,并且还导致更长的扫描持续时间。 因此,我们可以应用AI来自动执行此过程以在短扫描时间内获得最佳图像。 我们首先简要讨论当前在医学成像中使用的AI方法。
AI in medical imaging: Much of AI developments focus on the segmentation of images to automate analysis work flow. A generic widely employed network architecture is U-NET, which have been further optimised for range of applications. Some alternate task in cardiac MRI we are also outline here.
医学成像中的AI : AI的许多发展都集中在图像分割上,以实现分析工作流程的自动化。 U-NET是广泛使用的通用网络体系结构,已针对各种应用程序进行了进一步优化。 我们在这里也概述了心脏MRI的一些替代任务。
There are efforts to speed up the MRI acquisition by using AI techniques for reconstruction process. The accelerating is achieved by developing algorithms to reconstruct partially acquired data (in the k-space). Here again main focus is providing novel neural network architecture, see the fastMRI leader board for recent developments.
通过使用AI技术进行重建过程,人们正在努力加快MRI的采集速度。 通过开发算法来重建部分采集的数据(在k空间中)可实现加速。 这里的主要重点还是提供新颖的神经网络架构,有关最新发展 ,请参见fastMRI排行榜。
Another area of AI and deep learning which are not as widely investigated in medical imaging is generative modelling. Recent developments of generative modelling in other domains (NLP), demonstrate the power of generative models. They provide a single network which can be applied (fine tuned) to wide range of applications/tasks in NLP domain compared, rather then building application specific architectures. The key architecture is Transformer from Google and which be further improved with generative pre-training from OpenAI, know as GPT-1,2,3. Key tasks include sentiment analysis, text summarisation, translation.
人工智能和深度学习的另一个尚未在医学成像中广泛研究的领域是生成建模。 生成建模在其他领域(NLP)中的最新发展证明了生成模型的强大功能。 它们提供了一个单一的网络,相比于NLP域,该网络可以应用于(微调)广泛的应用程序/任务,而不是构建特定于应用程序的体系结构。 关键架构是Google的Transformer,并通过来自OpenAI的生成式预训练(称为GPT-1,2,3)进行了进一步改进。 关键任务包括情感分析,文本摘要,翻译。
Generative models can also be combined with control methods (e.g. reinforcement learning) to develop autonomous systems. A specific example being world model which using generative model (VAE) along RL to make autonomous decision. SPIRAL which is GAN policy search based architecture employed for the doodling, applications.
生成模型也可以与控制方法(例如强化学习)结合以开发自主系统。 一个具体的示例是世界模型,它沿RL使用生成模型(VAE)进行自主决策。 SPIRAL是用于涂鸦应用程序的基于GAN策略搜索的体系结构。
In post we apply a generative model (VAE) for reconstruction, along with memory network and controller to automate the image acceleration process to select optimal image acceleration. To highlight concept we employ the fastMRI challenge (single channel) data for this work, however the models can combine datasets from other modalities to add from different types of acquisition as we discussed at the end. This is key aspect of this work.
在后期,我们将应用生成模型(VAE)进行重建,并使用存储网络和控制器来自动执行图像加速过程以选择最佳图像加速。 为了突出概念,我们将fastMRI挑战(单通道)数据用于这项工作,但是,如我们在最后所讨论的,这些模型可以将其他方式的数据集相结合,以从不同类型的采集中添加数据。 这是这项工作的关键方面。
Our goal is to automatically select the acceleration factors via using a sampled from a distribution images from partial sampled k-space, rather than on specific set of the acceleration factors obtain on training set on a particular dataset. Below are examples of the fully sample k-space and knee images
我们的目标是通过使用从部分采样k空间的分布图像中采样来自动选择加速因子,而不是根据从特定数据集的训练集获得的特定加速因子集来选择加速因子。 以下是完整样本k空间和膝盖图像的示例
World Model: The world model agent contains three components, as shown below and figure (from the original article)
世界模型 :世界模型代理包含三个组件,如下图和图所示(来自原始文章)
- Vision model (generative, VAE) to obtain simpler representation of the data (FFT recons of under sampled K-space data), know as latent z via an encoder. A MSE loss with targets as fully sampled image and decoded z, therefore VAE also is a reconstructor. 视觉模型(生成式,VAE),用于获取数据的更简单表示形式(采样的K空间数据的FFT重构),通过编码器称为潜伏z。 MSE损失的目标是完全采样的图像并解码了z,因此VAE也是重建器。
- Memory model: world model we use a mixed density network: RNN to obtain probability distribution of latent at the next time point (t+1) z1, condition on current action a, latent z and hidden state, h at time t, i.e. p(z1|a,z,h) 记忆模型:世界模型,我们使用混合密度网络:RNN获取下一个时间点(t + 1)z1的潜在概率分布,当前动作a的条件,潜在z和隐藏状态h在时间t,即p (z1 | a,z,h)
- Controller: We train a Linear controller to map the latent z and hidden state h to the action a at time t. 控制器:我们训练了一个线性控制器,以在时间t将潜在z和隐藏状态h映射到动作a。
In accordance with world models we select the follows and code please see the link here
根据世界模型,我们选择以下代码,请参见此处的链接
- Train VAE for reconstruction: latent samples were generated with image x after application of acceleration factors 训练VAE以进行重建:应用加速因子后,使用图像x生成潜在样本
- Train an MDN RNN for obtain probability distribution of latent at every time point. Not in our environment the episode duration is always equal to one. 训练MDN RNN以获取每个时间点的潜在概率分布。 不在我们的环境中,情节持续时间始终等于1。
- Train Controller: A simple learning controller with action dim=2 (acceleration factor and centre fraction): 火车控制器:一个简单的学习控制器,其动作dim = 2(加速度因子和中心分数):
We highlight suggestions to improve in four areas in light of recent advancements in the literature
根据文献的最新进展,我们重点介绍了在四个方面进行改进的建议
Generative modelling:
生成建模 :
In this work we employed a standard VAE, which is know to produce blurred images to due to the use of mean square loss. Advanced generative models could be employed to improved image samples. Specifically, VAE-VQ-2 which employ autoregressive approaches to avoid gaussian assumption latent prior to obtain high resolution images. DC-GAN with Wasserstein Loss is another choice, but they are hard to train for large image sizes, particularity for density/probability distribution estimation and prone to sampling diversity issues in limited data req. Normalising flow as discussed here, provide a theoretically grounded approach for optimisation. Image based Transformer architecture is also appearing in literature, however the work mainly on lower resolution images. Contrastive predictive coding is a promising technique which can be employed to obtain efficient encoders for generative models, particularly in low data regimes like medical imaging.
在这项工作中,我们采用了标准VAE,由于使用均方损失,已知会产生模糊图像。 先进的生成模型可用于改进图像样本。 具体而言, VAE-VQ-2在获得高分辨率图像之前采用自回归方法来避免潜在的高斯假设。 具有Wasserstein损失的DC-GAN是另一种选择,但是它们很难训练大图像尺寸,密度/概率分布估计的特殊性,并且在有限的数据需求中易于采样多样性问题。 所讨论的规范化流程在这里 ,为优化理论基础的方法。 基于图像的Transformer体系结构也出现在文献中,但是该工作主要针对较低分辨率的图像。 对比预测编码是一种有前途的技术,可用于获得生成模型的有效编码器,尤其是在医学成像等低数据方案中。
2. Controller:
2.控制器:
World model learns a simple controller, and for this task, in principal we can employed bayesian optimisation used in auto-ml. World models however is frame work, in which we can experiments with different environments, generative models, and/or control methods. For e.g. PILCO can model dynamics (bloch equations) which form basis for a wider variety of tasks as we outline shortly. PILCO is model based reinforcement learning technique which is sample efficient. PICLO can also accommodate constrains to selecting our actions, e.g. we should not select -ve acceleration factor for example.
世界模型学习一个简单的控制器,对于这一任务,原则上我们可以采用auto-ml中使用的贝叶斯优化。 但是,世界模型是框架,我们可以在其中使用不同的环境,生成模型和/或控制方法进行实验。 例如, PILCO可以对动力学( 布洛方程)进行建模,这将构成我们稍后概述的各种任务的基础。 PILCO是基于模型的强化学习技术,具有高效的样本。 PICLO还可以适应选择我们的动作的约束,例如,我们不应选择-ve加速因子。
3. Reward and environment design:
3.奖励与环境设计 :
In this set up we employed a very simple reward design with average sum of square error for entire 3D stack of images with time/episode duration of 1. A much more sophisticated reward function based on physical model can be also be employed, and likely to play a key consideration in design framework. Environment design via a Bloch simulator to change the sequence setting on the fly based latent (z) is what will really useful.
在此设置中,我们采用了一个非常简单的奖励设计,对于时间/间隔持续时间为1的整个3D图像堆栈,其均方差平均为总和。还可以使用基于物理模型的更为复杂的奖励功能,并且可能在设计框架中起关键作用。 通过Bloch模拟器进行环境设计以更改基于飞行的潜伏(z)的序列设置是真正有用的。
4. Related areas in MRI acquisitions, where we can apply world model framework.
4. MRI采集中的相关领域,我们可以在其中应用世界模型框架。
Motion correction via predicting the next image in cardiac/body/time series imaging can be a potentail application via this framework. We can employ RNN to predict latent vector z (i.e. next frame) which is motion free, actions can be transformations/correction of physical or motion correction model, such a model can be learned. This is natural extension of our work from single time step to multiple time steps.
通过运动校正 通过此框架,预测心脏/身体/时间序列成像中的下一幅图像可能是一个潜在的应用。 我们可以采用RNN来预测不运动的潜矢量z(即下一帧),动作可以是物理或运动校正模型的变换/校正,可以学习这种模型。 这是我们工作从单个时间步长扩展到多个时间步长的自然扩展。
Diffusion imaging: PICLO (gaussian process based) controllers are particularly interesting for diffusion imaging acquisitions. PILCO employs gaussian processes to model system dynamics (environment). Gaussian process are employed for post processing of diffusion imaging data and more recently for speed up acquisition too in “q space imaging” q space is k space equivalent for diffusion imaging. Therefore we potentially can employ PILCO and both controller and diffusion environment model in the framework.
扩散成像: PICLO(基于高斯过程)控制器对于扩散成像采集特别有趣。 PILCO采用高斯过程对系统动力学(环境)进行建模。 高斯过程用于扩散成像数据的后处理,并且最近也用于在“ q空间成像”中加快采集速度。q空间等效于k空间用于扩散成像。 因此,我们有可能在框架中采用PILCO以及控制器和扩散环境模型。
MR fingerprinting related approach may be feasible to where to controller outputs sequence settings via using the dictionary matching technique change the acquisition setting, on the fly. Here the environment model will include a Gaussian process based model for bloch equations.
与MR指纹识别相关的方法对于通过使用字典匹配技术实时更改采集设置来控制输出序列设置的位置是可行的。 这里的环境模型将包括基于高斯过程的布洛克方程模型。
Acknowledgements: Google Cloud for research credits, David Ha for open sourcing his work and Professor Marc Deisenroth for helpful discussions on PILCO.
致谢: Google Cloud获得了研究学分,David Ha获得了开源工作,Marc Deisenroth教授获得了有关PILCO的有用讨论。
翻译自: https://medium.com/@jehillparikh/towards-autonomous-mr-imaging-using-world-models-accacce00b5a
mr成像