- 主要是为了提高写作能力，之前从来没有精度过文章，初看文章感觉语法和表达什么的都挺好的,reviewer的评价也是
Overall this is a well written paper. The presentation表述 of their methodology is clear, so are the numerical studies.什么什么也是clear的，so are the ..A.. 对于A来说也是如此，于是就选这篇文章进行精度了。
abstract
-
Despite much effort,而不是说尽管花费了很多努力在上面。 -
deep neural networks remain神经网络用复数。 - Despite much effort, deep neural networks remain highly susceptible to tiny input perturbations and
even for MNIST, one of the most common toy datasets in computer vision,no neural network model exists for which adversarial perturbations are large and make semantic sense to humans.and连接的句子，中间加了一个补足语，但是感觉which这里用where更好一点啊？？？用which是没错的which在此处是一个限定词，相当于exists for adversarial perturbations which are large and make sense
1. Which, can be used both before and after as a pronoun
and determiner. Here are some further examples.
Which coffee would you like, the cappuccino or expresso?
The cappuccino has milk, but the expresso doesn’t,
which one do you want?
A cappuccino is not as strong as an expresso which has
no milk.
2. The restaurant where my cousin works is really
expensive. where仅仅作为一个连接词，不在从句中做任何成分
The where in this sentence is to not referring to the place
but the situation of the cousin, because it was used after
the place had already been mentioned. To prove this
point, if we removed this part of the clause, the sentence
still makes sense – The restaurant is really expensive.
However, if we reword the sentence and use which as a
determiner, the focus of the sentence returns to the
place/restaurant as we are also using ‘at’ as a
preposition of place.
* The restaurant which my cousin works at is really
expensive. which的话要在从句中做成分
3. this IS THE hotel where we spent our summer last
year. Again the use of where in this sentence is to the
situation, not the hotel, as it comes after the place has
already been mentioned. To prove the point we could
eliminate the word entirely and use the preposition ‘at’
instead. 不用where就要从句中加上介词at
This is the hotel we spent our summer at last year.
* If you are focusing on a situation or place use where.
* If you are making a distinction between two or more
things, then use which.
也就是说which是强调两件事情的不同的，而where是用来
聚焦于地点或者情况的。
what与which在名词性从句中都可以指代物,共同担任的成
分有：主语、宾语、表语和宾语补足语.由于以上的共同
点,因此题目中经常会用这两个选项进行PK.
区别：what无范围,which有范围.
这点区别只要确定了什么情况算是有范围就可以区分开.那么有范围的情况：
题目中明确列出几个事物的名称；Do you know which
drink she likes,tea,coffee,or milk?这类题目高中的名词性
从句是不出的.因为比较简单.
题目中虽然没有列出名称,但是写出了数目；As there are
five courses,you are free to choose which you would like
to attend.
题目中没有列出名称,也没有数目,但是题目中提到的事物,
说话人与听话人都心知肚明其有固定选择的范围.如关注一
场体育赛事,谈到半决赛或决赛的时候,入围的数目是确定
的,因此范围也固定.It is unknown which team will win in
the final.
-
no ..B.. exists for..A..对于A还没有B ，例句At present, no widely approved vaccine exists for malaria. - We show that even the
widely recognized认可 and by far most successful广泛认可并且迄今为止最为成功by far 迄今为止L1 defenseby Madry et al. madry 等人提出的et al. 等人(1) has lower L0 robustness than undefendednetworks基本用复数.and is still highly susceptible to L2perturbations基本用复数,classifies可以看到三个动作并列的话A and B,Cunrecognizable images with highcertainty确定性是抽象名词,performs not much better than不比什么表现好simple input binarization and (4)featuresadversarial perturbations thatmake little sense to humans.对人类来说没什么意义..A.. features ..B.. A 有(比有的色彩更加强烈一点，大概是主要有，特色是有)B. These resultssuggest表明that MNIST isfar from being solved远没有被解决in terms of adversarial robustness在对抗robustness方面.Wepresent提出a novel robust classification model that performs analysis by synthesis using learned class-conditional data distributions.Wederive推导出bounds on the robustness此处显然不能用of，因为of the robustness是robustness本身的一个特性，on的话表示是模型在robustness上的一个boundsandgo to great length to竭尽全力做某事empirically evaluate经验性评估，意思是通过很多实验来评估our model using maximally effective adversarial attacks.go to great length to 表示someone goes to great lengths to achieve something, you mean that they try very hard and perhaps do extreme things in order to achieve it.The resultssuggest表明that our approachyields达到state-of-the-art robustness on MNIST against L0, L2 and L1 perturbations andwe demonstratethat most adversarial examples are strongly perturbed towards the perceptual boundary between the original and the adversarial class.
INTRODUCTION
- Deep neural networks (DNNs) are
strikingly特别的 醒目的susceptible to minimal adversarial perturbations (Szegedy et al., 2013),perturbations that对前面的进行补充说明are (almost)imperceptible细微的，感觉不到的to humansbut which不能连用两个thatcan switch the class prediction of DNNs to basically any desired target class.
One key problem一个关键问题in介词用in，problem infinding successful defenses isthe difficulty of reliably evaluating可靠的评估是很困难的，使用了n+of+adj+ving的形式表示可靠评估 **的** 困难性model robustness.It has been showntime and again一次又一次的被说明后面跟上很多的引用...怎么样(Athalye et al., 2018; Athalye & Carlini, 2018; Brendel &Bethge, 2017) thatit has been shown that已经证明basically基本上，句子前面放一个副词all defenses previously proposed因为已经有个all限定了，所以把修饰部分后置did not因为是过去已经证明的东西，所以用过去式increase model robustness butprevented和did not是并列的所以是过去式existing adj 目前的现有的attacksfrom finding prevent from doing somthingminimal adversarial examples,the most common reason being masking of the gradients on which most attacks rely （on which attacks rely on）.最常见的原因是掩盖了大多数攻击所依赖的梯度因为用逗号隔开，不能有两个谓语，因此后面的独立主格结构表示原因，翻译成主要的原因是梯度被mask掉了，梯度被mask掉了不说the most common reason being that gradients are masked而是用了一个简洁的结构the most common平凡的解释是 reason being masking of gradients,因此一个被动语态gradients are masked可以转换成一个of的结构masking of gradients，之前的从句太长了此处万万不能再用一个从句了。The few verifiable defensescan only guarantee robustnesswithin在一个很小的范围内根本不用用从句来表述，一个within就行了a small linear regime around the data points.(Hein & Andriushchenko, 2017; Raghunathan et al., 2018). - The only defense
currently considered effective (Athalye et al., 2018)后置，现在被认为有效的is a particular type of adversarial training (Madry et al., 2018).On MNIST, as of today截止今天 so far 也可以表示截止今天， So far 57 have taken the test and all have been negative. ，句子用过去完成时表示在说话之前已经this method is able to reach an accuracy of 88:79%foradversarial perturbationswithan L1 norm boundedby� = 0:3 (Zheng et al., 2018).这三个介词用的很难啊，n + with n 是后一个名词对前一个名词作补充，by a=0.3 by使用超参a=0.3
as of today 有两种截然相反的意思，一种意思是截止现
在，等于so far 一种是从今天开始，甚至还有不常用的第
三种意思，指的是仅仅在今天。
The different meanings of as of today
As of today can mean “from the beginning up until now,
including today,” as in this example:
As of today, only three survivors have been found.
This meaning is close to the meaning of the expression
so far.
On the other hand, it can also mean “starting today and
going forward into the future,” as in this example:
As of today, all passengers must check their luggage
before boarding the plane.
This meaning is close to the meaning of the expression
going forward.
As of today even has a third meaning, which is less
common than the other two. It can mean “today, only”
with the implication that things are likely to change.
介词for的用法
(1) 表示动作的目的,意图或利益,可翻译为”为……”
I want to go back for my pen.
我要回去拿我的钢笔.
Let’s go for a walk.
我们去散步吧.
Smoking is not good for the health.
吸烟有害健康.
(2) 表示用途,意为”适用于,适合,给”
Here are some bags for sports.
这里有许多运动包.
This is a book for children.
这是一本适合孩子读的书.
(3) 表示动作的方向,目的地,意为”向…….,往……”
He left for shanghai last week.
他上周出发去了上海.
This ship is for [NY.]
这艘船是开往纽约的.
(4) 表示时间,距离
We will stay there for two weeks.
我们会在那里待两周.
I usually walk for two hours.
我经常散步两小时.
（5) 表示原因,理由,常与”thank, famous连用”
Thank you for your help.
谢谢你的帮助.
Jackie chan is famous for his movie.
成龙因为他的电影而著名.
(6) 表示赞成,支持,反义词为”against”
Are you for the plan or against it?
你是支持这个计划还是反对它?
(7) 用在一些固定搭配中
look for 寻找
wait for 等待
be late for 迟到
In other words, if we allow an attacker to perturb the brightness of each pixel by up to 0.3 不超过 (range [0; 1]),then he can only trick the model on = 10% on sample of the samples. This is a成功可数 great success, but does the model really learn more causal features to classify MNIST? We here demonstrate that this is not the case情况并非如此:For one首先, the defense by Madry et al. (SOTA on L1) has lower L0 robustness than undefended networks and is still highly susceptible in the L2 metric. Second,其次 the robustness results by Madry et al. can also be achieved with a simple input quantization because of the binary nature of single pixels in MNIST (which are typically either completely black or white) (Schmidt et al., 2018).Third, it is straight-forward简单的，直接的 to find unrecognizable难以辨认的 images that are classified as a digit with high certainty高置信度. Finally, the minimum adversarial examples we find for the defense for + 目标，意图或利益 by Madry et al. make little to no sense对人类没有意义 to humans.
Taken together总结起来, even MNIST cannot be considered solved with respect to关于 adversarial robustness. By “solved”？？ we mean a model that reaches at least 99% accuracy (see accuracy-vs-robustness trade-off
(Tsipras et al., 2018; Bubeck et al., 2018)) and whose adversarial examples carry具有 semantic meaning
to对于人类来说 humans (by which we mean that 我们的意思是 they start looking like samples that could belong to either class).Hence因此, despite the fact that MNIST is considered “too easy” by many and a mere toy example又是, finding adversarially robust models on MNIST is still an open problem仍未解决的问题.
A potential solution we explore比较有用的动词，提出 in this paper is inspired by受什么的启发 unrecognizable images (Nguyen et al.,2015) or distal adversarials. Distal adversarials are images that解释一个名词的时候这么说 do not resemble images from the training set不像来自训练集的图片 but which ..A.. is ..B.. that ... but which 解释A的，A是什么，又不是什么typically look like noise while虽然 still being 与主句的主语是同一个主语，因此用现在分词 still being classified by the model通过模型 with high confidence给出一个. It seems difficult to prevent such images in feedforward networks在什么网络中 as因为 we have little control over how inputs are classified that are far outside of the training domain that指代的是前面的control training domain 训练领域的事. In contrast, generative models can learn the distribution of their inputs 不用the是因为前面确实没有提inputs的事 and are 两个动词使用and连接 thus able to gauge their confidence accordingly 策略置信度，accordingly修饰 gauge表示因此可以测量， are thus able to 因此可以. By additionally learning the image distribution within each class 各个class中的分布 前面的做条件状语 we can check检查 that the classification makes sense含义是to be reasonable in terms of 就什么而言 the image features being present做后置定语 be present是出现的意思，所以说不一定要用一个动词的动名词形式来做后置定语 in the input (e.g.例如 an image of a bus should contain actual bus features). Following this line of thought 顺着这一思路 from an information-theoretic perspective从信息论的角度，perspective观点, one arrives at the well-known concept of Bayesian classifiers这句话太有意思了，引出自己观点使用：Following this line of thought one arrives at，意思是自己使用了贝叶斯分类器，但是过度特别自然. We here introduce a fine-tuned variant 变体，VAE的变体 based on variational autoencoders (Kingma & Welling, 2013) that combines robustness with high accuracy.
1.如果该动词和主句主语之间为主动关系,则一般用其现在分词形式即Ving.
如：While watching TV,I heard a knock at my door.
= While (I was) watching TV,I heard a knock at my door.
2.但是,如果该动词和主句主语之间为被动关系时,则用其过去分词形式.
如：While beaten by his father,the boy cried.
make sense
Definition of make sense
1 : to have a clear meaning : to be easy to understand.
2 : to be reasonable
In summary, the contributions of this paper are as follows:
We show that MNIST is unsolved from the point of 在对抗鲁棒性方面adversarial robustness: the SOTA defense of Madry et al. (2018) is still highly vulnerable to tiny perturbations that are meaningless to humans.
- We
introduce提出a new robust classification model andderive从...中获取，推导出instance-specific robustness guarantees. - We
developa strong attack that leverages the generative structure of our classification model. - We
introduce提出a novel decision-based attack that minimizes L0. - We
perform进行an extensive evaluation of our defenseacross有表示很多的意思，从这个到那个many attacks to show that it surpasses SOTA on L0, L2 and L1 and features many adversarials that carry semantic meaning to humans.
We have evaluated过去完成时态说明再过去对之前提出的defense都进行了评估 the proposed defense to the best of our knowledge, but we are aware of the 后面可以连接一个句子(currently unavoidable) limitations of evaluating robustness. We will release the model architecture and trained weights as a friendly invitation 友好的邀请 to fellow researchers to evaluate our model independently
RELATED WORK
The many defenses against adversarial attacks can roughly be subdivided into four categories四种类型:
- Adversarial training: The training data is augmented
with adversarial examples with 给我的感觉是直接拿来用，by的话还要进行操作to make models more robust (Madry et al., 2018; Szegedy et al., 2013; Tramèr et al., 2017; Ilyas et al., 2017). - Manifold projections:
An input sample 每个每个，不用the 是不能当成整体同一投射在一个流行上面is projected onto a learned data manifold (Samangouei et al., 2018; Ilyas et al., 2017; Shen et al., 2017; Song et al., 2018). - Stochasticity:
Certain inputs or hidden activations复数名词如果不是要加the的情况下可以不加冠词are shuffled or randomized (Prakash et al.,2018; Dhillon et al., 2018; Xie et al., 2018). - Preprocessing: Inputs or hidden activations
are quantized, projectedinto a different representationor areotherwise preprocessed (Buckman et al., 2018; Guo et al., 2018; Kabilan et al., 2018).
There has been much work showing that 引用前人的观点basically alldefenses suggested后置定语so far in the literature do not substantially increase robustnessover在什么之上undefended neural networks.(Athalye et al., 2018; Brendel &Bethge, 2017).The only widely accepted exception according to Athalye et al. (2018) is the defense by Madry et al. (2018) which is based ondata augmentation with adversarialsfound by iterative projected gradient descent with random starting points. However, as we see in the results section, this defenseis limited to限制在the metricit is trained on定语从句在从句中做宾语的时候可以省略that(L1) and it is straight-forward to generate small adversarial perturbations that carry little semantic meaning for humans.
Some other defenses have been based ongenerative models. Typically these defenses use the generative model to project onto the (learned) manifold of “natural” inputs.This includes in particular DefenseGAN最典型尤其 (Samangouei et al., 2018), Adversarial Perturbation Elimination GAN (Shen et al., 2017) and Robust Manifold Defense (Ilyas et al., 2017), all of which project an image onto the manifold defined by a generator network G. The generated image is then classified by a discriminator in the usual way. A similar idea is used by PixelDefend (Song et al., 2018) which uses an autoregressive probabilistic method to learn the data manifold. Other ideas in similar directions其他相同的idea有 include the use of denoising autoencoders (Liao et al., 2017) as well as MagNets (Meng & Chen, 2017), which projects or rejects inputs depending on their distance to the data manifold. All of these proposed defenses except for the defense by Ilyas et al. (2017) have been tested by Athalye et al. (2018); Athalye & Carlini (2018); Carlini &Wagner (2017) and others, and shown to be ineffective. It is straight-forward to understand why: For one, many adversarials still look like normal data points to humans. Second, the classifier on top of the projected image is as vulnerable to adversarial examples as before像之前一样脆弱. Hence,forany data set with a natural amount of variation there will almost always be for...there be 对于什么总存在什么... a certain perturbation against which与前面的词构成状语 the classifier is vulnerable and which 先行词是前面的 perturbationcan be induced by the right inputs.
介词 + which 的用法，是定语从句中的，在从句中主要起关系副词的作用，在定语从句中作状语。
（1）表示地点，时间和原因的“介词+which”分别相当于 where,when,why。
（2）way后常用that 代替in which，可以省略that。
（3）of+ which相当于whose
（4）既可以引导限定性状语从句也可以引导非限定性定语从句
there is a rocket by which the direction of the satellite can be changed.
we carefully studied the photos, in which we could see signs of plant disease.
（5）介词的搭配要考虑（1）介词与先行词的搭配
there is no way in which
（2）介词与定语从句中的动词，形容词的习惯搭配
there are the wires with which different machines are connected
参考资料
We here follow a different approach by怎么做的方法 modeling the input distribution within each class 介词within的使用，含义是每个class的分布 (instead of modeling a single distribution for the complete data), and by classifying a new sample according to the class under which it has the highest likelihood. This approach, commonly referred to as被称为 a Bayesian classifier, gets away without any additional and vulnerable classifier. A very different but related approach is the work by George et al. (2017) which 代指work``suggested表明 a generative compositional model of digits to solve cluttered digit scenes like Captchas (adversarial robustness was not evaluated).
Model Description
Intuitively, we want to learn a causal model of the inputs输入因果模型 (Schölkopf, 2017). Consider a cat举例的时候用: we want a model to learn that cats have four legs and two pointed ears, and then use this model to check whether a given input can be generated with these features. with是直接拿来用，by是操作使用 This intuition can be formalized as follows. Let (x; y) with x 2 RN be an input-label datum. Instead of directly learning a posterior p(yjx) from inputs to labels we now learn generative distributions p(xjy) and classify new inputs using Bayes formula.The label distribution p(y) can be estimated from the training data. To learn the class-conditional sample distributions p(xjy) we use variational autoencoders (VAEs) (Kingma & Welling, 2013). VAEs estimate the log-likelihood log p(x) by learning a probabilistic generative model p�(xjz) by当用来讲的话有操作的含义不是直接拿来用的with latent variables z � p(z) and parameters � (see Appendix A.3 for the full derivation for+名词表示目的). For class-conditional VAEs 表示目的 we can derive a lower bound on一个公式上的界，不用of的原因是of的话有一个本身的归属的含义，bound的话是不属于这个方程的，所以是方程上的界 the log-likelihood log p(xjy) where p(z) = N(0; 1) is a simple normal prior and q�(zjx; y) is the variational posterior with parameters �. The first term第一项 on关于公式的介词用on the RHS is basically a reconstruction error while而，表示轻微的不同 the second term on the RHS is the mismatch between the variational and the true posterior. The term on the RHS is the so-called evidence lower bound (ELBO) on the log-likelihood (Kingma & Welling, 2013). We implement the conditional distributions p�(xjz; y) and q�(zjx; y) as implement as 讲什么实施成 normal distributions for which for这里是表示对象/目的 介词+ which在句中做关系副词，不用which是因为which要在居中担当成分 the means are parametrized as DNNs (all details and hyperparameters are reported in Appendix
Our Analysis by Synthesis model (ABS)必须要对模型进行操作 所以是by is illustrated in Figure 1. It combines several elements to todo是目的状语simultaneously achieve high accuracy and robustness against adversarial perturbations:
- Class-conditional distributions:
For each class y 为每一个class for 有一种含义就是 从句要为for后面的目标做点事的含义，而on就没有we train a variational autoencoder VAEyon 只表示相关the samples of class y to learn the class-conditional distribution p(xjy). This allows us to estimate a lower bound y(x)on有个含义是在什么之上the log-likelihood of samplex under each class y. 在每个类别标签y只上的x - Optimization-based inference: The variational inference q�(zjx; y)
is itself 本身就是a neural networksusceptible to adversarial perturbations 形容词作后置定语. We therefore only use variational inference during training and perform “exact” inferenceover p�(xjz; y)during evaluation. This “exact” inference is implemented using gradient descent in the latent space (with fixed posterior width) to find the optimal zy which maximizes the lower bound on the log-likelihood for each class:
on 和 for的辨析
on 只是相关，for是为...特殊提供，强调动作发生
这是朝着这个目标的
When you have some "ideas on how to improve
my team," you have ideas relating to ideas on
improving the team.
When you have "ideas for improving my team,"
you have ideas which specifically supports the
team. For example, when you say "I am for
peace-making" you are obviously supporting
peace-making. In the same way, using "for" in
ideas on improving the team means you support
improving the team while using "on" doesn't
necessarily mean so. It's all connotation and
subconscious language use and effects.
Note that we replaced the expectation in 公式中的期望 equation 2 with a maximum likelihood 用一个，是不加操作的所以用with不用bysample to avoid stochastic sampling and to simplify optimization. To avoid local minima we evaluate 8000 random points in the latent space of each VAE, from which从这8000个点中 we pick the best as a starting point for a gradient descent with 50 iterations with 通常后面跟一个设置 using the Adam optimizer (Kingma & Ba, 2014).
- Classification and confidence: Finally, to perform the actual classification, we
scale all �y (x) with a factor�, exponentiate, add an offset � and divide by the total evidence (like in a softmax). We introduced �for the following reason:even on on的话只是关于的意思pointsfar outside the data domain远离, where all likelihoods q(x; y) = e�� y(x) + � are small, the standard softmax (� = 0) can lead to sharp posteriors p(yjx) with high confidence scores for one class. This behavior isin stark赤裸裸的醒目的contrast to humans, who would report a uniform distributionover classes（部份或全部覆盖）在…上面 表示对全部的class使用相同的分布for unrecognizable images for的话report的动作是为image做的. To model this behavior we set � > 0: in this case the posterior p(yjx) converges to a uniform distribution whenever the maximum q(x; y) gets small relative to � . We chose � such that the median confidence p(yjx) is 0.9 for the predicted class on clean test samples. Furthermore, for a better comparison with cross-entropy trained networks, the scale � is trained to minimize the cross-entropy loss. We also tested this graded softmax in standard feedforward CNNs but did not find any improvementwith respect to 关于unrecognizable images. - Binarization (Binary ABS only): The pixel
intensities复数of MNIST images are almost binary. Weexploit利用this by projecting the intensity b of each pixel to 0 if b < 0:5 or 1 if b � 0:5 during testing. - Discriminative finetuning (Binary ABS only): To improve
the accuracy of the Binary ABS model 模型的准确率，accuracy on classwe multiply �y (x) with an additional class-dependent scalar y. The scalars are learned discriminatively (see A.7) and reach values in the range y 2 [0:96; 1:06] for all classes y.
One importantingredient 组件for the robustness of the ABS model is the Gaussian posterior in the reconstruction term which ensures that small changes in the input (in terms of L2) can onlyentail招致small changes to the posterior likelihood and thus to the model decision.
TIGHT ESTIMATES OF THE LOWER BOUND FOR ADVERSARIAL EXAMPLES
The decision of the model depends on the likelihood in each class 不是of是因为不是每个class的likelihood，而是当前样本在每个class上的likelihood，不用on是因为 没有likelihood on这个搭配, which for clean samples is mostly dominated by被什么所控制 the posterior likelihood p(xjz). Because we chose this posterior to be Gaussian, the class-conditionallikelihoods can only change gracefully with change with 随着什么而改变 changes in x 因为是x内部的一个小变化，changes of x就是x的变体了 ，a property which allows us to derive lower bounds on bounds on好像是固定搭配，之前见到的都是bound on the model robustness. To see this, note that equation 3 can be written as.$$.where we absorbed the normalization constants of p(xjz) into将什么吸收入 C and Gc(z) is the mean of p(xjz; c). Let y be the ground-truth class and let z� x be the optimal latent for the clean sample x for class y. We can then estimate a lower bound on �y (x + �) for a perturbation � with size � = k�k2 (see derivation in Appendix A.4).
Likewise, we can derive an upper bound of `�y (x + �) for all other classes c 6= y (see Appendix
Note that one assumption we make is that we can find the global minimum of kGc(z) xk2 2. In practice在实践中 we generally find a very tight estimate of the global minimum (and thus the lower bound) because we optimize in a smooth and low-dimensional space and because we perform an additional brute-force 强力攻击 sampling step. We provide quantitative values for � in section
感觉第四部分写得和前面三个部分是有差距的，用介词和表达都很混乱，可能是换了一个人来写的
5 ADVERSARIAL ATTACKS
Reliably evaluating model robustness is difficult because each attack only provides an upper bound on the size of the adversarial perturbations (Uesato et al., 2018). To make this bound as tight as possible we apply many different attacks and choose the best one for each sample and model combination (using the implementations in Foolbox v1.3 (Rauber et al., 2017) which often perform internal hyperparameter optimization). We also created a novel decision-based L0 attack as well as a
customized attack that specifically exploits the structure of our model. Nevertheless, we cannot rule out that more effective attacks exist and we will release the trained model for future testing.