神经网络与深度学习（第一章）（一）

Using neural nets to recognize handwritten digits 使用神经网络识别手写数字

The human visual system is one of the wonders of the world. Consider the following sequence of handwritten digits:
人类的视觉系统是世界上的一个奇迹。看看下面的手写数字序列：

Most people effortlessly recognize those digits as 504192. That ease is deceptive. In each hemisphere of our brain, humans have a primary visual cortex, also known as V1, containing 140 million neurons, with tens of billions of connections between them. And yet human vision involves not just V1, but an entire series of visual cortices - V2, V3, V4, and V5 - doing progressively more complex image processing. We carry in our heads a supercomputer, tuned by evolution over hundreds of millions of years, and superbly adapted to understand the visual world. Recognizing handwritten digits isn’t easy. Rather, we humans are stupendously, astoundingly good at making sense of what our eyes show us. But nearly all that work is done unconsciously. And so we don’t usually appreciate how tough a problem our visual systems solve.
大多数人毫不费力的识别出这些数字是504192。这种容易是虚假的。在我们大脑的每个半球，人类拥有一个主要大脑皮层V1，它包含1.4亿个神经元和数百亿的神经纤维连接它们。然而人类的视觉系统不仅仅包括V1，还包括一系列完整的视觉皮层——V2、V3、V4和V5——从事更加复杂的图像处理。我们把我们的大脑看作一台超级电脑，它被数亿年的进化所优化，极度适应于理解视觉世界。识别手写数字并不容易。更准确的说，我们人类惊人的擅长理解我们眼睛告诉我们的事情。不过这些事情是在不知不觉中完成的。并且我们不能经常意识到我们视觉系统解决的问题多么的难。

The difficulty of visual pattern recognition becomes apparent if you attempt to write a computer program to recognize digits like those above. What seems easy when we do it ourselves suddenly becomes extremely difficult. Simple intuitions about how we recognize shapes - “a 9 has a loop at the top, and a vertical stroke in the bottom right” - turn out to be not so simple to express algorithmically. When you try to make such rules precise, you quickly get lost in a morass of exceptions and caveats and special cases. It seems hopeless.
如果你尝试写一个计算机程序来识别上面那样的数字，你就会意识到视觉模式识别有多难。我们自己做起来似乎十分简单的事情变得非常难。一个简单的直觉是我们可以识别边沿——“数字9在顶部有一个环，在环的右下方有一竖”——结果是算法表达上并没有这么简单。当你尝试定义这样精确的条件时，你很快就迷失在例外、告警和特殊事件的陷阱中。这似乎毫无希望。

Neural networks approach the problem in a different way. The idea is to take a large number of handwritten digits, known as training examples

and then develop a system which can learn from those training examples. In other words, the neural network uses the examples to automatically infer rules for recognizing handwritten digits. Furthermore, by increasing the number of training examples, the network can learn more about handwriting, and so improve its accuracy. So while I’ve shown just 100 training digits above, perhaps we could build a better handwriting recognizer by using thousands or even millions or billions of training examples.
神经网络用其他方法来解决这个问题。这个方法是取得大量的手写数字，也就是训练样本，并且开发一个系统，它可以从这些训练样本中学习识别手写数字。换句话说，神经网络利用样本数据自动推导出识别手写数字的规则。更进一步，随着训练样本的增多，神经网络可以从手写体中学到更多从而提高它的准确性。所以当我们使用数千、数万或者数亿训练样本而不仅仅是上面100个训练数字时，我们可以建立一个更好的手写数字识别器。

In this chapter we’ll write a computer program implementing a neural network that learns to recognize handwritten digits. The program is just 74 lines long, and uses no special neural network libraries. But this short program can recognize digits with an accuracy over 96 percent, without human intervention. Furthermore, in later chapters we’ll develop ideas which can improve accuracy to over 99 percent. In fact, the best commercial neural networks are now so good that they are used by banks to process cheques, and by post offices to recognize addresses.
本章中我们将编写一个计算机程序实现一个神经网络来识别手写数字。这个程序只有仅仅74行，并且没有用到特殊的神经网络库。不过在没有人工干预的情况下这个小程序的识别准确率超过96%。更进一步，在最后一章我们将进一步将准确率提升到超过99%。事实上，最好的商业神经网络已经好到被银行用来处理支票，或是邮局识别地址。

We’re focusing on handwriting recognition because it’s an excellent prototype problem for learning about neural networks in general. As a prototype it hits a sweet spot: it’s challenging - it’s no small feat to recognize handwritten digits - but it’s not so difficult as to require an extremely complicated solution, or tremendous computational power. Furthermore, it’s a great way to develop more advanced techniques, such as deep learning. And so throughout the book we’ll return repeatedly to the problem of handwriting recognition. Later in the book, we’ll discuss how these ideas may be applied to other problems in computer vision, and also in speech, natural language processing, and other domains.
我们聚焦于笔迹识别是因为通常来说对于学习神经网络它是一个非常好的范例。作为一个范例，它击中了一个最有效的点：具有挑战性——识别手写数字并不是简单的事——它也不会难到需要特别复杂难懂的解决方案，或者惊人的计算资源。此外，它是发展更先进技术的绝佳路径，例如深度学习。在本书中我们将不断的回顾这个手写数字识别的问题。在后面的章节，我们将讨论如何将这些方法应用在其他问题上，例如计算机视觉、语音识别、自然语言处理和其他领域。

Of course, if the point of the chapter was only to write a computer program to recognize handwritten digits, then the chapter would be much shorter! But along the way we’ll develop many key ideas about neural networks, including two important types of artificial neuron (the perceptron and the sigmoid neuron), and the standard learning algorithm for neural networks, known as stochastic gradient descent. Throughout, I focus on explaining why things are done the way they are, and on building your neural networks intuition. That requires a lengthier discussion than if I just presented the basic mechanics of what’s going on, but it’s worth it for the deeper understanding you’ll attain. Amongst the payoffs, by the end of the chapter we’ll be in position to understand what deep learning is, and why it matters.
当然，如果本章只是编写一个识别手写数字的程序，那么本章将短小很多！不过沿着这个思路我们将延伸出许多关于神经网络的关键概念，包括两种重要的人工神经元（perceptron和sigmoid神经元），神经网络的标准学习算法，随机梯度下降。由始至终，我聚焦于为什么这么做和如何建立你自己的神经网络。更深入的理解你想实现的内容是非常值得的，但是需要比我仅仅呈现的基本过程更漫长的讨论。本章的最后我们将了解深度学习是什么和为什么它可以工作。