机器学习中的神经网络Neural Networks for Machine Learning：Lecture 11 Quiz

Lecture 11 QuizHelp Center

Warning: The hard deadline has passed. You can attempt it, but you will not get credit for it. You are welcome to try it as a learning exercise.

In accordance with the Coursera Honor Code, I  certify that the answers here are my own work.

Question 1

For a stochastic binary unit in a Boltzmann Machine, the probability of the unit turning on, i.e.  P(s=1) , is defined as  P(s=1)=11+exp(−ΔE/T) . Here,  ΔE  is the  energy gap, i.e. the energy when the unit is off, minus the energy when the unit is on.  T  is the  temperature.

If  ΔE=−3 , then:

P(s=1)  increases when  T  increases.

P(s=1)  decreases when  T  increases.

Question 2

The Hopfield network shown below has two visible units:  V1  and  V2 . It has a connection between the two units, and each unit has a bias. 



Let  W12=−10 ,  b1=1 , and  b2=1  and the initial states of  V1=0  and  V2=0 . 

If the network always updates both units simultaneously, then what is the lowest energy value that it will encounter (given those initial states)?

If the network always updates the units one at a time, i.e. it alternates between updating  V1  and updating  V2 , then what is the lowest energy value that it will encounter (given those initial states)?

Write those two numbers with a comma between them. For example, if you think that the answer to that first question is 4, and that the answer to the second question is -7, then write this:  4, -7

Answer for Question 2

Question 3

This question is about Boltzmann Machines, a.k.a. a stochastic Hopfield networks. Recall from the lecture that when we pick a new state  si  for unit  i , we do so in a stochastic way:  p(si=1)=11+exp(−ΔE/T) , and  p(si=0)=1−p(si=1) . Here,  ΔE  is the energy gap, i.e. the energy when the unit is off, minus the energy when the unit is on.  T  is the  temperature. We can run our system with any temperature that we like, but the most commonly used temperatures are  0  and  1 .

When we want to explore the configurations of a Boltzmann Machine, we initialize it in some starting configuration, and then repeatedly choose a unit at random, and pick a new state for it, using the probability formula described above.

Consider two small Boltzmann Machines with 10 units, with the same weights, but with different temperatures. One, the "cold" one, has temperature 0. The other, the "warm" one, has temperature 1. We run both of them for 1000 iterations (updates), as described above, and then we look at the configuration that we end up with after those 1000 updates.

Which of the following statements are true? (Note that an "energy minimum" is what could also reasonably be called a "local energy minimum")

If the cold one is exponentially unfortunate, it could end up in a configuration that's not an energy minimum.

For the cold one,  P(si=1)  can be any value between 0 and 1, depending on the weights.

If the weights are small, then over the course of those 1000 updates, the warm one is likely to have encountered more different configurations than the cold one.

The cold one is more likely to end in an energy minimum than the warm one.

Question 4

The Boltzmann Machine shown below has two visible units  V1  and  V2 . There is a connection between the two, and both units have a bias.


Let  W12=−2 ,  b1=1 , and  b2=1 .
What is  P(V1=1,V2=0) ? Write your answer with at least 3 digits after the decimal point.

Answer for Question 4

Question 5

The figure below shows a Hopfield network with five binary threshold units:  a ,  b ,  c ,  d , and  e . The network has many connections, but no biases.

Let  Wac=Wbc=1 ,  Wce=Wcd=2 ,  Wbe=−3 ,  Wad=−2 , and  Wde=3 . 

What is the energy of the configuration with the lowest energy? What is the energy of the configuration with the second lowest energy (considering all configurations, not just those that are energy minima)? 

Write your answer with a comma between the two numbers. For example, if you think that the energy of the lowest energy configuration is -17, and that the energy of the second lowest energy configuration is -13, then you should write this: -17, -13