Feedback Nerual Network(六）：Boltzmann Machine
It is a Random NN, is also a kind of Feedback NN
It is similar with DHNN in many aspects, big difference is that Boltzmann NN is a Random NN
When the temperature parameter T→ 0
⇒S-Type Function approaches to Binary Function
⇒Random NN degenerates to Deterministic NN(Same as DHNN)
It can be used in Pattern Classification, Prediction, Combination Optimization, Planning etc.
Structure of Networks & Work Mode
Structure:
Single layer Feedback Networks
Same as Hopfield, has Symmetric Link-Weight Matrix, i.g
wij= wji
, and wii= 0
Function:
Can be viewed as a Multi-Layer Networks
Part of nodes are input nodes, part of nodes are output nodes and part of nodes are hide nodes
The hide nodes have no relationship with outside, only implement the high-order relationship between I/O
Work Mode:
Synchronous Mode
Asynchronous Mode
Discuss the Convergence (2)
From probability point of view, the tendency of system energy is always towards to the minimum
Some states of neurons possible take the values oppositely, consequently will increase the Energy.
Sometimes this is good for jumping out of local minimum point. This is another difference compare with Hopfield NN.
Discuss the Convergence (3)
Mimic the Annealing Method 模拟退火
At beginning, use higher temperature T
Curve of S-Type Function is flatter, the difference of probability of each state is small
Can accurately move to a minimum and also prevent jumping out of this minimum
When near neighboring global minimum point, step by step reduce T
Curve of S-Type Function is more cliff the difference of probability of each state is bigger
Easy to jump out of local minimum point and go to global minimum point
Summary:
Energy is lower
the probability of being a certain state is bigger
T is biggerS-Type Function is flatter
the difference of probability of occurring different state is small
Accurately move to a minimum and prevent jump out of this minimum point
T is smallerS-Type Function is more cliffy
the difference of probability of occurring different state is big
Easy jump out of Local Minimum Point and reach the Global Minimum