Title
Deep Boltzmann Machines
Abstract
We present a new learning algorithm for Boltz- mann machines that contain many layers of hid- den variables. Data-dependent expectations are estimated using a variational approximation that tends to focus on a single mode, and data- independent expectations are approximated us- ing persistent Markov chains. The use of two quite different techniques for estimating the two types of expectation that enter into the gradient of the log-likelihood makes it practical to learn Boltzmann machines with multiple hidden lay- ers and millions of parameters. The learning can be made more efficient by using a layer-by-layer "pre-training" phase that allows variational in- ference to be initialized with a single bottom- up pass. We present results on the MNIST and NORB datasets showing that deep Boltzmann machines learn good generative models and per- form well on handwritten digit and visual object recognition tasks.
Year
Venue
Keywords
2009
AISTATS
layer by layer,visual object recognition,single mode,markov chain,bottom up,boltzmann machine
Field
DocType
Volume
Boltzmann machine,MNIST database,Computer science,Theoretical computer science,Artificial intelligence,Hidden variable theory,Restricted Boltzmann machine,Mathematical optimization,Inference,Markov chain,Boltzmann constant,Machine learning,Cognitive neuroscience of visual object recognition
Journal
5
Citations 
PageRank 
References 
172
16.88
12
Authors
2
Search Limit
100172
Name
Order
Citations
PageRank
Ruslan Salakhutdinov112190764.15
geoffrey e hinton2404354751.69