Deep learning collection

Deep learning collection. 

• Websites -  

  http://deeplearning.net

  http://www.kdnuggets.com

• Python implementations/examples

  • Introduction to Restricted Boltzmann Machines -- Great, easy to follow explanation + python implementation of RBMs
  • The CUV Library is a C++ framework with python bindings for easy use of Nvidia CUDA functions on matrices. It contains an RBM implementation as well as annealed importance sampling code and code to calculate the partition function exactly.
  • Modular toolkit for Data Processing collection of supervised and unsupervised learning algos that can be combined into complex feed-forward network architectures -- backpropagation + DBN examples at the bottom
  • Pietro Berkes -- (tgz) using DBNs for digit recognition
  • Deep Learning Tutorials -- examples of how to do Deep Learning with Theano (from LISA lab at University of Montreal)
  • Theano -- CPU/GPU symbolic expression compiler in python (from LISA lab at University of Montreal)
  • A Python implementation of the Replicated Softmax Topic Model An implementation of R. Salakhutdinov and G.E. Hinton's Replicated Softmax Topic Model 

• Conferences/Workshops

  • Nips 2011 workshop Challenges in Learning Hierarchical Models
• 
  

  Geoff Hinton (back-propagation, coinvented boltzmann machines)

  • Hinton, G. E. and Salakhutdinov, R. R. (2006) Reducing the dimensionality of data with neural networks. Science, Vol. 313. no. 5786, pp. 504 - 507, 28 July 2006. [ full paper ] [ supporting online material (pdf) ] [ Matlab code ]
  • YouTube (2007) The Next Generation of Neural Networks (1hr)
  • YouTube (2010) Recent Developments in Deep Learning (1hr)
• 
  

  Andrew Ng (bringing neural nets to the masses)

  • This tutorial will teach you the main ideas of Unsupervised Feature Learning and Deep Learning. By working through it, you will also get to implement several feature learning/deep learning algorithms, get to see them work for yourself, and learn how to apply/adapt these ideas to new problems.
  • Current class videos focused on deep learning
• 
  

  Yann LeCun (convolutional neural networks)

  • The Computational and Biological Learning Lab: Our research is focused on Machine Learning, Computer Vision, Robotics, Computational Neuroscience, and various related topics.
  • Convolutional Neural Networks Convolutional Neural Networks are a special kind of multi-layer neural networks. Like almost every other neural networks they are trained with a version of the back-propagation algorithm. Where they differ is in the architecture.
• 
  

  Marc'Aurelio Ranzato (Student of Hinton + LeCun -- now working for google)

  • Ph.D. Thesis Unsupervised Learning of Feature Hierarchies New York University, May 2009.
• 
  

  Ruslan Salakhutdinov

  • Replicated Softmax: an Undirected Topic Model The learned topics are more general than those found by LDA because precision is achieved by intersecting many general topics rather than by selecting a single precise topic to generate each word.
  • Deep Belief Networks. Matlab code for learning Deep Belief Networks.
  • Estimating Partition Functions of RBM's. Matlab code for estimating partition functions of Restricted Boltzmann Machines using Annealed Importance Sampling.
  • Learning Deep Boltzmann Machines Matlab code for training and fine-tuning Deep Boltzmann Machines.