（點(diǎn)擊尾部閱讀原文前往）

深度學(xué)習(xí)作為機(jī)器學(xué)習(xí)的一個(gè)分支，是近年來最熱門同時(shí)也是發(fā)展最快的人工智能技術(shù)之一，相關(guān)學(xué)習(xí)資源包括免費(fèi)公開教程和工具都極大豐富，同時(shí)這也為學(xué)習(xí)深度學(xué)習(xí)技術(shù)的IT人才帶來選擇上的困擾，Yerevann整理的這個(gè)深度學(xué)習(xí)完全指南，匯集了目前網(wǎng)絡(luò)上最優(yōu)秀的深度學(xué)習(xí)自學(xué)資源，而且會(huì)不定期更新，非常值得收藏關(guān)注，以下是IT經(jīng)理網(wǎng)編譯整理的指南內(nèi)容：

自學(xué)基本要求（數(shù)學(xué)知識(shí)、編程知識(shí)）

數(shù)學(xué)知識(shí)：學(xué)員需要具備普通大學(xué)數(shù)學(xué)知識(shí)，例如《Deep Learning》一書中若干章節(jié)提到的數(shù)學(xué)概念：

Deep Learning第二章：線性代數(shù)

Deep Learning第三章：概率與信息理論

Deep Learning第四章：數(shù)值計(jì)算

編程知識(shí)：你需要懂得編程才能開發(fā)和測試深度學(xué)習(xí)模型，我們建議在機(jī)器學(xué)習(xí)領(lǐng)域首選Python。同時(shí)也要用到面向科學(xué)計(jì)算的NumPy/SciPy代碼庫。資源鏈接如下（本文出現(xiàn)的星標(biāo)代表難度等級(jí)）：

Justin Johnson’s Python / NumPy / SciPy / Matplotlib tutorial for Stanford’s CS231n

Scipy lecture notes – 涵蓋了常用的各種庫，介紹也比較詳細(xì)，還涉及一些深入的技術(shù)話題

四大入門教程

如果你具備以上自學(xué)基本要求技能，我們建議從以下四大入門在線教程中任選一項(xiàng)或多項(xiàng)組合學(xué)習(xí)（星標(biāo)為難度等級(jí)）：

Hugo Larochelle’s video course 這是YouTube上很火的一個(gè)深度學(xué)習(xí)視頻教程，錄制于2013年，但今天看內(nèi)容并不過時(shí)，很詳細(xì)地闡釋了神經(jīng)網(wǎng)絡(luò)背后的數(shù)學(xué)理論。 幻燈片和相關(guān)資料傳送門 .

Stanford’s CS231n (應(yīng)用于視覺識(shí)別的卷積神經(jīng)網(wǎng)絡(luò)) 由已經(jīng)投奔Google的李飛飛教授和 Andrej Karpathy、Justin Johnson共同執(zhí)教的課程，重點(diǎn)介紹了圖像處理，同時(shí)也涵蓋了深度學(xué)習(xí)領(lǐng)域的大多數(shù)重要概念。 視頻 鏈接(2016) 、 講義傳送門?

Michael Nielsen的在線著作： Neural networks and deep learning 是目前學(xué)習(xí)神經(jīng)網(wǎng)絡(luò)最容易的教材，雖然該書并未涵蓋所有重要議題，但是包含大量簡明易懂的闡釋，同時(shí)還為一些基礎(chǔ)概念提供了實(shí)現(xiàn)代碼。

Ian Goodfellow、Yoshua Bengio and Aaron Courville共同編著的 Deep learning是目前深度學(xué)習(xí)領(lǐng)域最全面的教程資源，比其他課程涵蓋的范圍都要廣。

機(jī)器學(xué)習(xí)基礎(chǔ)

機(jī)器學(xué)習(xí)是通過數(shù)據(jù)教計(jì)算機(jī)做事的科學(xué)，同時(shí)也是一門藝術(shù)，機(jī)器學(xué)習(xí)是計(jì)算機(jī)科學(xué)和數(shù)學(xué)交匯的一個(gè)相對(duì)成熟的領(lǐng)域，深度學(xué)習(xí)只是其中新興的一小部分，因此，了解機(jī)器學(xué)習(xí)的概念和工具對(duì)我們學(xué)好深度學(xué)習(xí)非常重要。以下是機(jī)器學(xué)習(xí)的一些重要學(xué)習(xí)資源（以下課程介紹部分內(nèi)容不再翻譯）：

Visual introduction to machine learning – decision trees

Andrew Ng’s course on machine learning, the most popular course on Coursera

Larochelle’s course doesn’t have separate introductory lectures for general machine learning, but all required concepts are defined and explained whenever needed.

1、Training and testing the models (kNN)

2、Linear classification (SVM)

3、Optimization (stochastic gradient descent)

4、Machine learning basics

5、Principal Component Analysis explained visually

6、How to Use t-SNE Effectively

機(jī)器學(xué)習(xí)的編程學(xué)習(xí)資料：大多數(shù)流行機(jī)器學(xué)習(xí)算法都部署在Scikit-learn 這個(gè)Python庫中，從頭部署算法能夠幫我們更好地了解機(jī)器學(xué)習(xí)的工作原理，以下是相關(guān)學(xué)習(xí)資源：

1、Practical Machine Learning Tutorial with Python covers linear regression, k-nearest-neighbors and support vector machines. First it shows how to use them from scikit-learn, then implements the algorithms from scratch.

2、Andrew Ng’s course on Coursera has many assignments in Octave language. The same algorithms can be implemented in Python.

神經(jīng)網(wǎng)絡(luò)基礎(chǔ)

神經(jīng)網(wǎng)絡(luò)是強(qiáng)大的機(jī)器學(xué)習(xí)算法，同時(shí)也是深度學(xué)習(xí)的基礎(chǔ)：

A Visual and Interactive Guide to the Basics of Neural Networks – shows how simple neural networks can do linear regression

1、Feedforward neural network

2、Training neural networks (up to 2.7)

3、Backpropagation

4、Architecture of neural networks

5、Using neural nets to recognize handwritten digits

6、How the backpropagation algorithm works

7、A visual proof that neural nets can compute any function

8、Deep feedforward networks

Yes you should understand backprop explains why it is important to implement backpropagation once from scratch

Calculus on computational graphs: backpropagation

Play with neural networks!

神經(jīng)網(wǎng)絡(luò)實(shí)操教程

1、Implementing softmax classifier and a simple neural network in pure Python/NumPy–Jupyter notebook available

2、Andrej Karpathy implements backpropagation in Javascript in his Hacker’s guide to Neural Networks.

3、Implementing a neural network from scratch in Python

改進(jìn)神經(jīng)網(wǎng)絡(luò)學(xué)習(xí)

神經(jīng)網(wǎng)絡(luò)的訓(xùn)練可不容易，很多時(shí)候機(jī)器壓根不會(huì)學(xué)習(xí)（underfitting），有時(shí)候又“死學(xué)”，照本宣科你輸入的知識(shí)，無法總結(jié)歸納出新的數(shù)據(jù)（overfitting）,解決上述問題的方法有很多，如下是

推薦教程：

2.8-2.11. Regularization, parameter initialization etc.

7.5. Dropout

6 (first half). Setting up the data and loss

3. Improving the way neural networks learn

5. Why are deep neural networks hard to train?

7. Regularization for deep learning

8. Optimization for training deep models

11. Practical methodology

ConvNetJS Trainer demo on MNIST – visualizes the performance of different optimization algorithms

An overview of gradient descent optimization algorithms

Neural Networks, Manifolds, and Topology

常用的主流框架

目前很多深度學(xué)習(xí)算法都對(duì)最新的計(jì)算機(jī)硬件進(jìn)行了優(yōu)化，大多數(shù)框架也提供Python接口（Torch除外，需要Lua）。當(dāng)你了解基本的深度學(xué)習(xí)算法的部署后，是時(shí)候選擇一個(gè)框架開工了（這部分還可CTOCIO文章：2016年人氣最高的六款開源深度學(xué)習(xí)工具）：

Theano provides low-level primitives for constructing all kinds of neural networks. It is maintained by a machine learning group at University of Montreal. See also: Speeding up your neural network with Theano and the GPU – Jupyter notebook available

TensorFlow is another low-level framework. Its architecture is similar to Theano. It is maintained by the Google Brain team.

Torch is a popular framework that uses Lua language. The main disadvantage is that Lua’s community is not as large as Python’s. Torch is mostly maintained by Facebook and Twitter.

There are also higher-level frameworks that run on top of these:

Lasagne is a higher level framework built on top of Theano. It provides simple functions to create large networks with few lines of code.

Keras is a higher level framework that works on top of either Theano or TensorFlow.

如果你有框架選擇困難癥，可以參考斯坦福課程Lecture 12 of Stanford’s CS231n.

卷積神經(jīng)網(wǎng)絡(luò)

卷積神經(jīng)網(wǎng)絡(luò)Convolutional networks (CNNs)，是一種特定的神經(jīng)網(wǎng)絡(luò)，通過一些聰明的方法大大提高了學(xué)習(xí)速度和質(zhì)量。卷積神經(jīng)網(wǎng)絡(luò)掀起了計(jì)算機(jī)視覺的革命，并廣泛應(yīng)用于語音識(shí)別和文本歸類等領(lǐng)域，以下是

推薦教程：

9. Computer vision (up to 9.9)

6 (second half). Intro to ConvNets

7. Convolutional neural networks

8. Localization and detection

9. Visualization, Deep dream, Neural style, Adversarial examples

13. Image segmentation (up to 38:00) includes upconvolutions

6. Deep learning

9. Convolutional networks

Image Kernels explained visually – shows how convolutional filters (also known as image kernels) transform the image

ConvNetJS MNIST demo – live visualization of a convolutional network right in the browser

Conv Nets: A Modular Perspective

Understanding Convolutions

Understanding Convolutional neural networks for NLP

卷積神經(jīng)網(wǎng)絡(luò)框架部署和應(yīng)用

所有重要的框架都支持卷積神經(jīng)網(wǎng)絡(luò)的部署，通常使用高級(jí)函數(shù)庫編寫的代碼的可讀性要更好一些。

Theano: Convolutional Neural Networks (LeNet)

Using Lasagne for training Deep Neural Networks

Detecting diabetic retinopathy in eye images – a blog post by one of the best performers of Diabetic retinopathy detection contest in Kaggle. Includes a good example of data augmentation.

Face recognition for right whales using deep learning – the authors used different ConvNets for localization and classification. Code and models are available.

Tensorflow: Convolutional neural networks for image classification on CIFAR-10 dataset

Implementing a CNN for text classification in Tensorflow

DeepDream implementation in TensorFlow

92.45% on CIFAR-10 in Torch – implements famous VGGNet network with batch normalization layers in Torch

Training and investigating Residual Nets – Residual networks perform very well on image classification tasks. Two researchers from Facebook and CornellTech implemented these networks in Torch

ConvNets in practice – lots of practical tips on using convolutional networks including data augmentation, transfer learning, fast implementations of convolution operation

遞歸神經(jīng)網(wǎng)絡(luò)

遞歸神經(jīng)網(wǎng)絡(luò)Recurrent entworks（RNNs）被設(shè)計(jì)用來處理序列數(shù)據(jù)（例如文本、股票、基因組、傳感器等）相關(guān)問題，通常應(yīng)用于語句分類（例如情感分析）和語音識(shí)別，也適用于文本生成甚至圖像生成。

教程如下：

The Unreasonable Effectiveness of Recurrent Neural Networks – describes how RNNs can generate text, math papers and C++ code

Hugo Larochelle’s course doesn’t cover recurrent neural networks (although it covers many topics that RNNs are used for). We suggest watching Recurrent Neural Nets and LSTMs by Nando de Freitas to fill the gap

10. Recurrent Neural Networks, Image Captioning, LSTM

13. Soft attention (starting at 38:00)

Michael Nielsen’s book stops at convolutional networks. In the Other approaches to deep neural nets section there is just a brief review of simple recurrent networks and LSTMs.

10. Sequence Modeling: Recurrent and Recursive Nets

Recurrent neural networks from Stanford’s CS224d (2016) by Richard Socher

Understanding LSTM Networks

遞歸神經(jīng)網(wǎng)絡(luò)的框架部署與應(yīng)用

Theano: Recurrent Neural Networks with Word Embeddings

Theano: LSTM Networks for Sentiment Analysis

Implementing a RNN with Python, Numpy and Theano

Lasagne implementation of Karpathy’s char-rnn

Combining CNN and RNN for spoken language identification in Lasagne

Automatic transliteration with LSTM using Lasagne

Tensorflow: Recurrent Neural Networks for language modeling

Recurrent Neural Networks in Tensorflow

Understanding and Implementing Deepmind’s DRAW Model

LSTM implementation explained

Torch implementation of Karpathy’s char-rnn

Autoencoders

Autoencoder是為非監(jiān)督式學(xué)習(xí)設(shè)計(jì)的神經(jīng)網(wǎng)絡(luò)，例如當(dāng)數(shù)據(jù)沒有標(biāo)記的情況。Autoencoder可以用來進(jìn)行數(shù)據(jù)維度消減，以及為其他神經(jīng)網(wǎng)絡(luò)進(jìn)行預(yù)訓(xùn)練，以及數(shù)據(jù)生成等。以下課程資源中，我們還收錄了Autoencoder與概率圖模型整合的一個(gè)autoencoders的變種，其背后的數(shù)學(xué)機(jī)理在下一章“概率圖模型”中會(huì)介紹。

推薦教程：

6. Autoencoder

7.6. Deep autoencoder

14. Videos and unsupervised learning (from 32:29) – this video also touches an exciting topic of generative adversarial networks.

14. Autoencoders

ConvNetJS Denoising Autoencoder demo

Karol Gregor on Variational Autoencoders and Image Generation

Autoencoder的部署

大多數(shù)autoencoders都非常容易部署，但我們還是建議您從簡單的開始嘗試。課程資源如下：

Theano: Denoising autoencoders

Diving Into TensorFlow With Stacked Autoencoders

Variational Autoencoder in TensorFlow

Training Autoencoders on ImageNet Using Torch 7

Building autoencoders in Keras

概率圖模型

概率圖模型（Probabilistic Graphical model，PGM）是統(tǒng)計(jì)學(xué)和機(jī)器學(xué)習(xí)交叉分支領(lǐng)域，關(guān)于概率圖模型的書籍和課程非常多，以下我們收錄的資源重點(diǎn)關(guān)注概率圖模型在深度學(xué)習(xí)場景中的應(yīng)用。其中Hugo Larochelles的課程介紹了一些非常著名的模型，而Deep Learning一書有整整四個(gè)章節(jié)專門介紹，并在最后一章介紹了十幾個(gè)模型。本領(lǐng)域的學(xué)習(xí)需要讀者掌握大量數(shù)學(xué)知識(shí)：

3. Conditional Random Fields

4. Training CRFs

5. Restricted Boltzman machine

7.7-7.9. Deep Belief Networks

9.10. Convolutional RBM

13. Linear Factor Models – first steps towards probabilistic models

16. Structured Probabilistic Models for Deep Learning

17. Monte Carlo Methods

18. Confronting the Partition Function

19. Approximate Inference

20. Deep Generative Models – includes Boltzmann machines (RBM, DBN, …), variational autoencoders, generative adversarial networks, autoregressive models etc.

Generative models – a blog post on variational autoencoders, generative adversarial networks and their improvements by OpenAI.

The Neural Network Zoo attempts to organize lots of architectures using a single scheme.

概率圖模型的部署

高級(jí)框架（Lasagne、Keras）不支持概率圖模型的部署，但是Theano、Tensorflow和Torch有很多可用的代碼。

Restricted Boltzmann Machines in Theano

Deep Belief Networks in Theano

Generating Large Images from Latent Vectors – uses a combination of variational autoencoders and generative adversarial networks.

Image Completion with Deep Learning in TensorFlow – another application of generative adversarial networks.

Generating Faces with Torch – Torch implementation of Generative Adversarial Networks

精華論文、視頻與論壇匯總

Deep learning papers reading roadmap 深度學(xué)習(xí)重要論文的大清單。

Arxiv Sanity Preserver 為瀏覽 arXiv上的論文提供了一個(gè)漂亮的界面.

Videolectures.net 含有大量關(guān)于深度學(xué)習(xí)的高級(jí)議題視頻

/r/MachineLearning 一個(gè)非常活躍的Reddit分支. 幾乎所有重要的新論文這里都有討論。

鏈接：www.ctocio.com/ccnews/23027.html（點(diǎn)擊尾部閱讀原文前往）