不定时的看看深度学习deep learning

目的：不做研究，就是自己学起来，看看到底有什么用。不做严格计划了。

一楼贴总结和资料

==================================
Hands on practice with kaggle https://www.kaggle.com/c/facial- ... p-learning-tutorial
Or just follow the udacity MOOC's assignment with tensorflow https://github.com/tensorflow/te ... ow/examples/udacity
Books
先看这本书 http://www.deeplearningbook.org
Microsoft book https://www.microsoft.com/en-us/ ... ng-Vol7-SIG-039.pdf
Tutorialhttp://deeplearning.stanford.edu/tutorial/

Potential MOOC
Udacity: https://www.udacity.com/course/deep-learning--ud730
--- Finished the video part, it's a very very gentle intro, certainly not bad, but certainly not enough by itself.
to start in 2016/9: Coursera: https://www.coursera.org/learn/neural-networks
Udacity还有一门reinforcement learning, 先放这里 https://www.udacity.com/course/reinforcement-learning--ud600
Udacity self driving car nano degree: to start

Stanford undergrad
visual recognition http://cs231n.stanford.edu/
NLP http://cs224d.stanford.edu/

======================================
Andrew Ng

The andrew Ng course on ML is obviously all good, but my fav is this section: https://www.coursera.org/learn/machine-learning/lecture/x62iE/error-analysis This really is a largely ignored part of ML system design, the problem is so commonly seen in practice contrast this with a newer version of his talk with more emphasis on DL Nuts and Bolts of Applying Deep Learning (Andrew Ng) - https://youtu.be/F1ka6a13S9I fantastic stuff

=======================================
youtube上名人讲DL也是大堆大堆的，对于不喜欢看太多字，但是喜欢听人讲的来说也不错
Deep learning summer school 2016: http://videolectures.net/deeplearning2016_montreal
==================================

2016 deep learning school: From youtube: Published on Sep 27, 2016 The talks at the Deep Learning School on September 24/25, 2016 were amazing. I clipped out individual talks from the full live streams and provided links to each below in case that's useful for people who want to watch specific talks several times (like I do). Please check out the official website (http://www.bayareadlschool.org) and full live streams below. Having read, watched, and presented deep learning material over the past few years, I have to say that this is one of the best collection of introductory deep learning talks I've yet encountered. Here are links to the individual talks and the full live streams for the two days: 1. Foundations of Deep Learning (Hugo Larochelle, Twitter) - https://youtu.be/zij_FTbJHsk 2. Deep Learning for Computer Vision (Andrej Karpathy, OpenAI) - https://youtu.be/u6aEYuemt0M 3. Deep Learning for Natural Language Processing (Richard Socher, Salesforce) - https://youtu.be/oGk1v1jQITw 4. TensorFlow Tutorial (Sherry Moore, Google Brain) - https://youtu.be/Ejec3ID_h0w 5. Foundations of Unsupervised Deep Learning (Ruslan Salakhutdinov, CMU) - https://youtu.be/rK6bchqeaN8 6. Nuts and Bolts of Applying Deep Learning (Andrew Ng) - https://youtu.be/F1ka6a13S9I 7. Deep Reinforcement Learning (John Schulman, OpenAI) - https://youtu.be/PtAIh9KSnjo 8. Theano Tutorial (Pascal Lamblin, MILA) - https://youtu.be/OU8I1oJ9HhI 9. Deep Learning for Speech Recognition (Adam Coates, Baidu) - https://youtu.be/g-sndkf7mCs 10. Torch Tutorial (Alex Wiltschko, Twitter) - https://youtu.be/L1sHcj3qDNc 11. Sequence to Sequence Deep Learning (Quoc Le, Google) - https://youtu.be/G5RY_SUJih4 12. Foundations and Challenges of Deep Learning (Yoshua Bengio) - https://youtu.be/11rsu_WwZTc Full Day Live Streams: Day 1: https://youtu.be/eyovmAtoUx0 Day 2: https://youtu.be/9dXiAecyJrY Go to http://www.bayareadlschool.org for more information on the event, speaker bios, slides, etc. Huge thanks to the organizers (Shubho Sengupta et al) for making this event happen.

==================================
Learning Goal
个人对application更感兴趣，尤其在除了NLP, img/audio之外，在传统统计学习，机器学习领域的应用
Disclaimer: I am a "traditional" data scientist/statistician, with hands-on experience in traditional ML, so I will tend to approach DL from this angle, and not from a traditional "CS" angle.

Docker image
最好的宝贝，没有之一 https://github.com/saiprashanths/dl-docker 好用的想哭。不然就会抓狂到想哭。
caffe model zoo:
https://github.com/BVLC/caffe/wiki/Model-Zoo
有人说可以用这个https://github.com/ethereon/caffe-tensorflow 把其变成TF zoo Deep Viz toolbox
https://github.com/yosinski/deep-visualization-toolbox

course4 上完了。不错的。
fast.ai的pytorch 打算12月中登出，那时候估计有空看的。

earlgrey 发表于 2017-9-10 21:47
RL有什么好的资源吗？

udacity has one from gatech (but no hw)
berkeley has one
i liked the very light intro of "intro AI" back from 2011 too, on udacity

The andrew Ng course on ML is obviously all good, but my fav is this section:
https://www.coursera.org/learn/machine-learning/lecture/x62iE/error-analysis
This really is a largely ignored part of ML system design, the problem is so commonly seen in practice


contrast this with a newer version of his talk with more emphasis on DL
Nuts and Bolts of Applying Deep Learning (Andrew Ng) - https://youtu.be/F1ka6a13S9I

fantastic stuff

非常感谢分享哇 现在在申请中 感觉对ps的帮助很大哈

感谢K姐分享！

2016 deep learning school:
From youtube:

Published on Sep 27, 2016
The talks at the Deep Learning School on September 24/25, 2016 were amazing. I clipped out individual talks from the full live streams and provided links to each below in case that's useful for people who want to watch specific talks several times (like I do). Please check out the official website (http://www.bayareadlschool.org) and full live streams below.

Having read, watched, and presented deep learning material over the past few years, I have to say that this is one of the best collection of introductory deep learning talks I've yet encountered. Here are links to the individual talks and the full live streams for the two days:

1. Foundations of Deep Learning (Hugo Larochelle, Twitter) - https://youtu.be/zij_FTbJHsk
2. Deep Learning for Computer Vision (Andrej Karpathy, OpenAI) - https://youtu.be/u6aEYuemt0M
3. Deep Learning for Natural Language Processing (Richard Socher, Salesforce) - https://youtu.be/oGk1v1jQITw
4. TensorFlow Tutorial (Sherry Moore, Google Brain) - https://youtu.be/Ejec3ID_h0w
5. Foundations of Unsupervised Deep Learning (Ruslan Salakhutdinov, CMU) - https://youtu.be/rK6bchqeaN8
6. Nuts and Bolts of Applying Deep Learning (Andrew Ng) - https://youtu.be/F1ka6a13S9I
7. Deep Reinforcement Learning (John Schulman, OpenAI) - https://youtu.be/PtAIh9KSnjo
8. Theano Tutorial (Pascal Lamblin, MILA) - https://youtu.be/OU8I1oJ9HhI
9. Deep Learning for Speech Recognition (Adam Coates, Baidu) - https://youtu.be/g-sndkf7mCs
10. Torch Tutorial (Alex Wiltschko, Twitter) - https://youtu.be/L1sHcj3qDNc
11. Sequence to Sequence Deep Learning (Quoc Le, Google) - https://youtu.be/G5RY_SUJih4
12. Foundations and Challenges of Deep Learning (Yoshua Bengio) - https://youtu.be/11rsu_WwZTc

Full Day Live Streams:
Day 1: https://youtu.be/eyovmAtoUx0
Day 2: https://youtu.be/9dXiAecyJrY

Go to http://www.bayareadlschool.org for more information on the event, speaker bios, slides, etc. Huge thanks to the organizers (Shubho Sengupta et al) for making this event happen.

2016/10/18
- went through the udacity DL video part. I may not have time to go through all the tensorflow stuff until later.

will dab into traditional classification/regression tasks (for work), and then go through basic NLP/vision/sequence

https://www.coursera.org/learn/neural-networks/home/week/1

to follow by reading this tutorial http://ufldl.stanford.edu/tutorial

Udacity has an interesting course but it's too shallow to really "practice" DL. Good as a very first intro course, given you already know ML but knows little about DL.
The exercises were not helpful at all.

The coursera NN for ML course is dry as ****. Maybe good content, but I can't go through with it. Jeff Hinton is the giant, but that course is supremely dry.....maybe will revisit after going through the shallower tutorials.

Stanford 231n is a fantastic course, and like many other stanford courses (in all kinds of other areas), this one is engaging, and establishes the intuition superbly ---- this is a huge advantage that I see in learning. I am so green with envy, re Stanford students!!!!!

Also, instead of Andrew Ng's coursera one, I will review CS229 (the full course) instead.
https://see.stanford.edu/Course/CS229
Apparantly, there is a lot more content....

Also the optimization courses here would be a nice refresher
https://see.stanford.edu/Course

following the good tradition on this board, here's the learning plan for ML (not necessarily DL)

Stanford 231n
Stanford 229 (Andrew Ng's ML) review: Notes 1-12, Lecture 1-20, I don't plan to do much assignment
==========================
10/26 cs231n:
linear classification; back prop

10/27 cs231n:
neural net, train nn