Biological neuron and artificial neuron

Source: [10]

A deep representation is a composition of many functions

\( x \xrightarrow{w_1} h_1 \xrightarrow{w_2} h_2 \xrightarrow{w_3} ... \xrightarrow{w_n} h_n \xrightarrow{w_{n+1}} y \)

Easy? Difficult?

• walk
• talk
• play chess
• solve matrix computations

• controlled movement
• speech recognition
• speech generation
• object recognition and object localization

What are the correct pixel values for a “bike” feature?

• race bike, mountain bike, e-bike?
• pixels in the shadow may be much darker
• what if bike is mostly obscured by rider standing in front?

… a layer at a time

Source: [12]

Just a sec - let's meet a real neural network first!

Play around in the browser:

• ConvNetJS
• TensorFlow playground

We need:

• a way to quantify our current (e.g., classification) error
• a way to reduce error on subsequent iterations
• a way to propagate our improvement logic from the output layer all the way back through the network!

The loss (or cost) function indicates the cost incurred from false prediction / misclassification

Probably the best-known loss function in machine learning is mean squared error:

\( \frac{1}{n} \sum_n{(\hat{y} - y)^2} \)

Most of the time, in deep learning we use cross entropy:

\( - \sum_j{t_j log(y_j)} \)

This is the negative log probability of the right answer.

• basically, just the chain rule: \( \frac{dz}{dx} = \frac{dz}{dy} \frac{dy}{dx} \)
• chained over several layers:

Source: [14]

• imagine output of \( f = (x + y) * z = -12 \) “wants” to get bigger
• this could happen by \( q \) getting smaller -> \( q \) receives negative gradient \( \frac{df}{dq} = -4 \)
• \( q \) just passes on this gradient to \( x \) and \( y \), as \( \frac{dq}{dx} = 1 \) and \( \frac{dq}{dy} = 1 \)
• alternatively, it could happen by \( z \) getting bigger -> \( z \) receives positive gradient \( \frac{df}{dz} = 3 \)

Source: [13]

• CNNs (Convolutional Neural Networks) for computer vision
• RNNs (Recurrent Neural Networks) for Natural Language Processing
• Deep Reinforcement Learning for real-life learning

VISION

In classification, the required output is a probability for each class.

Source: [13]

In localization, the network needs to identify the position of an object in an image.

Source: [17]

In object detection (a.k.a. image recognition), the network has to classify and localize multiple objects in an image.

Source: [18]

In segmentation, the network needs to predict a class value for each input pixel.

Source: [19]

Semantic segmentation segments by class, instance segmentation by class instance.

Source: [20]

Convolutional Neural Networks

(Strictly, this is cross-correlation, but it doesn't matter)

Source: [13]

Blur: \( \begin{bmatrix}1 & 1 & 1\\1 & 1 & 1\\1 & 1 & 1\end{bmatrix} \), sharpen: \( \begin{bmatrix}0 & -1 & 0\\-1 & 5 & -1\\0 & -1 & 0\end{bmatrix} \), edge detect: \( \begin{bmatrix}0 & 1 & 0\\1 & -4 & 1\\0 & 1 & 0\end{bmatrix} \)

see: https://docs.gimp.org/en/plug-in-convmatrix.html

VISION

LANGUAGE

How do we handle sequences

• language: words, sentences, paragraphs…
• all kinds of serial information: sensor data, stock prices…

?

Jane walked into the room. John walked in too. It was late in the day, and everyone was walking home after a long day at work. Jane said hi to ___

Source: [21]

Recurrent neural networks

The basic RNN at every step combines new input and existing state.

Source: [22]

Sometimes we also need to forget!

The LSTM (Long Term Short Memory) architecture adds an additional state layer, the cell state

Source: [22]

The LSTM cell state is protected by three gates, the forget, input, and output gates:

Source: [22]

In translation, we have two sets of sequential data, one on the source and one on the target side!

Enter: sequence-to-sequence models

Source: [23]

VISION

LANGUAGE

VISION <-> LANGUAGE

VISION

LANGUAGE

VISION <-> LANGUAGE

VISION <-> SOUND

VISION

LANGUAGE

VISION <-> LANGUAGE

VISION <-> SOUND

LIFE (SORT OF)

If I get a reward many many actions later…

… how do I find out what concrete action I'm getting the reward for?

EXPLOITATION vs. EXPLORATION

Learn to maximize future (discounted) reward:

Source: [1]

• supervised learning: reasoning by memorization
• pure reinforcement learning: reasoning by trial-and-error
• can we get less brute-force here?

“Reinforcement learning + deep learning = AI” (David Silver, Google Deep Mind)

For structured reading:

• “Awesome deep learning”: https://github.com/ChristosChristofidis/awesome-deep-learning
• “Deep Learning Papers reading roadmap”: https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap
• Andrey Karpathy's “Arxiv Sanity Preserver”: http://www.arxiv-sanity.com/

Just wanna have some cool fun?

• Andrey Karpathy's blog: http://karpathy.github.io/ (especially: http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
• Christopher Olah's blog: http://colah.github.io/

[1] MIT 6.S094 Deep Learning for Self-Driving Cars Lecture Slides

[3] Esteva et al. Dermatologist-level classification of skin cancer with deep neural networks

[4] Wikipedia. AlphaGo versus Lee Sedol

[5] Yoshihara et al. Leveraging temporal properties of news events for stock market prediction

[6] The Weather Company. Seasonal forecast

[7] Neural Network Learns to Select Potential Anticancer Drugs

[8] Google Research Blog. AlphaGo: Mastering the ancient game of Go with Machine Learning

[9] The Weather Company Launches Deep Thunder - the World’s Most Advanced Hyper-Local Weather Forecasting Model for Businesses

[10] Stergiou, C. and Siganos, D. Artificial neurons

[11] University of Waterloo Deep Learning Slides

[12] Goodfellow et al. Deep Learning

[13] Stanford CS231n Convolutional Neural Networks Lecture Notes

[14] Chris Olah. Calculus on Computational Graphs: Backpropagation

[15] Parkhi et al. Cats and Dogs

[16] Krizhevsky et al. ImageNet Classification with Deep Convolutional Neural Networks

[17] Erhan et al. Scalable Object Detection using Deep Neural Networks

[18] ImageNet Large Scale Visual Recognition Challenge 2014 (ILSVRC2014)

[19] Long et al. Fully Convolutional Networks for Semantic Segmentation

[20] Silberman et al. Instance Segmentation of Indoor Scenes using a Coverage Loss

[21] Stanford CS 224D Deep Learning for NLP Lecture Notes

[22] Chris Olah. Understanding LSTM Networks

[23] Tensorflow seq2seq tutorial

[24] Wu et al. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

[25] Johnson et al. Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

[26] Karpathy, A. and Li, F. Deep Visual-Semantic Alignments for Generating Image Descriptions

[27] Owens et al. Visually indicated sounds

[29] Google Deep Mind. AlphaGo tutorial