Sigrid Keydana, Trivadis

02/06/2018

This is part 2, please see part 1 first :-)

How ever deep we make our network, if we just chain layers of matrix multiplication one after another, all we get is a linear combination of the inputs.

How can we solve non-linear problems with neural networks?

```
sigmoid <- function(x) 1/(1 + exp(-x))
x <- seq(-10,10, by = 0.01)
plot(x, sigmoid(x), type = "l", xlab = "", ylab = "")
```

```
relu <- function(x){
x[x<0] <- 0
x
}
x <- seq(-10,10, by = 0.01)
plot(x, relu(x), type = "l", xlab = "", ylab = "")
```

```
x <- seq(-10,10, by = 0.01)
plot(x, tanh(x), type = "l", xlab = "", ylab = "")
```

LeNet: First successful application of convolutional neural networks by Yann LeCun, Yoshua Bengio et al.