Neural Networks

Model Representation

1. input and output

   • x₁, x₂ are input layers which go through a node and then generate an output layer
   • h_θ(x) = 1 / (1 + -e^θTx)
   • x₀ is usually 1, which is also called a bias unit

2. sigmoid activation function

   • g(z) = 1 / (1 + e^-z)

3. Neural network is a group of neurons put together

   • hidden layers are between the input layer and the output layer
   • weights are also matrix
   • s_j units in layer j, s_j+1 units in layer j+1
   • the dimension will be s_j+1 * (s_j + 1)

Intuition

1. Non-linear classification
   • if data can be clusterd, try to use a simple representation of a given data set
   • x₁ XOR x₂: true if either one is true
   • x₁ XNOR x₂ : NOT (x₁ XOR x₂)
   • x₁ AND x₂
   • do some calculation and see if the function becomes 1 with x₁ and x₂ are either 0 or 1