- [[neural network matrix representation]] # Idea The activation of a unit/neuron in a [[neural networks|neural network]] is given by $ a_j^{[l]}=g\left(\overrightarrow{\mathrm{w}}_j^{[l]} \cdot \overrightarrow{\mathrm{a}}^{[l-1]}+b_j^{[l]}\right) $ - $a$ is activation or output of a neuron - $g$ is the activation function (often [[sigmoid function]] or [[rectified linear unit]]) - $w$ is a vector of weights (a parameter) - $b$ is bias (parameter; like intercept), a real value - superscript $[l]$ refers to layer, starting from 0 (input layer) - subscript $j$ refers to unit/neuron within a given layer > [!warning] Number of layers > By convention, the first layer is known as layer 0. If a neural network has 5 layers, it means it has 4 "hidden" (or inner) layers and 1 output layer. If we also include layer 0, we actually have 6 layers. ![[20231226144223.png]] ![[20231224173647.png]] ## Examples > [!question]- Write the expression for the activation of the 3rd neuron in layer 2. > $a_3^{[2]}=g\left(\overrightarrow{\mathrm{w}}_3^{[2]} \cdot \overrightarrow{\mathrm{a}}^{[1]}+b_3^{[2]}\right)$ > [!question]- Describe this expression. $a_4^{[6]}=g\left(\overrightarrow{\mathrm{w}}_4^{[6]} \cdot \overrightarrow{\mathrm{a}}^{[5]}+b_4^{[6]}\right)$ > The activation of the 4th neuron in layer 6 of a neural network. # References - [Neural network layer - Neural Networks | Coursera](https://www.coursera.org/learn/advanced-learning-algorithms/lecture/z5sks/neural-network-layer) - [More complex neural networks - Neural Networks | Coursera](https://www.coursera.org/learn/advanced-learning-algorithms/lecture/a5AfY/more-complex-neural-networks)