Introduction to Deep Learning

Perceptron

• The perceptron is a simple model of a biological neuron, introduced by Frank Rosenblatt in 1958.
• It is a binary classifier that maps input features to one of two classes using a linear decision boundary.
• The perceptron computes a weighted sum of the input features, applies an activation function (step function), and outputs a class label.
• The perceptron can be trained using the Perceptron Learning Algorithm (PLA), which adjusts the weights based on misclassifications.

Perceptron

Perceptron

Rosenblatt’s Perceptron Learning Algorithm (PLA)

1. Initialize the weight vector \[\mathbf w^{(0)} \leftarrow \mathbf 0 \;(\text{or small random values}), \qquad b^{(0)} \leftarrow 0\]

2. Loop over the training set \(\{(\mathbf x_i,\,y_i)\}_{i=1}^{N}\) with \(y_i \in \{-1,+1\}\)

   • Predict \(\hat y_i = \operatorname{sign}(\mathbf w\!\cdot\!\mathbf x_i + b)\)
   • If it is correct, do nothing
   • If it is wrong, update \[\mathbf w \gets \mathbf w + \eta \cdot y_i \cdot \mathbf x_i, \qquad b \gets b + \eta \cdot y_i\] where \(\eta>0\) is a fixed learning rate (often \(\eta = 1\)).

3. Repeat passes (epochs) until all points are classified correctly or a preset iteration budget is exhausted.

Perceptron

Demo