Forward Propagation

Forward propagation is the process by which input data passes through a neural network to generate predictions. It is the first step in training and using neural networks, allowing the model to compute outputs based on current weights and biases. Understanding forward propagation is essential for grasping how neural networks learn.

How Forward Propagation Works

1. Input Layer – Receives the input features of the dataset.
2. Weighted Sum – Each input is multiplied by its corresponding weight, and a bias is added:
   z = (w1 × x1) + (w2 × x2) + … + (wn × xn) + b
3. Activation Function – The weighted sum is passed through an activation function to introduce non-linearity:
   a = f(z)
4. Hidden Layers – The output from the activation function becomes the input for the next layer.
5. Output Layer – Produces the final prediction, which can be a number (regression) or probability (classification).

Mathematical Representation
For a single neuron:

• z = Σ (wi × xi) + b
• a = f(z)
  For multiple layers, this process is repeated for each neuron in every layer.

Example: Forward Propagation in Python

import numpy as np# Inputs
X = np.array([1, 2])# Weights and bias
weights = np.array([0.5, 0.3])
bias = 0.1# Activation function (ReLU)
def relu(x):
    return np.maximum(0, x)# Forward propagation
z = np.dot(X, weights) + bias
output = relu(z)print("Weighted sum (z):", z)
print("Output after activation (a):", output)

Key Points About Forward Propagation

• Deterministic Process: Computes output based on current parameters (weights and biases).
• No Learning Happens: Forward propagation only calculates outputs; learning occurs during backpropagation.
• Layer-by-Layer Computation: Each layer transforms the input before passing it to the next layer.

Applications

• Making predictions on new input data
• Evaluating the model during training
• Generating probabilities for classification problems
• Feature transformation in deep learning models

Lesson Summary
Forward propagation is the process of passing input data through a neural network to generate predictions. It involves calculating weighted sums, applying activation functions, and propagating outputs layer by layer. This process is fundamental to understanding how neural networks operate before applying backpropagation for learning.