Activation functions are a key component of neural networks. They introduce non-linearity into the model, allowing it to learn complex patterns and relationships in data. Without activation functions, a neural network would behave like a simple linear model and fail to capture real-world complexities.<\/p>\n\n\n\n
Why Activation Functions are Important<\/strong><\/p>\n\n\n\n 1. ReLU (Rectified Linear Unit)<\/strong> Formula<\/strong> How it Works<\/strong><\/p>\n\n\n\n Advantages<\/strong><\/p>\n\n\n\n Limitations<\/strong><\/p>\n\n\n\n Example<\/strong><\/p>\n\n\n\n 2. Sigmoid Function<\/strong> Formula<\/strong> How it Works<\/strong><\/p>\n\n\n\n Advantages<\/strong><\/p>\n\n\n\n Limitations<\/strong><\/p>\n\n\n\n Example<\/strong><\/p>\n\n\n\n 3. Tanh (Hyperbolic Tangent)<\/strong> Formula<\/strong> How it Works<\/strong><\/p>\n\n\n\n Advantages<\/strong><\/p>\n\n\n\n Limitations<\/strong><\/p>\n\n\n\n Example<\/strong><\/p>\n\n\n\n Comparison of Activation Functions<\/strong><\/p>\n\n\n\n When to Use Which Function<\/strong><\/p>\n\n\n\n Applications in Deep Learning<\/strong><\/p>\n\n\n\n Lesson Summary<\/strong>\n
ReLU is the most commonly used activation function in deep learning.<\/p>\n\n\n\n
f(x) = max(0, x)<\/p>\n\n\n\n\n
\n
\n
import numpy as npdef relu(x):
return np.maximum(0, x)print(relu(np.array([-2, -1, 0, 1, 2])))<\/pre>\n\n\n\n
Sigmoid is widely used for binary classification problems.<\/p>\n\n\n\n
f(x) = 1 \/ (1 + e^(-x))<\/p>\n\n\n\n\n
\n
\n
def sigmoid(x):
return 1 \/ (1 + np.exp(-x))print(sigmoid(np.array([-2, 0, 2])))<\/pre>\n\n\n\n
Tanh is similar to sigmoid but outputs values between -1 and 1.<\/p>\n\n\n\n
f(x) = tanh(x)<\/p>\n\n\n\n\n
\n
\n
def tanh(x):
return np.tanh(x)print(tanh(np.array([-2, 0, 2])))<\/pre>\n\n\n\n\n
\n
\n
In this lesson, you learned about activation functions and their role in neural networks. You explored ReLU, Sigmoid, and Tanh functions, their formulas, advantages, limitations, and use cases. Activation functions are essential for enabling neural networks to learn complex patterns and make accurate predictions.<\/p>\n\n\n