Regression models are Machine Learning algorithms used to predict continuous numerical values.
If the output is a number (price, score, temperature, salary), regression is used.
Example:
Predict house price
Predict sales revenue
Predict student marks
Predict stock value
What is Regression?
Regression finds the relationship between:
Independent variables (Features – X)
Dependent variable (Target – y)
The goal is to predict a continuous output value.
Example:
X → Study Hours
y → Exam Score
The model learns how study hours affect score.
Types of Regression Models
1. Linear Regression
The simplest regression model.
It assumes a linear relationship between input and output.
Equation:
y = mx + b
Used when data follows a straight-line pattern.
Example use cases:
House price prediction
Sales forecasting
Trend analysis
2. Multiple Linear Regression
Used when there are multiple input features.
Example:
Price = f(Size, Location, Bedrooms)
Instead of one variable, it uses many variables.
3. Polynomial Regression
Used when data is not linear.
It adds polynomial terms to capture curved relationships.
Example:
y = ax² + bx + c
Useful when data shows a curve.
4. Ridge Regression
Used to prevent overfitting.
Adds a penalty term to reduce large coefficients.
Useful when:
- Many features
- Multicollinearity exists
5. Lasso Regression
Similar to Ridge but can reduce some coefficients to zero.
Useful for:
- Feature selection
- Reducing model complexity
6. Decision Tree Regressor
Splits data into branches to make predictions.
Works well for:
- Non-linear data
- Complex relationships
7. Random Forest Regressor
Uses multiple decision trees.
Improves:
Accuracy
Stability
Generalization
Evaluation Metrics for Regression
To measure performance:
1. Mean Absolute Error (MAE)
Average of absolute differences between actual and predicted values.
Lower is better.
2. Mean Squared Error (MSE)
Average of squared errors.
Penalizes large errors more.
3. Root Mean Squared Error (RMSE)
Square root of MSE.
More interpretable because it is in original unit.
4. R-Squared (R²)
Shows how well model explains variance.
Value range:
0 to 1
Higher is better.
Example Using Scikit-Learn
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
import numpy as np# Sample Data
X = np.array([[1], [2], [3], [4], [5]])
y = np.array([2, 4, 6, 8, 10])# Split Data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)# Create Model
model = LinearRegression()# Train Model
model.fit(X_train, y_train)# Predict
predictions = model.predict(X_test)print(predictions)
When to Use Regression Models
Use regression when:
Target variable is numeric
You need forecasting
You want to understand relationships between variables
Common Real-World Applications
Real estate price prediction
Sales forecasting
Demand prediction
Stock price estimation
Risk analysis
Key Takeaway
Regression models are used to predict continuous numerical values.
From simple Linear Regression to advanced models like Random Forest, regression techniques help solve real-world prediction problems efficiently.