Data Cleaning is the process of detecting and correcting errors, missing values, and inconsistencies in a dataset.<\/p>\n\n\n\n
In Data Analytics, clean data is essential for accurate analysis and decision-making.<\/p>\n\n\n\n
First, import pandas:<\/p>\n\n\n\n
import pandas as pd<\/pre>\n\n\n\nLoad a dataset:<\/p>\n\n\n\n
df = pd.read_csv(\"data.csv\")<\/pre>\n\n\n\nWhy Data Cleaning is Important<\/h2>\n\n\n\n
Real-world data often contains:<\/p>\n\n\n\n
Missing values
Duplicate records
Incorrect data types
Extra spaces
Inconsistent formatting
Outliers<\/p>\n\n\n\nCleaning ensures reliable analysis.<\/p>\n\n\n\n
1. Handling Missing Values<\/h2>\n\n\n\n
Check missing values:<\/p>\n\n\n\n
df.isnull().sum()<\/pre>\n\n\n\nRemove rows with missing values:<\/p>\n\n\n\n
df.dropna()<\/pre>\n\n\n\nRemove columns with missing values:<\/p>\n\n\n\n
df.dropna(axis=1)<\/pre>\n\n\n\nFill missing values with a specific value:<\/p>\n\n\n\n
df.fillna(0)<\/pre>\n\n\n\nFill with mean value:<\/p>\n\n\n\n
df[\"Age\"].fillna(df[\"Age\"].mean())<\/pre>\n\n\n\n2. Removing Duplicates<\/h2>\n\n\n\n
Check duplicate rows:<\/p>\n\n\n\n
df.duplicated().sum()<\/pre>\n\n\n\nRemove duplicates:<\/p>\n\n\n\n
df.drop_duplicates()<\/pre>\n\n\n\n3. Fixing Data Types<\/h2>\n\n\n\n
Check data types:<\/p>\n\n\n\n
df.dtypes<\/pre>\n\n\n\nConvert data type:<\/p>\n\n\n\n
df[\"Age\"] = df[\"Age\"].astype(int)<\/pre>\n\n\n\nConvert to datetime:<\/p>\n\n\n\n
df[\"Date\"] = pd.to_datetime(df[\"Date\"])<\/pre>\n\n\n\n4. Renaming Columns<\/h2>\n\n\n\n
df.rename(columns={\"OldName\": \"NewName\"}, inplace=True)<\/pre>\n\n\n\n5. Removing Extra Spaces<\/h2>\n\n\n\n
df[\"Name\"] = df[\"Name\"].str.strip()<\/pre>\n\n\n\n6. Replacing Values<\/h2>\n\n\n\n
df[\"Gender\"] = df[\"Gender\"].replace(\"M\", \"Male\")<\/pre>\n\n\n\n7. Handling Outliers (Basic Method)<\/h2>\n\n\n\n
Using condition:<\/p>\n\n\n\n
df[df[\"Salary\"] < 100000]<\/pre>\n\n\n\nOr remove extreme values:<\/p>\n\n\n\n
df = df[df[\"Salary\"] < df[\"Salary\"].quantile(0.95)]<\/pre>\n\n\n\n8. Changing Text Case<\/h2>\n\n\n\n
df[\"Name\"] = df[\"Name\"].str.lower()
df[\"Name\"] = df[\"Name\"].str.upper()<\/pre>\n\n\n\nData Cleaning Workflow<\/h2>\n\n\n\n
\n
- Load dataset<\/li>\n\n\n\n
- Check structure and summary<\/li>\n\n\n\n
- Identify missing values<\/li>\n\n\n\n
- Remove duplicates<\/li>\n\n\n\n
- Fix data types<\/li>\n\n\n\n
- Standardize formatting<\/li>\n\n\n\n
- Handle outliers<\/li>\n<\/ol>\n\n\n\n
Why Data Cleaning is Critical in Analytics<\/h2>\n\n\n\n
Accurate insights depend on clean data.
Poor data quality leads to wrong decisions.
Data cleaning is often 60\u201370% of a data analyst\u2019s work.<\/p>\n\n\n\nKey Takeaway<\/h2>\n\n\n\n
Data Cleaning is the foundation of Data Analytics.
Before analyzing or visualizing data, always clean and prepare it properly for accurate and meaningful results.<\/p>\n\n\n