Student Dropout Risk Prediction

# 🎓 OULAD Student Dropout/Failure Prediction — Stage-Wise Pipeline **Goal:** Predict which students are at risk of dropping out or failing using the Open University Learning Analytics Dataset (OULAD) — a rich public dataset covering **demographics, assessment scores, and VLE (Virtual Learning Environment) clickstream activity**. **Dataset:** OULAD (Kaggle: `saksh1mishr4/oulada-clean-dataset-for-mooc-dropout-predition`) **Rows:** 32,593 student-course enrolments | **Features:** 21 columns (demographics + engagement + assessments) **Target:** Binary — `at_risk = 1` (Fail/Withdrawn) vs `0` (Pass/Distinction) **Models:** XGBoost Classifier + Logistic Regression baseline **Explainability:** SHAP (TreeExplainer)

%pip install kagglehub==0.3.10 shap==0.46.0 xgboost==2.1.3 -q

Note: you may need to restart the kernel to use updated packages.

# ============================================================
# Cell: Imports & Aesthetic Configuration
# ============================================================
import warnings
warnings.filterwarnings('ignore')

import kagglehub
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.ticker as mticker
import seaborn as sns
import shap
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (accuracy_score, precision_score, recall_score,
…

✅ Libraries loaded successfully.
✅ Style configuration loaded.

# ============================================================
# Step 1 — Download & Load Dataset
# ============================================================
print("=" * 70)
print("📥 DOWNLOADING OULAD DATASET")
print("=" * 70)

path = kagglehub.dataset_download("saksh1mishr4/oulada-clean-dataset-for-mooc-dropout-prediction")
print(f"  → Downloaded to: {path}")

df = pd.read_csv(f"{path}/final_dataset.csv")
print(f"\n  Shape: {df.shape[0]:,} rows × {df.shape[1]} columns")
print(f"\n  Columns ({len(df.columns)}):")
for c in df.columns:
    print(f"    • {c:30s}  [{df[c].dtype}]")

print("\n  ── First 5 rows ──")
display(df.head(5))
…

======================================================================
📥 DOWNLOADING OULAD DATASET
======================================================================
Warning: Looks like you're using an outdated `kagglehub` version (installed: 0.3.10), please consider upgrading to the latest version (1.0.2).
  → Downloaded to: /root/.cache/kagglehub/datasets/saksh1mishr4/oulada-clean-dataset-for-mooc-dropout-prediction/versions/1

  Shape: 32,593 rows × 21 columns

  Columns (21):
    • code…

# ============================================================
# Step 2a — EDA: Overview, Missing Values & Target
# ============================================================
print("=" * 70)
print("🔍 EDA: DATA OVERVIEW")
print("=" * 70)

# Missing values
mv = df.isnull().sum()
mv = mv[mv > 0]
if len(mv) > 0:
    print("\n  ⚠️  Missing Values Found:")
    for col, cnt in mv.items():
        print(f"      {col}: {cnt} ({cnt/len(df)*100:.2f}%)")
else:
    print("\n  ✅ No missing values — dataset is clean!")

# Target distribution
…

======================================================================
🔍 EDA: DATA OVERVIEW
======================================================================

  ✅ No missing values — dataset is clean!

  ── Target: final_result ──
      Pass              12361  (37.9%)
      Withdrawn         10156  (31.2%)
      Fail               7052  (21.6%)
      Distinction        3024  (9.3%)

  ── Binary Target: at_risk ──
      0 = Pass/Distinction:    15385  (47.2%)
      1 = Fail/Withdrawn:…

# ============================================================
# Step 2b — EDA: Target Distribution Visualizations
# ============================================================
print("📊 TARGET DISTRIBUTION VISUALIZATIONS\n")

fig, axes = plt.subplots(1, 3, figsize=(20, 6))
fig.suptitle('🎯 Target Variable: Student Final Outcome', fontsize=15, fontweight='bold', y=1.02)

# 1. Donut pie chart for original 4-class target
ax = axes[0]
colors_donut = ['#27ae60', '#e74c3c', '#f39c12', '#3498db']
explode = (0, 0.05, 0.05, 0)
wedges, texts, autotexts = ax.pie(
    target_counts.values, labels=None, autopct='%1.1f%%',
    startangle=90, colors=colors_donut, explode=explode,
    pctdistance=0.78, textprops={'fontsize': 9, 'fontweight': 'bold'})
# Draw centre circle for donut effect
centre = plt.Circle((0, 0), 0.55, fc='white', linewidth=1.5, edgecolor='#ddd')
…

📊 TARGET DISTRIBUTION VISUALIZATIONS

  ✅ Saved: fig_target_distribution.png

# ============================================================
# Step 2c — EDA: Feature Distributions by Risk (Violin + KDE)
# ============================================================
print("📊 FEATURE DISTRIBUTIONS BY RISK STATUS\n")

# Pick top 6 numeric features correlating with at_risk
target_corr = df[num_cols].corrwith(df['at_risk']).abs().sort_values(ascending=False)
top6_feats = target_corr.head(6).index.tolist()

fig, axes = plt.subplots(2, 3, figsize=(20, 12))
fig.suptitle('🎻 Feature Distributions: At-Risk (Red) vs Not at-Risk (Green)', fontsize=15, fontweight='bold', y=1.01)

for i, feat in enumerate(top6_feats):
    ax = axes[i // 3][i % 3]
    
    # Violin plot with split by class
    data_risk = df[df['at_risk'] == 1][feat].dropna()
    data_safe = df[df['at_risk'] == 0][feat].dropna()
…