Phishing Website Detection: 3-Model Comparison

# 🛡️ Phishing Website Detection — 3-Model Comparison ## Business Context Phishing websites trick users into revealing sensitive information. An automated ML-based detector can flag suspicious sites in real time. **False positives are costly** — blocking a legitimate bank login is worse than letting one phish through — so we optimise for **low false-positive rate** first, accuracy second. ## Dataset **UCI Phishing Websites** (30 features, 11,055 instances). Features span: - **URL structure** — IP address, URL length, `@` symbol, prefix/suffix, subdomains - **Domain properties** — SSL state, registration length, DNS record, domain age - **HTML/JS behaviour** — favicon, right-click disable, popup windows, iframe - **External reputation** — page rank, web traffic, Google index, statistical report ## Workflow | Step | Description | |------|-------------| | 1. Load | Download UCI dataset from Kaggle | | 2. EDA | Visualise feature distributions & class balance | | 3. Preprocess | Split (70/15/15), scale features | | 4. **Train 3 Models** | **Logistic Regression · Random Forest · XGBoost** (parallel branches) | | 5. **Converge** | Compare accuracy, FPR, ROC-AUC; pick winner | | 6. Explain | Example predictions with Low / Medium / High risk + reasons |

# ═══════════════════════════════════════════════════
# 📦 Load UCI Phishing Websites Dataset
# ═══════════════════════════════════════════════════

import subprocess, sys, importlib, pkgutil

# Install kagglehub if missing
if not pkgutil.find_loader("kagglehub"):
    print("Installing kagglehub...")
    subprocess.check_call([sys.executable, "-m", "pip", "install", "-q", "kagglehub"])

import kagglehub
import pandas as pd
import numpy as np
import warnings
warnings.filterwarnings('ignore')

# Download dataset from Kaggle
…

Installing kagglehub...
Downloading UCI Phishing Websites dataset from Kaggle...
Downloading to /root/.cache/kagglehub/datasets/mmelgizin/phishing-websites-dataset/1.archive...
Extracting files...
Downloaded to: /root/.cache/kagglehub/datasets/mmelgizin/phishing-websites-dataset/versions/1

Shape: 11055 rows × 32 columns

First 5 rows:

Column names:
['id', 'having_IP_Address', 'URL_Length', 'Shortining_Service', 'having_At_Symbol', 'double_slash_redirecting', 'Prefix_Suffix', 'having_Sub_Domain…

# ═══════════════════════════════════════════════════
# 🔍 Exploratory Data Analysis
# ═══════════════════════════════════════════════════

import matplotlib.pyplot as plt
import seaborn as sns

# Drop the 'id' column (just an index)
df = df.drop(columns=['id'])
print(f"Feature count: {df.shape[1] - 1} features + target")

# ── Target distribution ──
target_counts = df['Result'].value_counts().sort_index()
print(f"\nTarget values: {dict(target_counts)}")

plt.figure(figsize=(6, 4))
colors = ['#e74c3c', '#2ecc71']
ax = sns.countplot(data=df, x='Result', palette=colors)
…

Feature count: 30 features + target

Target values: {-1: np.int64(4898), 1: np.int64(6157)}

Top 10 features most correlated (abs) with Result (target):
SSLfinal_State                 0.714741
URL_of_Anchor                  0.692935
Prefix_Suffix                  0.348606
web_traffic                    0.346103
having_Sub_Domain              0.298323
Request_URL                    0.253372
Links_in_tags                  0.248229
Domain_registeration_length   -0.225789
SFH…

# ═══════════════════════════════════════════════════
# ⚙️ Preprocessing — Train/Val/Test Split + Scaling
# ═══════════════════════════════════════════════════

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# ── Encode target: -1 (phishing) → 1, 1 (legitimate) → 0
# So the positive class = phishing (what we want to detect)
y = (df['Result'] == -1).astype(int).values  # 1 = phishing, 0 = legitimate
X = df.drop(columns=['Result']).values

feature_names = [c for c in df.columns if c != 'Result']

print(f"X shape: {X.shape}")
print(f"y: {y.sum()} phishing ({y.sum()/len(y):.1%}), {(len(y)-y.sum())} legitimate ({(len(y)-y.sum())/len(y):.1%})")

# ── Train / validation / test split (70/15/15) ──
…

X shape: (11055, 30)
y: 4898 phishing (44.3%), 6157 legitimate (55.7%)

Train:   7738 samples (3428 phishing)
Val:     1658 samples (735 phishing)
Test:    1659 samples (735 phishing)

After scaling — mean ~0, std ~1 per feature
Train: (7738, 30), Val: (1658, 30), Test: (1659, 30)

✅ Preprocessing complete. Ready for model training.

# ═══════════════════════════════════════════════════
# Logistic Regression (retry)
# ═══════════════════════════════════════════════════

import json, numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, roc_auc_score, confusion_matrix

MODEL_LABEL = 'Logistic Regression'
VARIANT_ID = 'logistic_regression'

# Check data exists
print(f"X_train_scaled shape: {X_train_scaled.shape}")

model = LogisticRegression(C=1.0, max_iter=1000, random_state=42, class_weight='balanced')

# 5-fold CV
…

X_train_scaled shape: (7738, 30)
Logistic Regression — CV Accuracy: 0.9248 ± 0.0049

Logistic Regression — Test Set
  Accuracy:      0.9373
  Precision:     0.9340
  Recall:        0.9238
  F1:            0.9289
  ROC-AUC:       0.9810
  False Pos Rate:0.0519
  Confusion Mat:[[876 48]
                 [56 679]]
  (FP = 48 legitimate sites flagged as phishing)

Top 5 features by |coefficient|:
  URL_of_Anchor: -2.2576
  Prefix_Suffix: -2.1468
  SSLfinal_State: -1.4925
  SFH: -0.7292
  having_IP_A…

# ═══════════════════════════════════════════════════
# 📊 Model Comparison — Converge & Select Winner
# ═══════════════════════════════════════════════════

import json, os, numpy as np
import pandas as pd
from IPython.display import display, HTML

# ── Load all metrics ──
variant_ids = ['logistic_regression', 'random_forest', 'xgboost']
metrics_list = []
for vid in variant_ids:
    path = f'/home/user/metrics_{vid}.json'
    if os.path.exists(path):
        with open(path) as f:
            metrics_list.append(json.load(f))

# ── Build comparison table ──
…

==========================================================================================
              🔍 MODEL COMPARISON — PHISHING DETECTION
==========================================================================================

==========================================================================================
🏆 WINNER: XGBoost
   Accuracy:       0.9693
   False Pos Rate: 0.0195  (32 false positives on 1659 test samples)
   ROC-AUC:        0.9966
   F1:             0.9649
   Re…

# ═══════════════════════════════════════════════════
# 📉 Confusion Matrices & ROC Curves (All 3 Models)
# ═══════════════════════════════════════════════════

import json, numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import roc_curve, auc, confusion_matrix, ConfusionMatrixDisplay
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from xgboost import XGBClassifier

# ── Retrain all 3 models on main kernel (data already in memory) ──
models = {
    'Logistic Regression': LogisticRegression(C=1.0, max_iter=1000, random_state=42, class_weight='balanced'),
    'Random Forest': RandomForestClassifier(n_estimators=100, max_depth=10, min_samples_leaf=4,
                                             random_state=42, class_weight='balanced', n_jobs=-1),
    'XGBoost': XGBClassifier(n_estimators=100, max_depth=6, learning_rate=0.1,
                              subsample=0.8, colsample_bytree=0.8,
…