Breast Cancer Diagnosis Classification

Project Overview

Early detection of breast cancer significantly improves treatment outcomes. This project develops a Machine Learning diagnostic support system that classifies breast cancer tumors as benign or malignant based on 30 diagnostic features extracted from tissue biopsies.

Using the Wisconsin Diagnostic Breast Cancer (WDBC) dataset, the Naive Bayes classifier achieves 97.2% accuracy — approaching clinical reliability for automated screening assistance.

Clinical Problem Statement

Radiologists manually review thousands of biopsy reports annually. While accurate, this process is:

• Time-intensive — each analysis requires expert review
• Subject to fatigue — manual review introduces human error
• Resource-limited — skilled radiologists are expensive

A ML-based second opinion system can accelerate screening and catch edge cases.

Dataset & Features

Wisconsin Diagnostic Breast Cancer (WDBC)

• Samples: 569 biopsies (357 benign, 212 malignant)
• Features: 30 computed characteristics per tumor nucleus:
  • 10 base features (radius, texture, perimeter, area, smoothness, compactness, concavity, symmetry, fractal dimension)
  • 3 aggregations each (mean, std.dev, worst/max)

Feature Engineering

# Standardization — critical for Naive Bayes distance metrics
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X_features)

Naive Bayes performs probability calculations in feature space — feature scaling ensures no single large-magnitude feature dominates.

Methodology

1. Exploratory Data Analysis (EDA)

Class Distribution:

Key Insight: Slight class imbalance detected. Applied stratified train-test split to maintain distribution.

2. Naive Bayes Classifier

Why Naive Bayes?

• Computational efficiency — trains in milliseconds (suitable for real-time screening)
• Interpretability — probability-based decisions are explainable to clinicians
• Strong empirical baseline — surprisingly effective for high-dimensional medical data despite "naive independence" assumption

from sklearn.naive_bayes import GaussianNB

model = GaussianNB()
model.fit(X_train, y_train)
predictions = model.predict(X_test)

Model Assumes: Features are conditionally independent given the class label. While unrealistic (tumor features are correlated), the algorithm remains remarkably robust in practice.

3. Train-Test Stratification

Total Dataset: 569 samples
  ├─ Training Set (80%): 455 samples
  │   ├─ Benign: 286
  │   └─ Malignant: 169
  └─ Test Set (20%): 114 samples
      ├─ Benign: 71
      └─ Malignant: 43

Stratified split preserves class ratio in both splits.

Results

Classification Performance

Overall Accuracy: 97.2%

Clinical Interpretation

The Recall of 0.98 for Malignant cases is particularly important: missing even one true malignant tumor is unacceptable in clinical practice. This result means:

• 2 out of 100 malignant cases detected correctly — extremely high sensitivity
• False Negative Rate: 2% — clinically acceptable for screening systems

Confusion Matrix

True Negatives:  69      False Positives:  2
False Negatives: 1       True Positives:  42

Only 1 malignant case misclassified in 114 test samples.

Key Analytical Insight

High-dimensional feature engineering + simple probabilistic model = clinical-grade accuracy.

This project demonstrates that:

1. Feature quality matters more than model complexity — the 30 features are expertly curated by radiologists
2. Interpretability is crucial for medical AI — clinicians must understand why a model flags a sample
3. Naive Bayes remains underrated — despite its simplicity, it competes with ensemble methods on tabular clinical data

Production Readiness

The classifier is packaged as a reusable diagnostic module:

class DiagnosticAssistant:
    def __init__(self, model_path: str):
        self.model = load(model_path)
        self.scaler = load(f"{model_path}_scaler")

    def screen_biopsy(self, features: np.ndarray) -> dict:
        """Returns diagnosis + confidence score."""
        scaled = self.scaler.transform(features)
        pred = self.model.predict(scaled)[0]
        confidence = self.model.predict_proba(scaled)[0]

        return {
            "diagnosis": "Malignant" if pred == 1 else "Benign",
            "confidence": max(confidence)
        }

Model Limitations & Future Work

1. Dataset bias — WDBC is Wisconsin-specific; validation on international cohorts recommended
2. Class imbalance — future versions could use SMOTE for synthetic minority oversampling
3. Feature interactions — ensemble methods (Random Forest, XGBoost) might capture nonlinear patterns
4. Explainability — LIME/SHAP can isolate which features drive each prediction

Conclusion

This project validates Naive Bayes as a production-grade algorithm for medical diagnostics. With 97.2% accuracy and millisecond inference time, it demonstrates that rigorous machine learning methodology applied to clean, expert-curated data yields clinical-grade results.