Lecture 07: Classification Evaluation & Performance Metrics

Source Material: materials/AI-Classification-Evaluation.pdf / materials/AI-Classification-Evaluation.md
Course: Introduction to Data Analytics (CE KMITL)
Topics: Model Evaluation Methodology, 2x2 Confusion Matrix, Accuracy, Precision, Recall, Specificity, F1-Score, Class Imbalance (“We Need FISH”), ROC Curve, Area Under Curve (AUC), Multi-Class Evaluation (Macro, Micro, Weighted)

1. Model Evaluation Philosophy & Methodology

1.1 The Golden Rule of Machine Learning Evaluation

In supervised machine learning, models are trained on historical data to learn underlying patterns between input features $\mathbf{X}$ and target ground-truth labels $y$ . A model must never be evaluated on the same data used to train it. Evaluating on training data measures only memorization (overfitting), not genuine generalization capability on unseen future data.

flowchart TD Data["Full Labeled Dataset"] --> Split{"Train-Test Split"} Split -->|e.g. 70-80%| TrainData["Training Set (X_train, y_train)"] Split -->|e.g. 20-30%| TestData["Test Set (X_test, y_test)"] TrainData --> Model["ML Model Training / Fit"] Model --> Predict["Predict on Unseen X_test\n(y_pred = y_hat)"] TestData -.-> Eval{"Performance Evaluation"} Predict --> Eval Eval --> Metrics["Performance Metrics:\nConfusion Matrix, F1, ROC/AUC"]

1.2 Cross-Validation

While a simple Train-Test split is effective for large datasets, small or moderately sized datasets can suffer from sample selection variance (the evaluation score depends heavily on which specific samples happen to land in the test split).

To ensure robust, unbiased validation:

K-Fold Cross-Validation: The complete dataset is partitioned into $K$ equal-sized folds. The model is trained and evaluated $K$ times, rotating the hold-out validation fold in each round.
The overall model score is computed as the arithmetic mean across all $K$ rounds:

\bar{E} = \frac{1}{K} \sum_{k=1}^K E_k

Cross-Reference: For an in-depth mathematical walkthrough, code implementation, and visual explanation of $K$ -fold cross-validation, refer directly to the dedicated note:
🔗 K-Fold Cross-Validation

2. The 2x2 Confusion Matrix

2.1 Matrix Definition & Quadrants

For binary classification, test results are organized into a $2 \times 2$ Confusion Matrix comparing Actual Ground Truth against Predicted Labels:

	Predicted: Positive (YES)	Predicted: Negative (NO)	Total Actual
Actual: Positive (YES)	True Positive ( $TP$ ) (Correct hit)	False Negative ( $FN$ ) (Type II Error / Miss)	$P = TP + FN$
Actual: Negative (NO)	False Positive ( $FP$ ) (Type I Error / False Alarm)	True Negative ( $TN$ ) (Correct rejection)	$N = FP + TN$
Total Predicted	$TP + FP$	$FN + TN$	Total = $TP + TN + FP + FN$

2.2 Conceptual Meaning: “Don’t Be Emotional”

In machine learning nomenclature:

Positive ( $+$ ) does not mean “good” or desirable in an emotional sense. It designates the target condition or event the model is built to detect (e.g., disease present, fraudulent transaction, defective component, spam email).
Negative ( $-$ ) designates the absence of the target condition or the normal baseline status (e.g., healthy patient, legitimate transaction, functioning part, non-spam).

Accordingly:

True Positive ( $TP$ ): The model predicts Positive, and the condition is actually Positive (Hit).
True Negative ( $TN$ ): The model predicts Negative, and the condition is actually Negative (Correct rejection).
False Positive ( $FP$ ): The model predicts Positive, but the true condition is Negative (Type I Error / False Alarm).
False Negative ( $FN$ ): The model predicts Negative, but the true condition is Positive (Type II Error / Missed detection).

2.3 The 8-Scenario Evaluation Drill

To master classification outcomes, analyze these 8 foundational scenarios:

Scenario #	System Objective	Real-World Event & System Response	Evaluation Outcome	Real-World Consequence
1	Vehicle Detector (detect cars)	A car appears; the model predicts “car”.	$TP$	Correct detection.
2	Vehicle Detector (detect cars)	A bicycle appears; the model predicts “car”.	$FP$	False alarm (misidentification).
3	Vehicle Detector (detect cars)	A car appears; the model fails to select it.	$FN$	Missed target (vehicle passed undetected).
4	Vehicle Detector (detect cars)	A bicycle appears; the model ignores it.	$TN$	Correct rejection of non-target.
5	Fire Alarm (detect fire)	No fire exists, but the alarm sounds.	$FP$	False alarm (nuisance, unnecessary evacuation).
6	Burglar Alarm (detect thieves)	A burglar enters, but the system remains silent.	$FN$	Critical security failure (burglary occurs).
7	Fire Alarm (detect fire)	Normal condition, alarm remains silent.	$TN$	Correct normal operation.
8	Burglar Alarm (detect thieves)	A family member enters, but alarm sounds.	$FP$	False alarm (annoyance to homeowner).

Asymmetry of Costs: In real-world engineering, the cost of $FP$ vs. $FN$ is rarely symmetric. In medical screening or intrusion detection, a False Negative ( $FN$ ) can be fatal or catastrophic, whereas a False Positive ( $FP$ ) merely triggers a secondary confirmation test.

3. Performance Metrics Formulas

3.1 Accuracy

The overall proportion of correctly classified instances across all classes:

\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}

3.2 Error Rate (Misclassification Rate)

The overall proportion of incorrect classifications:

\text{Error Rate} = \frac{FP + FN}{TP + TN + FP + FN} = 1 - \text{Accuracy}

3.3 Precision (Positive Predictive Value - PPV)

Measures the reliability of positive predictions. Out of all instances the model labeled as Positive, how many are truly Positive?

\text{Precision} = \frac{TP}{TP + FP}

High Precision is critical when the cost of a False Positive is high (e.g., spam filtering where legitimate emails must not end up in Junk, or automated high-stakes criminal sentencing).

3.4 Recall / Sensitivity / True Positive Rate (TPR)

Measures the completeness of positive detection. Out of all actual Positive instances in reality, how many did the model successfully capture?

\text{Recall} = \text{Sensitivity} = \text{TPR} = \frac{TP}{TP + FN}

High Recall is critical when the cost of a False Negative is high (e.g., cancer detection, Covid-19 screening, fraud prevention, safety-critical alarms).

3.5 Specificity / True Negative Rate (TNR)

Measures the ability to correctly reject negative cases. Out of all actual Negative instances, how many did the model correctly identify as Negative?

\text{Specificity} = \text{TNR} = \frac{TN}{TN + FP}

3.6 False Positive Rate (FPR) / Fall-out

The proportion of actual negative instances that were incorrectly flagged as positive:

\text{FPR} = \frac{FP}{FP + TN} = 1 - \text{Specificity}

3.7 $F_1$ -Score ( $F$ -Measure)

The harmonic mean of Precision and Recall, providing a single balanced metric:

\frac{1}{F_1} = \frac{1}{2} \left( \frac{1}{\text{Precision}} + \frac{1}{\text{Recall}} \right)

Solving for $F_1$ :

F_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}} = \frac{2 TP}{2 TP + FP + FN}

Why Harmonic Mean Instead of Arithmetic Mean?

The arithmetic mean $\frac{\text{Precision} + \text{Recall}}{2}$ would allow a model with $\text{Precision} = 1.0$ and $\text{Recall} = 0.0$ to score an apparently acceptable $0.50$ ( $50\%$ ). In contrast, the harmonic mean strongly penalizes extreme imbalances: if either Precision or Recall collapses towards $0$ , $F_1$ drops sharply towards $0$ .

4. Class Imbalance & The “We Need FISH” Intuition

4.1 The Accuracy Paradox

When classes are severely imbalanced (e.g., $99\%$ negative and $1\%$ positive), Accuracy becomes a completely misleading metric.

4.2 The “We Need FISH” Narrative

Consider an automated fishing vessel dragging a massive net through the ocean:

The ocean contains vast amounts of seawater, seaweed, rocks, and debris (Class: Not-Fish / Negative).
Scattered inside the net are very rare valuable tuna (Class: Fish / Positive).
The objective: We need fish!

flowchart LR Haul["100 Total Items in Net"] --> Fish["3 Actual FISH (Positive)"] Haul --> Debris["97 Debris / Water (Negative)"] Fish --> ModelCatch["Model Catches 0 Fish (TP=0, FN=3)"] Debris --> ModelReject["Model Rejects 96 Debris (TN=96, FP=1)"] ModelCatch --> Result["Accuracy = (0 + 96) / 100 = 96%!\nRecall = 0.0%, F1 = 0.0!\nSystem is completely USELESS!"] ModelReject --> Result

Slide Scenario Comparison:

Dataset: $N = 100$ items ( $3$ Fish, $97$ Non-Fish).
Dummy Classifier: A trivial model that always predicts “Not-Fish” no matter what:
- $TP = 0, FN = 3, FP = 0, TN = 97$ .
- $\text{Accuracy} = \frac{0 + 97}{100} = 97.0\%$ .
- The model caught zero fish, yet boasts a stellar $97\%$ accuracy rating!
Model in Slide 57: $TP = 1, FN = 2, FP = 1, TN = 96$ $T P = 1, F N = 2, F P = 1, T N = 96$ .
- $\text{Accuracy} = \frac{1 + 96}{100} = 97.0\%$ .
- $\text{Recall} = \frac{1}{1 + 2} = \frac{1}{3} \approx 33.3\%$ . Missed $66.7\%$ of all fish!
Model in Slide 75: $TP = 0, FN = 3, FP = 1, TN = 96$ $T P = 0, F N = 3, F P = 1, T N = 96$ .
- $\text{Accuracy} = \frac{0 + 96}{100} = 96.0\%$ .
- $\text{Precision} = 0.0$ , $\text{Recall} = 0.0$ , $F_1 = 0.0$ .
- Caught zero fish, but Accuracy is still $96\%$ .

Takeaway Rule:

For balanced binary classes $\implies$ Accuracy is acceptable.

For imbalanced classes $\implies$ Accuracy must be abandoned in favor of Precision, Recall, and $F_1$ -Score.

5. Consolidated 5-Exercise Comparison Table

The lecture presents 5 progressive exercise scenarios (each with 10 total test samples: 5 Actual Positives, 5 Actual Negatives for Ex 1–3 and 5; Ex 4 has 5 and 5). The table below consolidates all metrics across the 5 exercises:

Exercise	Confusion Matrix $\begin{bmatrix} TP & FN \\ FP & TN \end{bmatrix}$	Accuracy	Precision	Recall (TPR)	Specificity (TNR)	FPR	$F_1$ -Score	Diagnostic Takeaway
Ex. 1	$\begin{bmatrix} 5 & 0 \\ 0 & 5 \end{bmatrix}$	$\frac{5+5}{10} = \mathbf{1.00}$	$\frac{5}{5+0} = \mathbf{1.00}$	$\frac{5}{5+0} = \mathbf{1.00}$	$\frac{5}{5+0} = \mathbf{1.00}$	$\frac{0}{0+5} = \mathbf{0.00}$	$\mathbf{1.00}$	Ideal Perfect Classifier: Zero classification errors ( $FP=0, FN=0$ ).
Ex. 2	$\begin{bmatrix} 3 & 2 \\ 0 & 5 \end{bmatrix}$	$\frac{3+5}{10} = \mathbf{0.80}$	$\frac{3}{3+0} = \mathbf{1.00}$	$\frac{3}{3+2} = \mathbf{0.60}$	$\frac{5}{5+0} = \mathbf{1.00}$	$\frac{0}{0+5} = \mathbf{0.00}$	$\mathbf{0.75}$	Conservative Model: Zero false alarms ( $\text{Precision}=1.0$ ), but misses 2 positive targets ( $\text{Recall}=0.60$ ).
Ex. 3	$\begin{bmatrix} 5 & 0 \\ 1 & 4 \end{bmatrix}$	$\frac{5+4}{10} = \mathbf{0.90}$	$\frac{5}{5+1} \approx \mathbf{0.833}$	$\frac{5}{5+0} = \mathbf{1.00}$	$\frac{4}{4+1} = \mathbf{0.80}$	$\frac{1}{1+4} = \mathbf{0.20}$	$\mathbf{0.909}$	Aggressive Model: Catches 100% of positive targets ( $\text{Recall}=1.0$ ), but generates 1 false alarm ( $\text{Precision}=0.833$ ).
Ex. 4	$\begin{bmatrix} 0 & 5 \\ 3 & 2 \end{bmatrix}$	$\frac{0+2}{10} = \mathbf{0.20}$	$\frac{0}{0+3} = \mathbf{0.00}$	$\frac{0}{0+5} = \mathbf{0.00}$	$\frac{2}{2+3} = \mathbf{0.40}$	$\frac{3}{3+2} = \mathbf{0.60}$	$\mathbf{0.00}$	Inverted / Broken Model: Complete failure to detect positive targets ( $TP=0$ ). Worse than random guessing.
Ex. 5	$\begin{bmatrix} 3 & 2 \\ 1 & 4 \end{bmatrix}$	$\frac{3+4}{10} = \mathbf{0.70}$	$\frac{3}{3+1} = \mathbf{0.75}$	$\frac{3}{3+2} = \mathbf{0.60}$	$\frac{4}{4+1} = \mathbf{0.80}$	$\frac{1}{1+4} = \mathbf{0.20}$	$\mathbf{0.667}$	Realistic Trade-off: Balanced trade-off with both moderate false alarms ( $FP=1$ ) and missed detections ( $FN=2$ ).

6. Receiver Operating Characteristic (ROC) & Area Under Curve (AUC)

6.1 Decision Thresholds and Probabilistic Classifiers

Most modern classification algorithms (such as Logistic Regression, Naive Bayes, Neural Networks) do not output a hard binary decision directly; they output a continuous predicted probability score:

P(y = 1 \mid \mathbf{x}) \in [0, 1]

To classify an instance as Positive or Negative, we compare this probability against a decision threshold $\tau \in [0, 1]$ :

If $P(y = 1 \mid \mathbf{x}) \ge \tau \implies \text{Predict Positive}$
If $P(y = 1 \mid \mathbf{x}) < \tau \implies \text{Predict Negative}$

Varying $\tau$ from $1.0$ down to $0.0$ sweeps the model from extremely conservative to extremely aggressive.

6.2 Consolidated Threshold Sweep Table (Rounds 0–10)

The 11 animation slides (Slides 83–93) in the source deck illustrate a step-by-step decision threshold sweep on a test set of 10 samples (5 actual positive, 5 actual negative). The table below consolidates the full sweep into a single unified reference:

Round	Threshold Stance	$TP$	$FN$	$FP$	$TN$	$\text{TPR} = \frac{TP}{TP+FN}$	$\text{FPR} = \frac{FP}{FP+TN}$	ROC Point Coordinate $(FPR, TPR)$	Interpretation
0	$\tau \to \infty$ (Predicts all Negative)	$0$	$5$	$0$	$5$	$\frac{0}{0+5} = \mathbf{0.0}$	$\frac{0}{0+5} = \mathbf{0.0}$	$(0.0, 0.0)$	Origin: Zero false alarms, but zero positive detections.
1	$\tau$ decreases (Top 1 sample accepted)	$1$	$4$	$0$	$5$	$\frac{1}{1+4} = \mathbf{0.2}$	$\frac{0}{0+5} = \mathbf{0.0}$	$(0.0, 0.2)$	First true positive captured with zero false alarms.
2	$\tau$ decreases (Top 2 samples accepted)	$2$	$3$	$0$	$5$	$\frac{2}{2+3} = \mathbf{0.4}$	$\frac{0}{0+5} = \mathbf{0.0}$	$(0.0, 0.4)$	Moving vertically upward along y-axis.
3	$\tau$ decreases (Top 3 samples accepted)	$3$	$2$	$0$	$5$	$\frac{3}{3+2} = \mathbf{0.6}$	$\frac{0}{0+5} = \mathbf{0.0}$	$(0.0, 0.6)$	Perfect selectivity continues.
4	$\tau$ decreases (Top 4 samples accepted)	$4$	$1$	$0$	$5$	$\frac{4}{4+1} = \mathbf{0.8}$	$\frac{0}{0+5} = \mathbf{0.0}$	$(0.0, 0.8)$	80% recall with 0% false alarms.
5	$\tau$ decreases (First negative misclassified)	$4$	$1$	$1$	$4$	$\frac{4}{4+1} = \mathbf{0.8}$	$\frac{1}{1+4} = \mathbf{0.2}$	$(0.2, 0.8)$	Curve steps rightward as first $FP$ occurs.
6	$\tau$ decreases (5th positive captured)	$5$	$0$	$1$	$4$	$\frac{5}{5+0} = \mathbf{1.0}$	$\frac{1}{1+4} = \mathbf{0.2}$	$(0.2, 1.0)$	100% recall achieved!
7	$\tau$ decreases (Second negative misclassified)	$5$	$0$	$2$	$3$	$\frac{5}{5+0} = \mathbf{1.0}$	$\frac{2}{2+3} = \mathbf{0.4}$	$(0.4, 1.0)$	Steps rightward along top horizontal edge.
8	$\tau$ decreases (Third negative misclassified)	$5$	$0$	$3$	$2$	$\frac{5}{5+0} = \mathbf{1.0}$	$\frac{3}{3+2} = \mathbf{0.6}$	$(0.6, 1.0)$	Steps rightward along top horizontal edge.
9	$\tau$ decreases (Fourth negative misclassified)	$5$	$0$	$4$	$1$	$\frac{5}{5+0} = \mathbf{1.0}$	$\frac{4}{4+1} = \mathbf{0.8}$	$(0.8, 1.0)$	Steps rightward along top horizontal edge.
10	$\tau = 0.0$ (Predicts all Positive)	$5$	$0$	$5$	$0$	$\frac{5}{5+0} = \mathbf{1.0}$	$\frac{5}{5+0} = \mathbf{1.0}$	$(1.0, 1.0)$	Top-right corner: 100% detection, but 100% false alarms.

6.3 ROC Curve Interpretation

A Receiver Operating Characteristic (ROC) Curve plots the True Positive Rate (TPR / Sensitivity) on the y-axis against the False Positive Rate (FPR / 1 - Specificity) on the x-axis across all possible decision thresholds $\tau \in [0, 1]$ .

flowchart TD subgraph ROC_Analysis["ROC Curve Anatomy"] Origin["(0,0): Predict All Negative\n(TPR=0, FPR=0)"] TopLeft["(0,1): Ideal Perfect Classifier\n(TPR=1.0, FPR=0.0)"] TopRight["(1,1): Predict All Positive\n(TPR=1.0, FPR=1.0)"] Diagonal["Diagonal Line y = x (AUC = 0.5)\nRandom Guessing Baseline"] TopLeft -.-> Curve["Realistic ROC Curve (AUC = 0.85 - 0.95)"] end

Key reference points on the ROC space:

$(0, 0)$ : High threshold ( $\tau = 1.0$ ). Classifies nothing as positive. $TPR = 0, FPR = 0$ .
$(1, 1)$ : Low threshold ( $\tau = 0.0$ ). Classifies everything as positive. $TPR = 1, FPR = 1$ .
$(0, 1)$ : The ideal theoretical point (Top-Left Corner). 100% True Positive Rate with 0% False Positive Rate.
Diagonal line ( $y = x$ ): Represents an uninformative classifier that makes random coin-flip predictions.

6.4 Area Under the ROC Curve (AUC)

The Area Under the Curve (AUC) calculates the two-dimensional area beneath the entire ROC curve from $(0,0)$ to $(1,1)$ .

Range: $0.0 \le \text{AUC} \le 1.0$ .
Probabilistic Meaning: AUC equals the probability that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative instance.

Traditional Diagnostic Test Rating Scale:

AUC Range	Academic Grade	Diagnostic Quality
$0.90 - 1.00$	A	Excellent: Near-perfect discrimination between classes.
$0.80 - 0.90$	B	Good: Strong clinical or operational utility.
$0.70 - 0.80$	C	Fair: Acceptable discrimination, but noticeable error rate.
$0.60 - 0.70$	D	Poor: Weak predictive power; barely outperforms baseline.
$0.50 - 0.60$	F	Fail / Worthless: Indistinguishable from random guessing ( $\text{AUC} = 0.50$ ).

7. Multi-Class Evaluation

7.1 Multi-Class Confusion Matrix ( $K \times K$ )

When dealing with $K > 2$ classes (e.g., classifying images into Cat, Dog, or Bird), the confusion matrix expands to a $K \times K$ table:

	Predicted: Cat	Predicted: Dog	Predicted: Bird	Actual Total
Actual: Cat	$C_{11}$ ( $TP_{\text{Cat}}$ )	$C_{12}$	$C_{13}$	$N_{\text{Cat}}$
Actual: Dog	$C_{21}$	$C_{22}$ ( $TP_{\text{Dog}}$ )	$C_{23}$	$N_{\text{Dog}}$
Actual: Bird	$C_{31}$	$C_{32}$	$C_{33}$ ( $TP_{\text{Bird}}$ )	$N_{\text{Bird}}$
Predicted Total	$\sum_i C_{i1}$	$\sum_i C_{i2}$	$\sum_i C_{i3}$	$N_{\text{total}}$

Diagonal Elements ( $C_{ii}$ ): Correct predictions ( $TP$ for each specific class $i$ ).
Off-Diagonal Elements ( $C_{ij}, i \ne j$ ): Classification errors and confusions.

7.2 One-vs-Rest (OvR) Decomposition for Class $k$

To compute metrics for individual class $k$ :

$TP_k = C_{kk}$ (diagonal element)
$FP_k = \sum_{i \ne k} C_{ik}$ (column sum minus diagonal)
$FN_k = \sum_{j \ne k} C_{kj}$ (row sum minus diagonal)
$TN_k = \sum_{i \ne k} \sum_{j \ne k} C_{ij}$ (sum of all entries outside row $k$ and column $k$ )

7.3 Multi-Class Averaging Strategies

To summarize class-wise performance into a single global metric:

1. Macro-Averaging

Computes the metric independently for each class and calculates the unweighted arithmetic mean:

\text{Precision}_{\text{macro}} = \frac{1}{K} \sum_{k=1}^K \text{Precision}_k

\text{Recall}_{\text{macro}} = \frac{1}{K} \sum_{k=1}^K \text{Recall}_k

Properties: Treats all classes equally regardless of support size. Emphasizes performance on rare minority classes.

2. Micro-Averaging

Aggregates the individual contributions of all $TP, FP, FN$ across all classes first, then computes the metric:

\text{Precision}_{\text{micro}} = \frac{\sum_{k=1}^K TP_k}{\sum_{k=1}^K (TP_k + FP_k)}

\text{Recall}_{\text{micro}} = \frac{\sum_{k=1}^K TP_k}{\sum_{k=1}^K (TP_k + FN_k)}

Properties: In single-label multi-class classification, $\text{Micro-Accuracy} = \text{Micro-Precision} = \text{Micro-Recall} = \text{Micro-}F_1$ .

3. Weighted-Averaging

Weights each class metric by its true support $N_k$ (the number of actual instances in that class):

\text{Metric}_{\text{weighted}} = \sum_{k=1}^K \frac{N_k}{N_{\text{total}}} \text{Metric}_k

Properties: Balances global accuracy while accounting for class prevalence.

8. Summary & Practical Metric Selection Guide

flowchart TD Start["What is your evaluation goal?"] --> Q1{"Are classes balanced?"} Q1 -->|Yes, balanced| BalancedAcc["Use Accuracy\n(Simple, intuitive overall measure)"] Q1 -->|No, imbalanced| Q2{"Which error is more costly?"} Q2 -->|False Positives are costly\n(e.g. spam filter, fraud accusation)| Prec["Prioritize Precision\nMaximize TP / (TP + FP)"] Q2 -->|False Negatives are costly\n(e.g. disease, intrusion, fire alarm)| Rec["Prioritize Recall\nMaximize TP / (TP + FN)"] Q2 -->|Both errors matter equally| F1["Use F1-Score\nHarmonic mean of Precision and Recall"] Start --> Q3{"Evaluating full threshold spectrum?"} Q3 -->|Yes, model comparison| ROC["Use ROC Curve & AUC\nEvaluates probabilistic ranking quality"]

Strategic Checklist for Practitioners:

Always build a Confusion Matrix: For true transparency, present the raw confusion matrix rather than a single aggregated scalar.
Never rely on Accuracy alone on imbalanced data: As demonstrated by the “We Need FISH” principle, high accuracy can conceal total detection failure.
Select metrics based on operational risk: Tailor threshold $\tau$ to minimize whichever error carries catastrophic business or safety costs.
Report ROC/AUC for model comparison: AUC evaluates general ranking power across all operational thresholds simultaneously.

Module Material: 07. Classification Evaluation & Metrics