AI Risk Metrices

🏦 KEY BANKING RISK METRICS (EXPLAINED SIMPLY)

🔍 What is AUC (in Credit / Risk Models)?

AUC = Area Under the ROC Curve

In simple terms:

AUC measures how well a model can distinguish between good and bad customers (non-default vs default).

🎯 Intuitive Meaning (Very Important)

• AUC = 0.5 → Model is no better than random guessing

• AUC = 1.0 → Perfect separation (never happens in real life)

Typical BFSI Ranges

AUC Range	Interpretation
0.50–0.60	Poor
0.60–0.70	Weak
0.70–0.75	Acceptable
0.75–0.80	Strong
0.80+	Excellent (rare in production)

Most production credit models sit between 0.70–0.78.

📌 What Does AUC Actually Measure?

AUC answers this question:

If you randomly pick one good customer and one bad customer, what is the probability that the model scores the good customer as less risky than the bad one?

Example:

• AUC = 0.75

• Means 75% of the time, the model ranks a good borrower better than a bad one.

This is why regulators like AUC — it’s threshold-independent.

📊 AUC vs Accuracy (Key Interview Point)

Metric	Why / Why Not
Accuracy	Misleading for imbalanced data
Precision / Recall	Threshold dependent
AUC	Stable, comparable, regulator-friendly

👉 That’s why credit risk teams prefer AUC.

📈 What Does “+5–10% AUC Uplift” Mean?

Example

• Baseline AUC = 0.72

+5% uplift

• 0.72 × 1.05 ≈ 0.76

+10% uplift

• 0.72 × 1.10 ≈ 0.79

✅ These numbers are very strong improvements in real credit systems.

🧠 How GenAI Impacts AUC (Safely)

GenAI improves AUC indirectly by:

• Extracting features from unstructured data

• Improving document accuracy (bank statements, income proofs)

• Reducing manual noise and bias

• Enhancing analyst decision support

GenAI does not replace the statistical credit model.

🎤 Perfect Interview Answer (Short & Crisp)

“AUC measures a model’s ability to separate good and bad customers.An AUC of 0.75 means that in 75% of cases, the model ranks a good borrower as lower risk than a bad one.In credit risk, we focus on improving AUC rather than accuracy, and a 5–10% uplift—from say 0.72 to 0.76–0.79—is considered a strong, regulator-acceptable improvement.”

1️⃣ KS Statistic (Kolmogorov–Smirnov)

What it means

KS measures how well the model separates good vs bad customers.

It is the maximum gap between:

• Cumulative % of good customers

• Cumulative % of bad customers

Typical banking range

KS	Interpretation
< 0.25	Weak
0.25 – 0.30	Acceptable
0.30 – 0.45	Good / Strong
> 0.50	Rare / suspicious

Example you can say

“Our KS improved from ~0.32 to ~0.38, showing stronger separation between good and bad borrowers.”

2️⃣ Approval Rate @ Same Risk

What it means

How many more customers you can approve while keeping risk constant.

This is very important to business.

Example

• Baseline model approves 60% at a fixed bad-rate (say 3%)

• New model approves 66–68% at the same bad-rate

👉 Approval rate @ same risk = +6–8%

What this tells leadership

“We’re growing business without increasing risk.”

3️⃣ Bad-Rate Reduction

What it means

Reduction in default rate while approving the same number of customers.

Example

• Baseline bad rate = 3.5%

• New bad rate = 3.0–3.2%

👉 Bad-rate reduction = 0.3–0.5%

Why this is powerful

• Even 0.3% reduction = crores saved at scale

4️⃣ PSI (Population Stability Index)

What it means

PSI measures whether the customer population has shifted compared to training data.

Used to detect model drift.

PSI benchmarks (bank-standard)

PSI	Meaning
< 0.10	Stable
0.10 – 0.25	Monitor
> 0.25	Action required

Example you can say

“Our PSI stayed under 0.1, indicating population stability and no material drift.”

📊 HOW THEY WORK TOGETHER (EXECUTIVE VIEW)

Metric	What It Proves
KS	Model discrimination power
Approval rate @ same risk	Business growth
Bad-rate reduction	Risk control
PSI	Model stability over time

Together, they show value, safety, and sustainability.

🎤 Summary

“We measured risk performance using KS, approval rate at same risk, bad-rate reduction, and PSI.KS improved from ~0.32 to ~0.38, approvals increased 6–8% at the same risk, bad rates reduced ~0.3–0.5%, and PSI stayed below 0.1, indicating a stable and well-governed model.