Case Study: Production Bug

The Hook (The "Byte-Sized" Intro)

Monday morning: error rates are 10x normal. Payments are failing. The on-call engineer's phone is buzzing. No one knows what changed. This case study walks through the entire incident — from alert to root cause — using the Git tools you've learned: git log, git diff, git bisect, git blame, and git revert.

📖 Case Study: Tracking Down a Production Bug

A fictional but realistic scenario demonstrating how Git tools work together during an incident.

Conceptual Clarity

The scenario:

Fact	Detail
Symptom	Payment processing fails with 500 errors
When noticed	Monday 8am
Last known good	Friday 5pm
Deploys since then	3 PRs merged over the weekend
Goal	Find the breaking commit, fix it, prevent recurrence

The investigation timeline:

Step	Tool Used	Finding
1. What changed?	`git log`	3 PRs merged since Friday
2. What files changed?	`git diff`	8 files across 3 PRs
3. Narrow to suspect	`git bisect`	Commit `abc123` breaks payments
4. Who wrote it?	`git blame`	Developer X, line 42
5. What's the fix?	code review	Missing null check
6. Immediate fix	`git revert`	Revert the commit
7. Proper fix	hotfix PR	Add null check + test

Real-Life Analogy

This case study follows the detective methodology: what happened (symptoms), when did it start (timeline), who touched it (blame), narrow the suspects (bisect), and catch the culprit (root cause).

Visual Architecture

flowchart TD ALERT["🚨 Payments failing"] --> LOG["git log 3 recent PRs"] LOG --> DIFF["git diff 8 changed files"] DIFF --> BISECT["git bisect Commit abc123"] BISECT --> BLAME["git blame Line 42: null issue"] BLAME --> REVERT["git revert Restore service"] REVERT --> HOTFIX["🩹 Hotfix PR Null check + test"] style ALERT fill:#2d1b1b,stroke:#e94560,color:#e94560 style HOTFIX fill:#1b2d1b,stroke:#53d8fb,color:#53d8fb

The Full Walkthrough

bash

# ─── Step 1: What changed since Friday? ───
git log --oneline --since="3 days ago" main
# abc123 feat(payments): optimize card processing
# def456 docs: update API docs
# ghi789 chore: upgrade lodash

# ─── Step 2: What files changed in the suspect PR? ───
git diff abc123^..abc123 --stat
# src/payments/process.js | 25 +++-
# src/payments/validate.js | 12 +++-

# ─── Step 3: Bisect to find the exact commit ───
git bisect start
git bisect bad HEAD          # Current state is broken
git bisect good abc123^      # Commit before abc123 was fine
# Git checks out a commit for you to test
npm test -- --grep "payments"
git bisect good              # or git bisect bad
# ...repeat until...
# abc123 is the first bad commit!
git bisect reset

# ─── Step 4: Who wrote the breaking line? ───
git blame src/payments/process.js
# abc123 (Dev X 2026-02-10) if (card.isActive) {
# The card object can be null for expired subscriptions!

# ─── Step 5: Immediate fix — revert ───
git revert abc123 --no-edit
git push origin main
# Service restored!

# ─── Step 6: Proper fix ───
git checkout -b hotfix/null-card-check
# Add: if (card && card.isActive) { ... }
# Add test: "should handle null card gracefully"
git commit -m "fix(payments): handle null card object

Fixes PROD-501. The card optimization (abc123) assumed card
is always present, but expired subscriptions have null cards."

Key Takeaways

git log --since + git diff narrow the window of change.
git bisect efficiently finds the exact breaking commit.
git blame identifies the specific line and author.
Revert first to restore service, then create a proper hotfix.

Interview Prep

Q: Walk me through how you'd debug a production issue using Git. A: 1) git log --since to see recent changes. 2) git diff to see what files changed. 3) git bisect to find the exact failing commit. 4) git blame to identify the line. 5) git revert to restore service immediately. 6) Create a hotfix PR with the proper fix and tests.
Q: How does git bisect help in production debugging? A: It performs a binary search through commit history. You mark a known-good commit and the current bad state. Git checks out the midpoint for testing. In ~7 steps, it finds the exact breaking commit among 100+ commits.
Q: After reverting a broken commit, what should you do next? A: Create a hotfix branch, write the proper fix with a test that catches the original bug, get it reviewed, and merge. The revert was the emergency measure; the hotfix is the permanent solution.