Paper - Review

DOI: 10.1038/nmeth.3945

❓ how to evaluate classifier performance?
→ using 1⃣ a numeric metric 2⃣ a graphical representation of performance

∴ Examine some common classifier metrics & Discuss the pitfalls of relying on a single metrics

❗ Metrics
→ how a classifier performs
→ critical for evaluating reports

Simulate → a hypothetical diagnostic test
→ on the basis of multiple clinical factors

❗ Confusion matrix
Relevance (← of each of these 4 quantities)
← depend on 1⃣ the purpose of the classifier 2⃣ motivate the choice of metric

High accuracy
→ does NOT ❌ necessarily characterize → a good classifier

Predicting will be negative offers high accuracy
← if the disease is rare
→ is NOT ❌ useful for diagnosis

NO ❌ single metric
→ can distinguish → all (the strengths & weaknesses) ← of a classifier

Important factor
← in interpreting classification results
→ is (class balance)

Imbalance makes
→ understanding (FPs & FNs) → more important
→ Extra care is required → to interpret the results

To evaluate the classifier
→ without having to select ← a specific threshold
∴ Can be visualized → using the ROC curve

ROC curve
→ A good classifier should aim → to reach as close to the top-left corner as possible

Class imbalance can cause ROC curves
→ to be poor visualizations of classifier performance

---