Discussions: Google Notebook LM & Amazon's Hiring
In-Class Case: Amazon’s Hiring Algorithm
Video:
https://www.youtube.com/watch?v=V2gfNKxpVmg
Article:
https://www.reuters.com/article/world/insight-amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK0AG/
Case (3 minutes)
Amazon developed a machine learning model to screen job candidates.
· Trained on 10 years of past resumes
· Used historical hiring decisions as the label (hire vs. reject)
· Goal: predict which candidates should be hired
Important: Gender was NOT included as an input feature in the model.
However, the model learned patterns that led to:
· Lower rankings for resumes mentioning “women’s chess club”
· Penalties for graduates of women’s colleges
Amazon ultimately abandoned the system.
This is a real-world example of bias emerging from data—not explicit variables.
Model Setup
· Type: Supervised learning
· Features: resume text, education, experience
· Label: past hiring decisions
· Output: candidate ranking score
Performance Snapshot
|
Metric |
Model |
Human Recruiters |
|
Accuracy (predicting past decisions) |
90% |
75% |
|
Screening time per candidate |
Seconds |
Several minutes |
|
Estimated hiring cost reduction |
30% |
— |
|
Evidence of bias |
Significant gender disparity |
Present but less systematic |
Group Task (12 minutes)
Be concise and analytical.
1. Diagnose the Failure
Identify 2–3 specific issues with the model.
Focus on:
· training data
· labels
· features
2. Data Problem
Where is the root issue? Choose at least two:
· biased training data
· flawed label (past hiring decisions)
· proxy variables in features
Explain briefly.
3. Decision: What Should Amazon Do?
Choose one option:
A. Deploy the model as-is B. Use the model only as a support tool (human oversight required) C. Redesign the model before any use D. Abandon the approach entirely
You must justify your answer using:
· performance data (accuracy, cost)
· risk (bias, legal, reputational)
4. If You Chose B or C — What Would You Fix?
Propose 1–2 concrete changes, such as:
· changing the label (e.g., success on the job vs past hiring)
· removing or auditing features
· adding fairness constraints
· collecting new training data
5. Report Out (8–10 minutes)
Each group shares:
· One failure
· Their decision (A–D)
· One recommended fix