Discussions: Google Notebook LM & Amazon's Hiring

In-Class Case: Amazon’s Hiring Algorithm

Video:

https://www.youtube.com/watch?v=V2gfNKxpVmg

Article:

https://www.reuters.com/article/world/insight-amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK0AG/

Case (3 minutes)

Amazon developed a machine learning model to screen job candidates.

· Trained on 10 years of past resumes

· Used historical hiring decisions as the label (hire vs. reject)

· Goal: predict which candidates should be hired

Important: Gender was NOT included as an input feature in the model.

However, the model learned patterns that led to:

· Lower rankings for resumes mentioning “women’s chess club”

· Penalties for graduates of women’s colleges

Amazon ultimately abandoned the system.

This is a real-world example of bias emerging from data—not explicit variables.

Model Setup

· Type: Supervised learning

· Features: resume text, education, experience

· Label: past hiring decisions

· Output: candidate ranking score

Performance Snapshot

Metric	Model	Human Recruiters
Accuracy (predicting past decisions)	90%	75%
Screening time per candidate	Seconds	Several minutes
Estimated hiring cost reduction	30%	—
Evidence of bias	Significant gender disparity	Present but less systematic

Group Task (12 minutes)

Be concise and analytical.

1. Diagnose the Failure

Identify 2–3 specific issues with the model.

Focus on:

· training data

· labels

· features

2. Data Problem

Where is the root issue? Choose at least two:

· biased training data

· flawed label (past hiring decisions)

· proxy variables in features

Explain briefly.

3. Decision: What Should Amazon Do?

Choose one option:

A. Deploy the model as-is B. Use the model only as a support tool (human oversight required) C. Redesign the model before any use D. Abandon the approach entirely

 You must justify your answer using:

· performance data (accuracy, cost)

· risk (bias, legal, reputational)

4. If You Chose B or C — What Would You Fix?

Propose 1–2 concrete changes, such as:

· changing the label (e.g., success on the job vs past hiring)

· removing or auditing features

· adding fairness constraints

· collecting new training data

5. Report Out (8–10 minutes)

Each group shares:

· One failure

· Their decision (A–D)

· One recommended fix