COMP 2300 Winter 26 / Anisha Arumilli
If you work in HR or in college admissions, you’ve probably started to love your job now that automated screening tools have taken the grunt work out of reviewing hundreds of resumes and applications.
They screen resumes in seconds, flag the most qualified candidates, and organize applications neatly. But they also filter out voices and perspectives that could change the company for the better.
Automated systems can make bias more consistent instead of making hiring more fair.
How Automated Screening Works
AI makes hiring decisions by identifying patterns in past recruitment data. It analyzes resumes and applications to find traits common among previously successful candidates, then finds new applicants who match those traits.
This seems like the perfect way to speed up recruitment, but AI hiring tools are only as fair as the data they’re trained on. Even when they appear objective, they can replicate or amplify existing biases.
Bias in the Training Data
Amazon discovered this with an AI recruiting tool trained on resumes from its mostly male workforce. The system began penalizing resumes that mentioned “women,” reinforcing the same demographic patterns that already existed rather than correcting them.
These patterns often come from human hiring decisions that contain both conscious and unconscious bias. Research published through VoxDev in May 2025 found that AI hiring tools systematically favored female applicants over Black male applicants with identical qualifications after learning patterns from biased historical hiring data.
Instead of making hiring more efficient, the technology just makes bias more consistent.
When the System Scores People Incorrectly
Bias can also appear in AI systems when the training data doesn’t capture the full diversity of people it evaluates. HireVue’s AI video interview system, used by over 1,000 companies including JPMorgan Chase, Goldman Sachs, and Amazon, has been shown to score non-white and deaf applicants lower because it was trained mostly on data from white and hearing candidates.
It misinterprets accents, speech patterns, and facial expressions it hasn’t seen before, giving these applicants lower scores.
Even when AI systems are built to be neutral, human bias can slip in through the structure of the system itself. The University of Texas at Austin used a machine-learning system called GRADE to evaluate candidates for the PhD in CS program which was found to systematically reject women and POC while favoring white men.
Letters of recommendation are influenced by social bias, often describing women as “caring” and men as “assertive” or “outstanding.” GRADE, trained on these letters, favored the language more often used for men, which led it to select more men.
Why Diversity Changes Outcomes
These biases affect more than individual applicants. Reduced diversity can limit collaboration and problem-solving because teams lack a wide range of perspectives and experiences. Over time, workplaces and academic programs become less adaptable and less innovative, reinforcing structures that favor only certain groups (Forbes).
The lack of diversity in tech is already creating real harm in the systems we use every day. When the people developing algorithms and AI systems do not represent the diversity of the people who will use those systems, we get facial recognition that fails to identify Black faces accurately.
We get health apps that ignore medical conditions primarily affecting women and people of color. We get voice assistants that fail to recognize accents from non-native English speakers.
The Feedback Loop
As organizations and universities rely more on automated systems, applicants are adjusting to fit the patterns these tools favor. Resumes and applications increasingly emphasize keywords, phrasing, and styles that AI systems recognize as “ideal,” and some students even turn to AI tools to help craft their submissions.
This behavior creates a feedback loop where AI not only filters candidates but also shapes how candidates present themselves, narrowing the range of experiences, perspectives, and approaches that are considered desirable.
Just as these automated systems become standard across universities and companies, we’re watching DEI initiatives get scaled back or eliminated entirely. IBM, Meta, and numerous other companies have rolled back diversity programs and universities are cutting similar initiatives.
The very programs designed to counterbalance structural inequities and create pathways for underrepresented students are disappearing right when automated screening is becoming more normalized. We’re eliminating the programs meant to counteract bias while encoding that bias into our hiring systems.
What Organizations Should Do
Companies and universities must adjust their screening systems to ensure equality. If you’re using AI to filter applications, you should know exactly what criteria it’s using and whether those criteria are actually predicting success or just replicating who you’ve historically hired.
You should be testing whether candidates from different backgrounds are being screened out at different rates, and if they are, you need to fix it or stop using the system entirely.
This means questioning whether the algorithm is actually assessing what you think it’s assessing. A 3.8 GPA isn’t necessarily more impressive than a 3.3 from a student who worked thirty hours a week to pay for school. The algorithm doesn’t know the difference, but companies should. Humans must be involved to contextualize these metrics and evaluate the full picture.
Some companies will say they can’t afford to review applications manually, that they receive thousands of resumes and need automation to manage the volume.
But if your screening process is systematically excluding talented people, you’re not actually managing anything efficiently. You’re just automating discrimination at scale. The question isn’t whether you can afford to review more applications carefully. It’s whether you can afford to keep missing the candidates who could actually solve your hardest problems.
The solution isn’t to abandon technology entirely. Keep humans in the decision-making process and continuously measure outcomes. Track who advances, who is filtered out, and whether the people your system selects actually succeed.
