Key Takeaways
- Despite marketing promises, research shows AI hiring tools don’t reliably reduce bias, and in some studies, they replicate or worsen it based on the data they’re trained on.
- The risk isn’t only in the technology itself. Teams that implicitly trust AI output over their own judgment are less likely to catch bias, errors, or bad trends before they affect hiring decisions.
- LLMs are built to generate confident answers, not to say “I don’t have enough information,” which can make them more decisive but also more prone to bias than a human reviewer would be.
- Telling an AI tool what not to do (like “don’t be biased”) is often less effective than telling it what to do. HR teams should audit AI tools’ inputs and outputs regularly rather than treating them as a fully automated solution.
Over the last four years large language models have been incorporated into many aspects of our lives. Despite their significant usage, many are starting to question the reliability of these tools. People find these tools useful in particular contexts but not necessarily universally successful at everything, which goes against the way many of them are marketed. Even with the added functionality of AI agents and harnesses, many still struggle with nuanced or context-dependent tasks. One of the prime examples of this is the use of AI in recruiting and hiring.
Despite promises to the contrary, studies are showing that AI does not make hiring and recruiting less biased, and in some cases may even make it worse. The reasons for this are complex and have as much to do with a misuse of LLMs as they do with flaws in the technology itself. These studies don’t show that AI is wholly useless or untrustworthy, but rather, that it could benefit from more guardrails and more careful implementation. These findings are an opportunity for organizations to develop a deeper understanding of these tools and reassess how they are using them in their work.
The promise of AI
Organizations have been using automated tools to screen resumes and applications for a long time, and they’ve had similarly troubling results in the past. The paradigm of LLMs changed things, at least on the surface, as these tools can be repurposed for multiple aspects of the process, not just screening resumes but also transcribing videos, conducting interviews, and summarizing information.
For many organizations, these tools are an irresistible force-multiplier, able to remove person-hours from portions of the process and also go through significantly more applications than a person could hope to. Finally, because of both the way AI is marketed and the perceptions that software is incapable of bias or other human errors, many hoped that it would be uniquely capable of sorting out the best candidate, irrespective of their age, gender, race or other identities.
How AI affects hiring
The issue starts with the way LLMs function, as they are trained using data from multiple sources, and all of it, on some level, comes from human decisions, which include biases. As with any system that transacts data, the outcomes will depend on the quality and patterns in that data, just like how the quality of a meal will vary depending on the quality of ingredients that go into it. If an LLM is trained on a dataset of hires that tend to belong to a particular race, socioeconomic background, gender, or other characteristic, this bias becomes a rule that the LLM will follow.
However, AI also introduces new risks and issues, particularly in organizations which do not fully understand or have experience working with an LLM. One of the most common is the assumption that AI is inherently ‘better’ in some aspect than human judgement is. While it is certainly faster and can quickly generate and summarize text, whether it is ‘good’ or not can be more difficult and subjective. Despite this, many teams choose to implicitly trust AI output over their own judgement, which means they’re less likely to notice trends, mistakes, and biases.
This issue is only exacerbated by LLMs’ tendency to generalize or outright make up information. Because many LLMs are incentivized to provide an answer, they sometimes give one they shouldn’t, since they’re designed to be decisive and authoritative. For this reason, AI can actually be more biased than a person, because it is optimized to give an answer rather than to say there isn’t enough information to make a decision.
Taking a step back, the term “artificial intelligence” itself can be misleading to people using it, as they assume that it “knows” things in the way a person does and can learn, when in reality, it’s just a model that provides outputs based on the probability of input data. This is most evident when users tell an LLM not to do something, for instance, saying “don’t be biased.” While this phrase would be meaningful to a person, an LLM does not “know” what bias means in the way the user intends. Negative instructions like this can actually backfire, making the unwanted outcome more likely, a pattern researchers have compared to ironic process theory, the well-documented tendency for humans to fixate on a thought after being told to suppress it. Whether the same mechanism is at work in an LLM is still being studied, but the practical lesson holds: it’s more effective to tell an AI tool what to do than what to avoid.
Reassessing AI in hiring
The common thread in all of these is a misunderstanding of the inner workings of these tools and their limitations. The deluge of marketing and promises of superhuman intelligence have made many feel they no longer have to check the work of automated tools. However, it’s clear now that AI can replicate issues like bias or even exacerbate them.
In order to avoid these issues, HR teams are starting to reassess the inputs and outputs of their AI tools, examining what data each tool relies on and checking the results of the output against past results. They are also looking for blind spots and addressing training that might lead users to cut corners by using AI output without checking it. Overall, efforts are being made to slow adoption and to view LLMs and other tools like any other software, which includes their misuse or mistakes.
Even putting aside bias, assessing recruiting performance can be challenging, as the success or failure of each hire may not be apparent until months after their hire date, and because you won’t know how other potential candidates might have performed, there’s an unfalsifiability problem as well. Because AI tools increase the velocity and volume of hiring, it can be tempting to view this as an obvious benefit, as you are increasing your hiring pool and assessing more candidates. However, unless you check the quality of hires after some time has passed, you won’t know for sure.
None of this means AI should be pulled from the hiring process, but it does mean it can’t be treated as a shortcut around bias rather than a tool that requires the same scrutiny as any other part of your process. The organizations getting this right are the ones auditing their AI tools’ inputs and outputs regularly, questioning results that look too clean, and holding onto human judgment rather than deferring to it. Bias in hiring was never going to be solved by software alone, and treating AI as a fix rather than a factor is how the same old problems find their way back in.
About Us
For more than 40 years, 501(c) Services has been a leader in offering solutions for unemployment costs, claims management, and HR support to nonprofit organizations. Two of our most popular programs are the 501(c) Agencies Trust and 501(c) HR Services. We understand the importance of compliance and accuracy and are committed to providing our clients with customized plans that fit their needs.
Contact us today to see if your organization could benefit from our services.
Are you already working with us and need assistance with an HR or unemployment issue? Contact us here.
The information contained in this article is not a substitute for legal advice or counsel and has been pulled from multiple sources.
(Images by User4894991 and Vwalakte)
