AI can insert bias before an application reaches a recruiter

Generative AI is increasingly presented as a way to make job searches more efficient: applicants can use it to draft a résumé, tailor a cover letter or identify relevant skills, while employers can use related systems to screen candidates. New research suggests that this convenience can create a subtler problem. The same kinds of language models may encode gender stereotypes at both ends of the hiring process.

In a study published in the Proceedings of the National Academy of Sciences, researchers examined how six language models wrote résumés and evaluated job candidates across 98 occupations in the United States. Their finding was not simply that models treated explicitly labeled men and women differently. The résumés generated for women contained different language and emphases from those generated for men, even when later evaluators were not told the applicant’s gender.

That matters because an applicant may use AI to create a professional-looking document, only for an employer’s AI-assisted screening process to draw inferences from the patterns embedded in that text. The researchers describe the risk as a double filter: bias can enter in the writing process and be reinforced during selection.

Six models, thousands of résumés

The researchers tested GPT 5.4, Claude Opus 4.6, DeepSeek 3.2, Mistral Large 3, Grok 4.1 and Llama 3.3 70B. In an initial phase, they generated 35,280 résumés while randomly assigning a candidate gender. They then asked systems to recommend salaries and assign employability scores, an approach intended to separate the effects of a résumé’s content from the gender associated with the applicant.

Across the experiment, the writing was not equivalent. Résumés written for women more often referenced helping others, contributing to organizational life and undertaking tasks that are less likely to lead to promotion. Such language may sound positive in isolation, but it can map onto familiar workplace assumptions about who is expected to perform supportive work and who is expected to lead.

The effect was detectable even after direct references to gender had been removed. According to the report, a classifier could identify whether a text had been generated for a woman or a man in 81% of cases. In other words, the gender signal was not confined to names or pronouns. It remained in the wording, the qualities highlighted and the profile constructed for the candidate.

Why polished language is not enough

The study challenges a common way of judging AI writing tools. A résumé can be fluent, professional and grammatically sound while still nudging a candidate toward a stereotyped presentation. If women are more often framed as helpful or organizationally supportive, while other traits receive greater emphasis in résumés for men, users may be receiving guidance that changes how their experience is interpreted.

Even AI thinks it's a man's job
Gender differences in the content of AI-generated CVs. Credit: Proceedings of the National Academy of Sciences (2026). DOI: 10.1073/pnas.2534118123

This is especially consequential in labor markets because résumés are compressed documents. They determine which accomplishments are foregrounded, which skills appear central and how a candidate’s trajectory is narrated. A small shift in emphasis can affect whether a reader sees someone as an operational contributor, a collaborative teammate, a specialist or a prospective leader.

Automated assessment can compound the issue. Hiring systems and recruiters often seek signals of fit in a limited amount of text. If generated language carries recognizable gendered patterns, a downstream model may treat those patterns as evidence about an applicant even when the applicant’s gender is not explicitly disclosed. The result is not necessarily an intentional discriminatory rule. It can be a feedback loop in which stereotype-shaped writing becomes an input to stereotype-sensitive evaluation.

A governance question for AI-mediated hiring

The research does not establish that every use of generative AI in recruitment is biased, nor does it show that all models behave identically in every setting. It does, however, show why claims that automation inherently makes hiring more impartial deserve scrutiny. Replacing one human step with an AI system does not eliminate judgment if the system has learned patterns from human-produced language and data.

For employers, the practical lesson is that testing cannot stop with a model’s accuracy or the apparent neutrality of a prompt. Organizations using AI in recruitment need to assess the full workflow: how applicants are encouraged to use writing tools, what text those tools produce, what signals screening systems extract and whether outcomes differ across groups.

For job seekers, the study is a reminder that AI-generated text should be treated as a draft rather than an authoritative account of professional identity. Applicants can review whether a tool has overemphasized personality traits, volunteer-style work or supporting tasks at the expense of concrete achievements, technical expertise, responsibility and measurable results.

The larger lesson

AI systems are increasingly positioned between people and opportunity. The PNAS study shows that this position can matter before a formal hiring decision is made. When a model helps shape a résumé, it is not merely correcting grammar or improving formatting. It can influence which story an applicant tells about their work.

That makes transparency, auditing and meaningful human review central questions for AI-mediated labor markets. The challenge is not only preventing a system from reading an explicit gender label. It is understanding the less visible assumptions carried through generated language, and ensuring that tools marketed as assistance do not quietly reproduce the inequalities they are supposed to reduce.

This article is based on reporting by Phys.org. Read the original article.

Originally published on phys.org