Recruiting and the Use of Artificial Intelligence - Survey Design Project
Skills
Collaborators
Class
Resources
About the Project
Project Overview
Five of us set out to understand something most AI hiring research skips past, not whether these systems work technically, but how differently the people using them and the people being evaluated by them actually experience the same tools. Our team designed and administered surveys to capture the perceptions of both organizations and job seekers, treating hiring as a real test case for a much broader question, whether the people building and deploying AI systems actually understand how those systems land with the people on the receiving end of them. Hiring made an especially useful test case, since it's one of the few contexts where an AI system's judgment carries immediate, personal consequences for the person being evaluated, and where that person almost never has visibility into how the decision was actually made.
Through rigorous survey design and statistical analysis, we uncovered significant perception gaps between how organizations view their AI recruitment practices and how job seekers experience them. Those gaps didn't show up as vague dissatisfaction on either side, they showed up as measurable, specific disagreements about fairness, transparency, and trust that a purely qualitative approach would have been much harder to pin down with any precision. Quantifying the gap mattered as much as identifying it, since a policy recommendation built on "some people feel uneasy about AI hiring" carries far less weight than one built on statistically significant differences between two clearly defined groups.
As a five-person team, the work divided across survey design, data collection, statistical analysis, and ethical framework application, converging into a single set of findings that treated AI hiring fairness as something that had to be measured from both sides at once, not assumed from either one alone. Getting a true read on the perception gap required treating organizational confidence and candidate anxiety as two separate datasets to be compared, not reconciled in advance, and that structural choice shaped every survey question we wrote from the very beginning of the project.
The Issue
Recruitment pipelines have moved fast, resume screening, candidate matching, automated interviews, all increasingly AI-driven, yet there is little understanding of how these tools are perceived by the people they evaluate. Adoption was outpacing any real accountability for how those systems were actually landing with the candidates being scored by them, and most of the existing research on AI hiring focused on technical accuracy or legal compliance rather than the actual lived experience of the people being screened.
Job seekers report feeling uncertain, excluded, and unfairly assessed by AI systems they cannot see or understand. Meanwhile, organizations assume their AI practices are transparent and fair, creating a dangerous perception gap, one side confident it's being fair, the other side experiencing something that doesn't feel fair at all, with neither side necessarily aware the disagreement even exists. That blind spot is exactly what makes the gap dangerous rather than just uncomfortable, an organization that believes its process is already fair has little internal motivation to examine it further, while the candidates experiencing the unfairness have no channel to surface what they're actually encountering.
Key Objectives
Design comprehensive surveys using Qualtrics for both organizations and job seekers.
Collect and manage survey data with rigorous statistical methodology.
Identify perception gaps between employers and candidates on AI fairness.
Apply ethical decision-making frameworks to evaluate recruitment practices.
Develop actionable recommendations for improving AI transparency.
Analyze trends and correlations in AI recruitment perceptions.
My Role & Impact
Lead Survey Design
Led the design of comprehensive surveys using Qualtrics for both target audiences, structuring parallel questions that let organization and job seeker responses be compared directly.
Data Collection & Management
Managed survey distribution and data organization using Excel, keeping response data clean and consistent across both survey groups throughout collection.
Statistical Analysis
Performed statistical analysis to identify trends and correlations in responses, turning raw survey answers into measurable patterns rather than anecdotal impressions.
Perception Gap Analysis
Revealed critical differences in how organizations and job seekers perceive AI fairness, the core finding the rest of the project's recommendations were built around.
The Process
- A literature review on AI adoption in recruitment and existing fairness research came first, establishing what was already known before designing new survey instruments to fill the gaps that research hadn't yet answered. That review made clear that most existing work approached AI hiring fairness from a technical or legal angle, leaving the actual perception gap between employers and candidates largely unmeasured.
- Dual surveys targeting organizations, HR and hiring managers, and job seekers went out through Qualtrics, deliberately structured with parallel question sets so responses from both groups could be compared directly rather than analyzed as two unrelated datasets. Writing genuinely parallel questions took real care, the same underlying concept, trust, transparency, fairness, had to be asked about in language that made sense from both an employer's operational perspective and a candidate's personal one.
- Distributing the surveys and collecting responses meant actively managing data quality and integrity throughout, not just sending a form out and letting it fill itself in. Response quality mattered especially on the organizational side, where getting genuine, unfiltered answers from HR professionals about their own practices required survey design that didn't feel like an audit.
- Statistical analysis identified trends, correlations, and perception gaps between groups, ethical decision-making frameworks contextualized the findings, and everything synthesized into actionable recommendations for transparency and fairness in AI hiring. The ethical framework step kept the statistical findings from becoming just numbers on a page, turning "organizations overestimate fairness by X amount" into an actual argument about what obligations that gap creates.
The Solution
Organizations significantly overestimate how fair job seekers perceive AI tools to be, that was the headline finding. Job seekers strongly prefer to know when AI is evaluating their applications, and organizations that disclose AI use build greater trust with candidates, a finding that reframed transparency less as a compliance requirement and more as something that directly benefits the organization deploying the AI in the first place. That reframing mattered for how the recommendations landed, since a policy proposal that only appeals to candidate protection is an easier sell to ignore than one that also serves the organization's own interest in being trusted.
Our recommendations include adopting transparency-driven AI recruitment policies, implementing clear disclosure practices when AI is used in hiring decisions, and conducting regular bias audits to ensure fairness across candidate demographics. Disclosure alone wasn't treated as sufficient, since telling candidates AI is involved without ever auditing whether that AI is actually fair would just be transparency about an unexamined problem rather than a real fix for it, and the combination of disclosure plus ongoing audit was designed specifically to close the perception gap from both directions at once, giving candidates real information while giving organizations an actual mechanism to verify the fairness they currently only assume.
Common Questions
Frequently Asked Questions
A five-person research project revealing how differently organizations and job seekers see the same AI hiring tools.