Bias and Fairness in AI Outputs: INFO 1111 Journal Project Redesign
Skills
Class
Resources
About the Project
Project Overview
As part of my INFO 1111 course at the University of Colorado Boulder, I redesigned the course's generative AI journal series around a single critical question: whose experiences and voices does AI reflect, and whose does it exclude? The redesign draws on three foundational readings, Langdon Winner's argument that technology carries politics in its design, Amy Ko's framework for how information systems reproduce hierarchy, and Kalai et al.'s research on hallucination as a statistical inevitability rewarded by current AI benchmarks, using all three as a connected analytical framework rather than isolated assigned texts.
The final deliverable was a full project redesign including learning goals, phased assignments, and a detailed grading rubric, presented in both written and slide format. Rather than treating students as passive observers of AI output, the redesign repositions them as auditors, building a four-phase framework that moves from personal reflection to comparative analysis to documented investigation to critical synthesis.
The Issue
The original journal series treated students as observers of AI, prompting reflection without a structured path toward evidence-based critique. Students could notice that an AI system behaved a certain way, but the assignments never asked them to go further than that, to gather evidence, compare cases, or build an actual argument about what that behavior meant and who it affected. Observation without a framework for pushing past it just produces more observation, not critical understanding.
Separately, the three assigned readings existed as isolated texts rather than a connected framework. Students had theory on design politics from Winner, theory on systemic hierarchy from Ko, and research on hallucination from Kalai et al., but nothing connected the three into a single lens they could actually apply. Each reading made its own argument in isolation, and it was left to individual students to notice, if they noticed at all, that these three arguments were really describing different angles of the same underlying problem: that AI systems are not neutral, and their failures are not random.
Key Objectives
Reposition students from observers of AI to active auditors of AI output.
Move students through four phases: reflection, comparison, investigation, and synthesis.
Connect Winner, Ko, and Kalai et al. into one applied analytical framework.
Link hallucination research directly to fairness for underrepresented communities.
Build phased deliverables that move students from assumption to evidence-based argument.
Develop a grading rubric emphasizing reflection, evidence, and critical engagement.
My Role & Impact
Curriculum Design
Built the four-phase journal structure from the ground up, sequencing assignments so each phase required the skill the previous one had just built, rather than letting students skip ahead.
Research Synthesis
Applied Winner, Ko, and Kalai et al. as one connected analytical framework, turning three separate academic arguments into a single lens students could actually apply to real AI output.
Assessment Design
Developed structured deliverables and a detailed grading rubric built alongside the assignments themselves, so credit stayed tied to the exact skill each stage was designed to develop.
Equity Framing
Connected hallucination, a statistical behavior current benchmarks actually reward, directly to real-world fairness, treating a technical detail as relevant to whose voices get misrepresented or left out.
The Process
- Identified the central audit question, whose experiences does AI reflect, and whose does it exclude, and worked backward from it to structure assignments capable of actually answering that question with evidence rather than impression. The framework needed to move students past noticing a pattern in AI output toward actually being able to defend a claim about it, which meant every phase had to build a specific skill rather than just prompting more open-ended reflection.
- Mapped a four-phase progression, personal reflection, comparative analysis, documented investigation, and critical synthesis, sequencing the phases so each one built the skill the next phase would require. Students couldn't reach documented investigation without first developing the comparative instincts from phase two, and synthesis was only possible once they'd actually gathered evidence in phase three rather than working from assumption.
- Synthesized Winner, Ko, and Kalai et al. into a single applied framework, translating three separate academic arguments, design politics, systemic hierarchy, and hallucination as statistical incentive, into one lens students could actually use on real AI output. Rather than treating the readings as background theory to be summarized, each one became a specific tool: Winner for questioning what a system's design assumes, Ko for tracing whose interests get served by that design, and Kalai et al. for understanding why confident, wrong answers get rewarded in the first place.
- Built the grading rubric alongside the phases rather than after them, so what earned credit was tied directly to the specific skill that stage of the framework was meant to develop. A rubric written after the fact tends to reward polish over the actual thinking the assignment was designed to produce, building it in parallel kept credit tied to depth of reflection and evidence use at each stage, not just a well-written final paragraph.
The Solution
Delivered a full redesign of INFO 1111's generative AI journal series, restructuring it around a four-phase audit framework grounded in three connected academic lenses. What began as a set of reflective prompts became a structured method for actually interrogating AI output, asking not just how a system responded, but whose experience that response reflected and whose it left out.
The project strengthened applied research, academic writing, and critical AI literacy skills, and built a transferable framework for auditing AI outputs for representation, omission, and overconfidence, a lens directly applicable to responsible AI and governance work beyond the classroom. More broadly, it reinforced an instinct that shows up across my other work too: understanding a system well enough to use it isn't the same as understanding it well enough to hold it accountable, and building the second usually takes deliberate structure, not just good intentions.
Common Questions
Frequently Asked Questions
A course redesign that turns students from passive AI users into active auditors of what the technology gets wrong, and who it leaves out.