Project Scope Statement
The AI “Hallucination” Project is a semester-long digital humanities research initiative about how large language models distort or erase the histories of marginalized communities. The project will produce two parallel annotated datasets — one focused on 19th to 20th century Black history (BKED) and another on 19th to 20th century Puerto Rican history— generated by running structured query prompts across GPT-5, Gemini, and Claude AI models and subjecting outputs to systematic human fact-checking and annotation. The final deliverables are a public-facing explorable website cataloguing hallucination instances by type and community, and a white paper documenting methodology, data collection procedures, and findings. The project is scoped for one semester, with all deliverables completed by May 1, 2026.
Scoping and Scheduling the Work
Translating the project’s conceptual goals into a concrete, time-bound work plan has been one of the more clarifying exercises of the semester so far. The Gantt chart we developed maps the full arc of the project across four broad phases: pre-production and outreach, dataset development, analysis and development, and final production. Each phase carries its own dependencies and risks, and understanding how they connect is essential to keeping the project on track.
The pre-production phase, running from late February through mid-March, focuses on establishing the structural and logistical foundations the rest of the project depends on. This includes finalizing the Project Work Plan and Data Management Plan, conducting an outreach and distribution strategy discussion, and beginning early community-facing work such as scheduling a consultation with a GCDI fellow and preparing promotional materials for a NYC Open Data Week presentation. In parallel, development of the project website skeleton (comprising the About, Methods, and placeholder Search Interface pages) begins on March 10th, with a dependency on the Designer/UX role providing sufficiently finalized design direction beforehand.
The dataset development phase constitutes the methodological core of the project. Beginning March 17th, the team will collaboratively draft approximately forty-five (45) query prompts drawn from verified Puerto Rican historical sources. Prompt writing is followed by a split execution strategy: The first fifty percent of raw model queries will be run between March 24th and 27th, at which point the Research Lead will review and approve the outputs before the full batch proceeds. This checkpoint is an intentional quality control measure, designed to catch errors in prompt design or model behavior before they propagate across the entire dataset. The second half of the query execution is scheduled for April 10th through 14th, immediately following spring break, allowing the team to return to the work with fresh perspective and sufficient time for the intensive annotation and verification work that follows.
The post-spring break period from mid-to-late April is the most logistically demanding phase of the project, with several parallel workstreams converging in a short window. Between April 13th and 17th, with advice from the research lead, the team will build a fact-checking and annotation workflow, compile and clean the verified dataset, and flag hallucination instances with accompanying notes. These tasks are largely sequential and directly gate the development work that follows: comparative hallucination rate charts cannot be built, and data visualizations cannot be integrated into the website, until both datasets are clean and verified. The design and UX role will additionally advise on chart style and color palette during this period, ensuring visual consistency across the archive. During this team, public-facing distribution and outreach strategies will launch: These will include strategic dissemination of the website URL and accompanying platform marketing efforts — an Instagram page, Substack newsletter, or Are.na channel — depending on what’s decided in the development of the distribution strategy.
The final production phase runs from April 25th through May 1st and involves simultaneous completion of the project’s three major deliverables: testing and deploying the final website, populating and finalizing the explorable database on the live site, and writing the technical sections of the white paper (including methodology, data collection, and findings).
