Lessons on teaching and grading with AI from four case runs

Adam W. Stauffer with Claude · October 2026

When students may use AI on a case and the grading is generous, the grades end up at the ceiling and stop saying who understood anything. Across four case runs in 2026, what moved learning was making students run the thing they specified and then explain where it went wrong; what the cases should grade next is whether a student's checks could ever have failed.

At a glance. About 210 students have worked through these cases: 7 cohorts across 6 case runs, 3 case topics and 4 terms, in multiple countries. 132 of them are in the four runs reviewed here (5 cohorts, 3 terms); the two earlier runs, Fall 2025 FX hedging and the Spring 2026 undergraduate ratios case, add about 80 more but left no grading record to review (§§ 2 and 4). A seventh run, Fall 2026 FX hedging, is in progress and not reviewed here. Four cohorts were undergraduate and three graduate, and most of the graduate students were working professionals. The reviewed runs had 20 graded stages, with AI permitted in every one, and the same policy throughout: students disclose which AI they used and for what, and disclosed use is never marked down.

The three cases.

• Perfect competition and marginal cost. A small market garden must choose how many beds of three crops to plant at fixed market prices. Students build its cost structure, derive each crop's marginal cost, and use Excel's Solver to find the most profitable mix within the farm's limits. First run Fall 2026: one cohort finished, one in progress. Case page
• FX hedging. A firm will receive a payment in a foreign currency. Students model four ways to manage that exposure (a forward contract, a money-market hedge, currency options and doing nothing) across a range of future exchange rates, and recommend one to the CFO. Ran Fall 2025, Spring 2026 and Summer 2026; the Fall 2026 run is in progress. Case page
• Performance ratios. Students pick a listed company, load two years of its statements into a ratios workbook, and write a specification of the model and the analysis, drafted with AI, precise enough that an AI with no other context can run it. They check what comes back by hand and make the recommendation the AI couldn't. Ran Spring 2026 (undergraduate) and Summer 2026 (graduate); in revision for its next run in Spring 2027. Case page

Across all three, a spec is the student's written specification of the model before it exists, and running it cold means handing only that specification to a fresh AI session and comparing what comes back with the student's own work.

Cases built but not yet run. Three more cases are on the case site and haven't reached students yet, so nothing in this report draws on them:

• Economic profit and rent (microeconomics). Four working lives taken from accounting profit down to economic profit, then a taxi medallion's climb past $1 million and its collapse, reproduced by a single division: rent ÷ required return. Case page
• Imperfect competition (microeconomics). One company, two market structures: commodity corn seed priced at P = MC against patented seed priced at MR = MC. Students specify the model, have AI build it, audit the result, then put a number on what the patent costs society. Case page
• Team case study (micro and macro). A consulting team takes on a geopolitical challenge, such as trade and supply-chain fragility or climate policy, and pitches a recommendation live. The deck is generated from a ten-slide content map and a design spec, audited against both, and built in a shared repository through branches and pull requests. Case page

A synthesis of four case post-mortems written on the same day against the same lens, the Shen and Tamkin (2026) study of how AI use affects skill formation. The four detailed reports are private, because they describe individual cohorts; this one is the public summary. Every claim here traces to one of them; where they disagree or the evidence is thin, I say so. None of it is a measure of what students learned.

1. Executive summary

• Four cases, one result on grades: the ceiling. In every vintage, stage medians sit at or near full marks. Grade floors, regrades that can only raise a score, and structural first passes did that. The grades say who finished and revised. None of them says who understood the concept.
• The cases improved where they made students run something. The Spring FX prompt was written and never run. The Summer ratios spec was run cold, and its silences turned into the AI's errors that students then had to explain. That mechanic was copied into the Summer FX redesign, where every final stage ran it, and into the Fall economics case as spec → build → audit. It is the clearest lineage in the portfolio.
• Disclosure follows the template, everywhere. Where a field or a log was required, students filled it in, including "none." Where it wasn't, they almost never volunteered. Disclosed use was never marked down in any vintage, and no record suggests anyone feared it would be.
• Most of the friction sat off the concept. Links, paths, folders, file extensions, saved values, collaborator access, template residue: frequent in every case, expensive in hours, nearly empty of economics or finance. The concept-bearing errors were rarer and, more often than not, caught by the reviewer rather than the student.
• The strongest evidence of learning is students finding faults themselves — in their own build, in the AI's output, and in the course's own materials. Its mirror image is the most common weakness: a check that could not fail, reported as verification.
• The course disagreed with itself in every vintage, and my side of the process produced errors of its own in every vintage. Those are findings about the instructor, not the students.
• What comes next is a check, not a redesign. An end-of-case course evaluation with a short, ungraded concept check is the only thing in the portfolio that could say whether the cases taught what they set out to teach.

2. The portfolio at a glance

The evidence is not even. Spring FX read the artifacts but had no process to read. Summer FX read only what graders wrote. Summer ratios read the grade reports and a self-selected subset of the prompt logs. Fall read the repositories, but for presence, order and tool names, not content. Any comparison across rows inherits that. The two earlier runs have nothing to read beyond their designs, which is why neither has a post-mortem.

3. What held across every case

3.1 Ceiling grades

Every report reaches the same verdict in nearly the same words: the grading regime converges on the ceiling and cannot be used to decide what to change. The mechanisms are the same in all four — a floor on every stage, regrades that can only raise, and a first pass that counted structure (a heading, a named range, a word count, a list marker). In the Summer FX case the automated first pass found nothing in half the audit notes because it counted list markers, and many results had to be corrected by hand (Claude and Stauffer, 2026c). In the Summer ratios case, repeated instructor overrides fixed a first pass that missed content (Claude and Stauffer, 2026b).

I would keep the spirit of the generous regime — every report says so. What it needs is a companion measure, not a harsher rubric.

3.2 Disclosure follows the template

Students did not hide AI use. They disclosed exactly where the course asked, in the form the course asked for. The design question is therefore where to ask, not whether students will answer. The corollary: in two vintages the logs existed and nobody read them as evidence. That is the cheapest unread data in the portfolio.

3.3 Friction off the concept

The non-concept errors are the most frequent in every case:

• Submission and paths. Some Spring FX final-stage links failed; Fall students hit wrong folders, .gitignore saved without its dot, and empty folders that never reached GitHub; Summer ratios students saved .md.txt files the grader couldn't see.
• Save state. Workbooks with formulas and no saved values appear in Spring FX and, far more often, in Fall, where about half the online cohort also committed no saved Solver model.
• Repository mechanics and polish. More than half the Summer FX skeletons were incomplete, and most final repositories had no license or description; placeholder READMEs and default commit messages showed up in the ratios cohort; collaborator access took almost the whole Fall case to reach every student.
• Template residue. Placeholder titles, and a template's own vocabulary or metadata kept over the brief's, in three of the four cases.

This is the study's distinction made concrete. Errors like these cost time and teach nothing about parity, ratios or marginal cost. The Fall report puts it bluntly: the case produced plenty of friction, but much of it missed the concept.

3.4 Self-audit as the strongest evidence

The best learning moment in every report has the same shape: a concept-bearing error met and resolved by the student.

• Spring FX: more than a third of specs named a concrete, quantified defect in the student's own model.
• Summer ratios: in roughly a third of final-stage repositories the hand check caught an AI figure that didn't match; several analyses separated AI mistakes from gaps in the student's own spec and from convention differences.
• Summer FX: several students caught the course's reference calculator using a different premium-timing convention from the one their specs required, and worked out which was right.
• Fall economics: several students found real errors in their own work; only a few caught an error in the course's own material.

The mirror image is just as consistent: a check that cannot fail, reported as verification. In Spring FX, several specs asserted that parity held while the workbook beside them showed a large gap. In Summer FX, several students reported perfect matches between calculations built from the same inputs. Several ratios specs carried validation rules with no expected value. In Fall, a few checks could not fail, and most specs stated no tolerance at all. Four cases asked for verification. Mostly they rewarded confirmation.

3.5 Materials disagreeing with the course

Every report has a "the materials must agree" learning:

• Spring FX: the spec template had no model-review section while the brief required one; the Excel template's names disagreed with the brief's; the brief's point values disagreed with the gradebook.
• Summer FX: the reference calculator's premium convention differed from what the spec template required; the spec template still carried old stage numbers.
• Summer ratios: a named range the provided template didn't carry cost nearly every workbook; the bridge template styled input cells as calculated cells.
• Fall economics: stage point values didn't match the gradebook, a brief asked for something the course doesn't require, and a brief carried an error of its own.

The students who trusted the materials paid for it, and the students who noticed did the best work. That is a bad trade to leave to chance, and every report says so.

3.6 Process-side errors

My side of the process made errors in every vintage:

• structural scorers that misread content (Summer FX, Summer ratios)
• files credited without being opened, stale claims in comments, a wrong due date sent to most of a cohort, and student threads left unread for weeks (Fall)
• feedback that supplied the corrected formula and so took away the very error the student should have resolved (Summer ratios)
• a revision pass that could not open the workbooks it was grading (Summer ratios)

Several reached students. The Fall report's line holds for the portfolio: most of the learnings are about the instructor, and that is itself the finding.

4. How each vintage fed the next

The five handoffs that shaped the portfolio, drawn from the four reports' "What changed" sections. Dates are when the record shows the change, not when it reached students.

Two things stand out. First, the one change the portfolio argues against — moving the build ahead of the spec in Spring 2026 — was reversed within two vintages, and the reversal borrowed its mechanics from the ratios case rather than inventing its own. Second, ideas crossed topics more readily than they crossed terms within a topic: the staged flow went from ratios to FX to economics, not down a single course's line.

5. What differs by topic and cohort

By topic, the concept-bearing error has a different home.

The Summer FX report names the general pattern: clean workbooks move the errors into the prose, where a structural check can't see them. The ratios case shows the same thing from the other side — the AI rarely got the arithmetic wrong; it got the choices underneath wrong.

By cohort, outside knowledge mattered most for working professionals. The best ratios analyses came from students writing about their own company or industry, and from statements reported under a different accounting standard from the template's. In the Fall case, several students independently raised the costing question the case exists to raise, before the case asked. The undergraduate FX records show nothing equivalent, though their records are also the thinnest on student questions.

Online and in-person: alike on workbook habits, too early on the concepts. The Fall in-person cohort's early builds repeat the online cohort's workbook habits, but several students built a repository and never submitted its link, and fewer than half of the early briefs used the AI critique the way the stage asked. On the concepts it is too early to say.

Calendar matters. The ratios case ran six stages in two and a half weeks, and feedback on one stage could not always reach students before the next asked them to act on it. The Summer FX redesign landed in week one of a six-week term and cost throughput: several stages were re-reviewed five or six times. The Fall case ran over eight weeks.

Undergraduate versus graduate is confounded with everything else. Both reviewed undergraduate runs were FX, and the one undergraduate ratios run left no record to set beside them; the graduate runs were ratios and economics. I can't separate cohort from topic with this portfolio, and I won't try.

6. Open questions and what I'll look at next

The portfolio cannot answer the question that matters for the next run of each case: did students learn the concept, and did the way they used AI change that? Three of the four reports say so explicitly.

What I'll add to each case is ordinary course evaluation:

• an optional end-of-case questionnaire on how students used AI at each stage, which errors they hit, who fixed them, and where their hours went
• a short concept check done without AI and never graded, in the case's own terms, for example recomputing one hedge or one ratio by hand, or finding the misconception in a broken cost cell

Self-report alone would mislead. In the study, the AI group found the task easier and learned less, so the check matters more than the questionnaire. Both are course evaluation: their results stay inside the course, and any later write-up would go through the university's usual review first.

Two cheaper steps can happen now. First, read the prompt logs the cases already require as a record of how the work was done. Second, for every check in every case, ask what error it could have caught.

7. Limits and issues

• This is four readings of grading records, not a measure of learning. No vintage has an unassisted check. Every count is a reviewer's count — structural, approximate, and often a floor or an upper bound depending on how it was read.
• The evidence depth differs by report (§ 2), so a pattern that appears in one report and not another may be a difference in what was read, not in what happened.
• AI use is what students disclosed or what the files plainly show. Nothing was inferred from how anything was written, and no authorship or style tool was run on any student's work in any vintage. Undisclosed use is invisible; the study's own pilots found 25–35% of its no-AI group used AI anyway.
• Small, non-random cohorts, one instructor, one year. 132 students across five cohorts; topic and level are confounded (§ 5). Nothing here is causal, and nothing here is offered as a general finding about AI and learning. It describes these cases, for the purpose of improving them.
• The author is the instructor. I designed the cases, graded them, and wrote these reports with Claude. The process-side errors in § 3.6 are there because the records show them, but a reviewer reading their own record has an interest.
• The lens is borrowed. Shen and Tamkin studied 52 programmers for an hour, on a coding task, with immediate recall and no transfer test. Their result tells me where to look — the mode of AI use, and errors as the mechanism — not what I'll find in a business case over weeks.
• Revision is slower than I'd like. Each case is frozen for the term it runs in, so no student is graded against a moving brief — but a lesson from one vintage reaches students a term later at the earliest. The Spring FX diagnosis came in July and shaped the Summer run; the Fall economics changes wait for Spring 2027. The one time a redesign went in fast — Summer FX, live in week one of its term — it cost throughput (§ 5), which is the case for the freeze and also its price.

References

Sources

• Shen, J. H., and Tamkin, A. (2026). "How AI Impacts Skill Formation." arXiv:2601.20245v2 [cs.CY], 1 February 2026. https://arxiv.org/abs/2601.20245. Pre-registration: https://osf.io/w49e7. Annotated transcripts: https://github.com/safety-research/how-ai-impacts-skill-formation. The lens throughout; the self-report point in § 6; the 25–35% pilot figure and the study's limits in § 7.
• Borio, C., McCauley, R. N., McGuire, P., and Sushko, V. (2016). "Covered interest parity lost: understanding the cross-currency basis." BIS Quarterly Review, September 2016. https://www.bis.org/publ/qtrpdf/r_qt1609e.htm (accessed 2026-10-15). Parity between the forward and the money-market hedge, and the forward premium against the interest differential (§§ 3.4, 5).
• Git project. "gitignore — Specifies intentionally untracked files to ignore." Git documentation. https://git-scm.com/docs/gitignore (accessed 2026-10-15). The .gitignore naming problem in § 3.3.
• Git project wiki (archived by kernel.org). "GitFaq — Can I add empty directories?" https://archive.kernel.org/oldwiki/git.wiki.kernel.org/index.php/GitFaq.html (accessed 2026-10-15). Empty folders that never reach GitHub (§ 3.3).
• Microsoft Support. "Define and solve a problem by using Solver." https://support.microsoft.com/en-us/office/define-and-solve-a-problem-by-using-solver-5d1a388f-079d-43ac-a7eb-f63e45925040 (accessed 2026-10-15). Saved Solver models (§ 3.3).

What a workbook written by a script looks like before Excel saves it (§ 3.3) is a practitioner observation, not a cited claim.

Reports covered (private)

• Claude and Stauffer, A. W. (2026a, October 15). Spring 2026 — FX hedging: a case post-mortem. Private case report, not published.
• Claude and Stauffer, A. W. (2026b, October 15). Summer 2026 — Performance ratios (graduate cohort): a case post-mortem. Private case report, not published.
• Claude and Stauffer, A. W. (2026c, October 15). Summer 2026 — FX hedging: a case post-mortem. Private case report, not published.
• Claude and Stauffer, A. W. (2026d, October 15). Fall 2026 — Perfect competition and marginal cost: a case post-mortem. Private case report, not published.

Their evidence base is stage review and grading records from April to October 2026; submitted repositories, workbooks, specifications and memos, read structurally where a report says so; a subset of student prompt logs; and case design memos dated 2026-02-23 to 2026-10-02.