Deliverables, Sprint Rules, and Grading Rubric

Orientation

This is a study-and-analysis course. Projects exist to test empirical claims about data, methods, or systems — not to ship software. Every team builds something, but the build is instrumental: just enough engineering to let the study run, reproducibly, end-to-end. Effort beyond that floor is not rewarded and is often a warning sign.

The scaffolding contract lives in universal_scaffolding.md. These deliverables sit on top of it.

1. What every project ships

Five things at the end. Everything else is optional polish.

1. Versioned study plan, related-work document, and study design — docs/study_plan.md, docs/related_work.md, and docs/study_design.md, each with a dated changelog. Initial v0 of the study plan and related-work doc are committed by end of Sprint 1 — the starting research questions and the initial anchor set of prior work. Both evolve across versions as the project learns what the right questions are and which threads of the literature actually matter; version bumps in the study plan usually pull corresponding bumps in related work. By the time the main experiment runs, the current version of each is committed. These are the artifacts that let the reader reconstruct what was planned vs. what emerged.
2. Minimum viable build — a git repo someone else can clone and run. README, dependency file, seed control, config-driven conditions. "Works on a machine other than yours" and "reproduces the headline result from a committed log" are the bar. No UI is required beyond what the study itself needs.
3. Project-specific data artifacts and run logs — whatever the study produces: cleaned datasets, rater responses, LLM run logs, extracted graphs, cluster traces, survey responses. Versioned; runs replayable from what's in the repo.
4. Technical report — research-paper-style writeup (8–15 pages): intro, related work, study design, method, results with confidence intervals, findings, discussion, limitations, ethics. A well-executed null result counts as full success.
5. Final presentation — 12 min (solo) / 20 min (pair): question → method → findings → limitations → individual contributions per team member. See §5 for what to cover.

2. Scope philosophy: "just enough to study what's needed"

Every project in this course is study-first. There is no build-heavy track.

The build exists to enable the empirical claim. Its quality is judged by three questions:

• Does it actually run the experiment end-to-end?
• Does it produce logs the analysis can consume?
• Can someone else reproduce the headline result from the repo?

If yes to all three, the build is sufficient. Polish beyond that — nicer UIs, extra features, auth, caching, abstractions for "future extensibility" — is out of scope and will be flagged in review as engineering performed in place of science.

Where teams do invest beyond minimum is in Part A of the scaffolding: deeper related work, a sharper study design with more controls, a more rigorously validated quality spec, a more careful data-exploration and ethics treatment. That is where depth is rewarded.

Team scope examples (but all students must be able to explain all parts):

• Solo (1 student). Tightest scope. One primary hypothesis. One extraction method, one condition set. Cut aggressively; finish.
• Pair (2 students). One owns the study design + analysis; the other owns the MVB + data pipeline. Writeup is shared.
• Trio (3 students). Richer study: a primary plus one secondary hypothesis, or one primary hypothesis with a strong control condition. Three clearly-distinct gradable contributions — e.g., method / runtime / evaluation+analysis.

3. Sprint rules

At the start of each sprint:

• Every team member has one clearly-defined job for the week. Written down. Owned. Not "we'll figure it out as we go."

At the end of each sprint, we want to see:

• Commits from each person, under their own name, in the repo. "We worked from the same computer" is not a thing in this course. Pair-programming is fine — the commit history still shows individual contributions.
• Results. Whatever your job was supposed to produce — code, data, analysis, writing — shows concrete output. No "I thought about it a lot."
• A one-page-per-person report. Three things: what I did this sprint, what blocked me, what I'm doing next. In markdown
• Lessons are project meetings. We use class time to discuss each project together — progress, decisions, open problems, stuck points. Come prepared to present and defend your work.

You own your commits. We can ask you, specifically, about any commit under your name. "My teammate wrote that" is not an answer.

Tracking. All work happens in the project's git repo on GitHub:

• Ideally, one or two issues per person per sprint. Filed at sprint start, describing that person's job for the week. Assigned to themselves. Closed at sprint end by a commit or PR that references it (e.g., Closes #7).
• One milestone per sprint. Issues attach to the current sprint's milestone — this gives a built-in "what got done vs. what's left" view at each sprint boundary.
• Sprint-end markdown note (/notes/sprint-<N>-<yourname>.md). Issues track the job; the markdown note is the narrative.

Optional, not required: Pull Requests, GitHub Projects kanban board, GitHub Actions / CI. In a course with a short horizon and small teams, these are overhead more often than they are value.

4. Grading rubric

This section is the single source of truth for how projects are graded. universal_scaffolding.md describes what the ten scaffolding components are; proposal_template.md describes how the proposal is structured. Neither restates the rubric — they point here.

We grade on the end deliverables and on how you got there. Process is not neutral — a beautiful report produced by one person while the others coasted is not a team that earned a team grade. The rubric is organized to match the scaffolding: study design is the bulk of the grade, engineering is a floor, reporting ties it together. A clean null result with good methodology beats a positive result with bad methodology — every time.

Grading is two-pass. First, a scaffolding gate: does the project clear the minimum-acceptable bar on all scaffolding components? If not, the project is not graded on content — feedback is methodology-focused and the issue must be fixed before content grading begins. Second, a content grade on the weighted areas in §4.1 below.

4.1 Weighting (indicative, out of 30)

The Engineering floor is pass/fail, not additive: if it fails, we don't grade content until it's fixed. Passing it does not earn points; it only unlocks the rest of the rubric.

5. Final presentation — what to cover

12 minutes for solo (1-person) projects, 20 minutes for pairs (2-person). This is the expanded version of deliverable #5. You don't have to follow this order rigidly, but everything below should be in there somewhere. Above all: tell a story, don't read a checklist at us.

Aim for fewer than 10 slides. If you want to keep extra detail on hand, add backup slides after a "The End" slide — you can pull them up if a question calls for them, without cluttering the main talk.

• Demo, if you have one. Lead with it. A live or recorded demo earns more attention than any slide. Show a base case first so we understand the normal behavior, then the interesting cases. Don't manufacture interesting examples — but if you genuinely saw surprising, funny, or revealing ones during your work, that's exactly what to show.
• Problem focus. Where did you actually put your energy? What do you see as the distinctive or interesting thing — the feature you built, or the question you studied? This is the heart of the talk; make sure it's unmistakable.
• Related work. What areas did you look into? What did you learn and reuse from prior work? And — just as important — what did you not find, where you had to come up with an answer yourself? Be honest about both: standing on existing work is good, and the gaps you had to fill yourself are often your real contribution.
• System architecture. How the thing is built. Be specific about which agents you use and how, and how many different kinds of prompts there are. They needn't be many — a single agent and one prompt is a fine answer — but if you have distinct agent types or prompt roles, name them and say what each does.
• Evaluation. How did you design it? Which questions did you actually want to answer? Are there quantitative metrics — and if so, which and why? If the evaluation is qualitative, how did you assess it (rubric, raters, agreement)? And if you have a working implementation, show us: a few interesting examples where the system works well, and — just as honestly — where it doesn't. Tie all of this back to the claim you set out to test.
• What was hard. The real obstacles — dead ends, things that didn't work, decisions you'd make differently. This is where we learn the most about your project.
• Contribution of each team member. Who did what, concretely. Every member should be able to speak to their own part.