Research Methodology

Course Focus

This course hopes to help design research projects that are methodologically sound, properly scoped, and accurately reported.

1

Problem Definition

Finding the right problem and formulating research questions we can tackle. Framing problems in a way that is both practically useful and scientifically interesting.

2

Scoping & Assumptions

Limiting domain and data types, identifying reasonable assumptions that keep the problem interesting yet manageable.

3

Hypotheses & Experiments

Detailing research questions into experiments and hypotheses with clear falsification criteria.

4

Evaluation & Reporting

Designing credible evaluations, understanding uncertainty, and reporting results honestly.

Pedagogical Rhythm

Think individually → Breakout groups → Report back → Instructor synthesis

Sessions

Session 1 3 hours

Tackling a Problem End to End

From motivations to research questions, scoping, state of the art, datasets, evaluation metrics, process. The output is a well-defined, “executable” research problem — at least for the first iteration.

Session 2 2h 15m

Experimentation & Evaluation

How would you know if you succeeded?

Experiment Design

From hypotheses to experiments
Hidden degrees of freedom
The portfolio problem and winner's curse
Exploratory vs confirmatory experiments

Evaluation Pitfalls

The evaluation trinity: construct, measurement, credibility
Failure modes: leakage, gaming, baseline dressing
Intrinsic vs extrinsic evaluation

Ground Truth & Baselines

Where does ground truth come from?
Baselines as honesty tools
Ablations and failure cases
Honest reporting and uncertainty

Session 3 3h 30m

Student Presentations

Defend your research design

5-Slide Presentation

Apply the course methodology to your own research (or a topic of your choice).

Slide 1: Problem & research questions — what and why it matters
Slide 2: Assumptions & scope — what you're NOT addressing
Slide 3: Definition of done — when is this "enough" for your PhD?
Slide 4: Data & experiments — where data comes from, concrete experiments
Slide 5: Next steps, blockers & risks — what you'll do next, what could go wrong, what you don't know yet

Course Outcomes

By the end of the course, each student has:

A scoped research problem with clear stakeholder context
Artifact requirements analysis
Clear, answerable research questions
Explicit, falsifiable hypotheses with falsification criteria
A complete evaluation design (metrics, ground truth, baselines, ablations)
A risk and uncertainty analysis
A peer-reviewed research methodology plan