Designing Large Scale AI Systems 2025/26
CS Master Course | Spring 2026 | 12 weeks
About This Course
This course provides a comprehensive introduction to designing, building, and evaluating large-scale AI systems. Students will gain hands-on experience building AI-powered applications and learn rigorous evaluation methodologies essential for deploying AI systems in production.
Basic software engineering experience is preferred — but not a must. The main prerequisite is the passion to build great systems.
Learning Objectives
By the end of this course, you will:
1. Build Effective, Reliable AI Systems at Scale
Solve complex problems by decomposing them in a principled way. Ship a reliable AI feature into an existing product.
2. Use AI to Build AI
Leverage AI for development, evaluation, and system management. Understand what this unlocks and where the limits are.
3. Iterate Towards Success
Master the science of monitoring, analysis, and estimation. Stand up an evaluation harness you can run in CI.
4. Identify and Master the Key Abstractions
Build observability, cost controls, and incident-ready guardrails. Take lessons from software engineering past, but do so critically.
5. Think in Systems
Understand which engineering principles carry forward and which new abstractions are essential.
Sessions
Phase 1 Building AI Agents
Connecting to AI via API: chat, streaming, voice, and image
Prompt engineering, structured outputs, best practices for reliable single calls
Stateless & stateful agents, conversation history, cost & latency tradeoffs, memory strategies
Tool calling, function schemas, orchestration loops, when to call another tool vs. stop
Phase 2 "Eval" in Software 3.0: Iterating Towards Success
Phase 3 Building and Managing Complex Systems
Basic Systems and Non-AI Abstractions
Adding AI Abstractions
Council of Agents and Other Agentic Patterns
Shipping into existing products, maintaining large AI systems, multi-version considerations
Without Losing Control (and Mental Health)
When and why to use frameworks; LangChain as a case study
Phase 4 Capstone Project
Lessons in this phase are project meetings, running in parallel with the Lectures on Complex Systems.
Assessment
| Component | Weight | Description |
|---|---|---|
| Part I: Midterm Exam | 30% | Individual, written |
| Part II: Course Project | 70% | In-class, assisted, attendance required (Track A) or individual, unassisted (Track B); four sprints plus presentation prep |
Office Hours
Office hours are held right after each class. Additional appointments can be scheduled via email.
Have 20 seconds?
What's the one thing you'd want to learn about building AI systems at scale?