AI Design 2026 — Additional Exercises

Key Formulas Reference

Bayes' Theorem $P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}$

Beta Posterior Update $\text{Beta}(\alpha, \beta) + s \text{ succ.}, f \text{ fail.} \to \text{Beta}(\alpha+s,\;\beta+f)$

Beta Mean $E[\theta] = \frac{\alpha}{\alpha + \beta}$

Sum of Independents $E[X+Y] = E[X]+E[Y], \quad \text{Var}(X+Y) = \text{Var}(X)+\text{Var}(Y)$

CLT (sum) $S_n = \sum X_i \;\dot\sim\; N(n\mu,\; n\sigma^2)$

Standard Error $SE = \frac{\sigma}{\sqrt{n}} \;\;\Rightarrow\;\; SE \propto \frac{1}{\sqrt{n}}$

Section A — Probability & Bayes 20 pts

Q110 pts

An incident is truly critical with probability 0.08. A detector flags 90% of critical incidents and 12% of non-critical incidents. Compute $P(\text{critical} \mid \text{flagged})$.

Bank ref: Q2

Solution

$P(\text{flagged}) = P(F \mid C)P(C) + P(F \mid \neg C)P(\neg C) = 0.9 \times 0.08 + 0.12 \times 0.92 = 0.072 + 0.1104 = 0.1824$

$P(C \mid F) = \frac{0.072}{0.1824} \approx 0.395$

Even with a 90% detection rate, the low base rate (8%) means most flagged incidents are false positives. This is the classic base-rate neglect trap.

Q210 pts

A retrieval agent calls the search tool with probability 0.7. Conditional on calling search, it answers correctly with probability 0.85. Conditional on not calling search, it answers correctly with probability 0.40. Compute the overall probability of a correct answer, and compute $P(\text{used search} \mid \text{correct})$.

Bank ref: Q3

Solution

$P(\text{correct}) = 0.7 \times 0.85 + 0.3 \times 0.40 = 0.595 + 0.12 = 0.715$

$P(\text{search} \mid \text{correct}) = \frac{P(\text{correct} \mid \text{search}) \cdot P(\text{search})}{P(\text{correct})} = \frac{0.595}{0.715} \approx 0.832$

The law of total probability decomposes overall performance by pathway. The inverse conditional tells you: among correct answers, 83% came via search.

Section B — Binomial, Sums & CLT 20 pts

Q310 pts

Retrieval latency $R$ has mean 200 ms and variance 2500. Generation latency $G$ has mean 800 ms and variance 10000. Assuming independence, what are the mean and variance of $T = R + G$?

Bank ref: Q33

Solution

$E[T] = E[R] + E[G] = 200 + 800 = 1000$ ms

$\text{Var}(T) = \text{Var}(R) + \text{Var}(G) = 2500 + 10000 = 12500$

Under independence, means and variances both add. Standard deviations do not add.

Q410 pts

An LLM judge scores 40 agent responses independently. Each score has mean 6 and variance 9. Use the CLT to approximate the distribution of the sum of all 40 scores, and approximate the probability that the sum exceeds 270.

Bank ref: Q37, Q38

Solution

Total $S = \sum_{i=1}^{40} X_i$. By CLT:

$E[S] = 40 \times 6 = 240, \quad \text{Var}(S) = 40 \times 9 = 360, \quad \text{SD}(S) = \sqrt{360} \approx 18.97$

$S \;\dot\sim\; N(240, 360)$

Since $S \;\dot\sim\; N(240, 360)$, we can express: $P(S > 270)$. The value 270 is about 1.6 standard deviations above the mean — so the probability is small (around 5–6%).

Section C — Beta-Binomial 20 pts

Q510 pts

You start with prior $\text{Beta}(30,10)$ for an intent classifier's accuracy on a known customer domain. On a new sample of 12 cases, the classifier gets 9 correct. What is the posterior distribution and posterior mean?

Bank ref: Q13

Solution

Prior: $\text{Beta}(30, 10)$. Data: 9 successes, 3 failures.

Posterior: $\text{Beta}(30+9,\; 10+3) = \text{Beta}(39, 13)$

Posterior mean: $\frac{39}{39+13} = \frac{39}{52} = 0.75$

Q610 pts

A team says, "We have 85% accuracy on Customer A, so the prior for Customer B should be $\text{Beta}(85,15)$." What hidden assumption are they making, and when might it be dangerous?

Bank ref: Q24

Solution

They are assuming Customer B's domain is as similar to Customer A's as 100 pseudo-observations would warrant. A $\text{Beta}(85,15)$ prior has effective sample size 100, meaning 100 new observations would be needed to move the posterior substantially. If Customer B's distribution differs, this overly strong prior resists updating and can lead to dangerously wrong deployment decisions.

The hidden assumption: "Customer A is so similar to B that seeing 85 successes in 100 trials on A is as informative as running those trials on B."