Session 12

Conversational "X"

4 hours · Week 12 · conversational approaches to extraction, clustering, exploration, and evaluation

Framing: Conversational systems are not only chatbots. They are interfaces for steering extraction, clustering, exploration, recommendation, evaluation, and decision support when the goal is underspecified.

Learning Objectives

Explain when a conversational workflow is useful for knowledge discovery
Use dialogue to refine representations, similarity criteria, clusters, and labels
Design conversational extraction and abstraction workflows
Understand conversational clustering and human-in-the-loop preference elicitation
Evaluate conversational systems when there is no single fixed ground truth

Core Story

Conversation is a control surface

Many knowledge discovery tasks start with an underspecified goal: "find themes," "organize these documents," "extract the important facts," or "show me what is surprising." A conversational interface lets the analyst iteratively steer the system instead of specifying the full target upfront.

The pattern

Represent something, propose a structure, ask for feedback, update the representation or objective, and repeat until the result is useful enough.

Conversational Approaches To...

Task	Conversation helps by	Typical feedback
Extraction	Clarifying schema, edge cases, and acceptable evidence	"Split this field," "ignore boilerplate," "include citations"
Clustering	Aligning clusters with a human concept of usefulness	"Merge these," "separate these," "name this group differently"
Search and recommendation	Eliciting preferences that are hard to encode upfront	"More like this, but cheaper," "not this category," "show diverse options"
Graph exploration	Navigating entities, relationships, and suspicious patterns	"Expand this neighborhood," "explain this path," "hide weak links"
Evaluation	Turning vague quality concerns into concrete checks	"This is wrong because...," "this failure type matters more"

Design Questions

What can the user change?

The conversation must expose meaningful control: representation choice, cluster granularity, ranking criteria, extraction schema, graph neighborhood, or evaluation rubric.

What does the system remember?

Feedback is only useful if it becomes state: constraints, examples, rejected alternatives, labels, preferences, or revised objectives.

How do we know it is improving?

Conversational workflows need progress measures: fewer unresolved cases, higher user satisfaction, more stable clusters, better agreement with held-out labels, or lower review effort.