Knowledge Discovery
Session 07

Creating Useful Representations

4 hours · Week 7

Conceptual position: Session 06 showed that simple representations (sets, shingles, MinHash) answer wording-level similarity well. Harder questions — semantic similarity, user preference, relational structure — demand richer representations. This chapter develops them: embeddings, user/item/product vectors, and graphs.

Learning Objectives

  • Explain why moving beyond lightweight representations unlocks harder similarity questions
  • Compare representations for text, users, movies, products, and graph-structured data
  • Understand embeddings as learned vector representations that capture semantic meaning
  • Design user, item, and product representations for preference and recommendation tasks
  • Recognize graph representations: nodes, edges, attributes, adjacency matrices, and embeddings
  • Choose a representation based on what should count as “similar” or “related”

Core Story

Representation is the modeling choice before the model

Raw objects are not automatically useful for computation. A document, movie, customer, product, or graph first has to be represented in a form that preserves the information we care about.

The same object can have many valid representations. A movie can be a row of metadata, a vector of viewing behavior, a text embedding of its description, or a node in a graph of users and items. Each representation makes some questions easy and others hard.

Key principle
A representation defines what the system is able to notice. Before asking which algorithm to use, ask what information the representation preserves, compresses, or throws away.

Representation Families

Object Representation Preserves Useful for
Document Bag of words / TF-IDF Important terms and frequencies Keyword search, classification, simple retrieval
Document / sentence Embedding vector Semantic meaning learned from data Semantic search, paraphrase retrieval, clustering by topic
User Interaction vector Clicks, ratings, purchases, views Recommendation, segmentation, personalization
Movie / product Content + behavior embedding Attributes and usage patterns Similar-item retrieval, recommendations, cold-start analysis
Network Graph: nodes and edges Relationships and structure Communities, influence, paths, graph clustering
Node / entity Graph embedding Structural role and neighborhood context Link prediction, entity similarity, downstream ML

When to Use What

Session 06 already covered wording-level similarity

Shingles and MinHash handle exact or near-exact reuse (copied articles, duplicated policies, boilerplate). This session picks up where those representations run out of room.

Use embeddings when meaning matters

Embeddings represent objects as vectors whose geometry is learned from data. This is the right representation when two objects can be similar despite using different words: paraphrases, related topics, user preferences, or product intent.

Use graphs when relationships matter

Graphs represent objects as nodes and relationships as edges. They are appropriate when the structure between things is itself the signal: citations, friendships, purchases, dependencies, knowledge bases, and user-item interactions.

Examples to Discuss

Movies

A movie can be represented by genre tags, plot text, cast and crew, user ratings, co-watch behavior, or a node in a user-movie graph. These answer different questions: "same genre," "same story," "watched by similar users," or "connected in the graph."

Products

A product can be represented by structured attributes, title and description text, image embeddings, purchase co-occurrence, or a graph of products, users, and categories.

Documents

A document can be represented as terms, shingles, MinHash signatures, semantic embeddings, entities, citations, or links. The best representation depends on whether we care about copied text, meaning, provenance, or structure.

Hands-on demos

Small Python demos that match this lecture — runnable on a laptop in seconds. Code lives in docs/07_creating_representations/demos/.

D3 · User & movie embeddings

Train tiny 2-D user and movie embeddings on a synthetic 62-user × 50-movie ratings matrix and watch random noise turn into a clean five-genre pentagon. The notebook then reframes the same data as a bipartite graph, trains a GCN on it, and compares the two recommenders side by side. Includes an overfitting / dimension-sweep section that contrasts under-, sweet-spot, and over-fitting empirically.

  • Notebook (.ipynb) — narrated, with markdown explanations and a long discussion
  • Script (.py) — same code, runs end-to-end and saves embeddings_evolution.png
  • README — what each file does

How to run

One-time setup (creates a virtual environment in this folder):

cd docs/07_creating_representations/demos
python -m venv .venv
source .venv/bin/activate           # on macOS / Linux
pip install -r requirements.txt

Open the notebook:

jupyter lab demo_d3_user_movie_embeddings.ipynb

Or run the script directly to regenerate the figure:

python demo_d3_user_movie_embeddings.py

Materials

Companion lecture: MLVU on Embedding Models

Peter Bloem's MLVU course covers embedding models in depth and complements this session well. It walks through how embeddings are trained, what their geometry encodes, and how the same recipe applies across text, users, items, and graphs — the same families we organize this session around.