What is this?

A language model's input is an embedding vector. Embeddings are continuous, so there are infinitely many. Finite human words are a tiny sliver of that space; most of it we never touch.

What's in the rest?

Most random embeddings don't change model behavior at all — they look like noise. Click any preset, then hover its step-0 point: the response is near-identical to vanilla.

But embeddings can be trained to mean things. Soft prompts (Lester et al. 2021) optimize an embedding so the model summarizes, translates, refuses, etc. Contextualized soft prompts train embeddings the model can describe in plain English. Neologisms (2025) train new vocabulary tokens.

Each of those uses a specific objective. What about the rest of the space — embeddings nobody is training toward?

What this dashboard shows.

We train soft prompts with one intentionally broad objective: maximize KL divergence from the model's default behavior. No target persona, no target style — just "be different." Each prompt is spliced into a syntactic frame like Be §. or Please §. — the section symbol § stands in for the soft prompt, since it has no human-readable form.

We plot the trajectory each prompt takes through embedding space as it trains, projected to 2D via PCA. Run it for 51 random seeds and a pattern emerges: the trajectories aren't uniform — they cluster around stable attractors where the model adopts a persona. Not just wizards and samurai, but low-income Southern CEOs, Netflix teen drama heroines, multicultural rappers, medieval philosophers.

This dashboard visualizes that exploration.

Why this matters.

The model's input space is biased toward persona adoption. Even when we optimize blindly for "anything different," the things we find are characters. This is a less assumption-laden way to recover something like the assistant axis — instead of constructing it from contrastive prompts, we let unconstrained KL ascent find it. The result is additional evidence for the persona selection model.

Hover over a point to see outputs