ThoughtTrace: Understanding User Thoughts in Real World LLM Interactions 🌐 Project Page · 📄 Paper · 💻 Code Conversational AI has reached billions of users, yet existing datasets capture only what people say , not what they think . ThoughtTrace is the first large scale dataset that pairs real world multi turn human–AI conversations with users' self reported thoughts : their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, and 10,174 thought annotations collected across 20 language models. Our analysis shows that ThoughtTrace captures long horizon, topically diverse interactions, and that thoughts are semantically distinct from messages, difficult for frontier LLMs to infer from context, and tied to conversation stages. Thoughts also provide actionable signals for user behavior prediction (+41.7% relative gain) and model alignment (+25.6% win rate). What's Inside a Thought? Every thought is anchored to a single message and falls into one of two kinds: Reasons are attached to user messages and capture the underlying motivation, prior context, expectations, or constraint that shaped the prompt. They are organi…
Runs entirely in your browser via DuckDB-Wasm — this dataset's real data file is loaded once, then queried locally. Nothing is sent to a server.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy