Hindsight·Agent Memory·Knowledge Graph

What is Hindsight? Agent memory, knowledge graph and use cases

Hindsight is a memory system for AI agents. This guide explains why agents need memory, why entities and a knowledge graph help, how the operations retain, recall and reflect work, how Hindsight differs from RAG, and its use cases.

By Rahmatullah Musawi · 7 October 2026 · updated 8 November 2026 · 8 min read · sources checked 7 October 2026

One short exchange runs through this article as the example. A user tells an agent: "Alice joined Google in March, she loves the research team." The agent replies: "Noted, I'll suggest her for the ML project."¹ It comes from the Architecture Deep Dive, the interactive figure of the Hindsight documentation that is shown below.

The four interactive figures in this article are the documentation's own. The Hindsight repository, which contains them, is published under the MIT License.⁴ This article describes the documentation labelled version 0.10, as of October 2026.¹

Why AI agents need memory

An AI agent without memory starts every conversation from zero: it has no context about who you are, what was discussed or what it has learned.¹ The Hindsight documentation says the problem is harder than it looks, and names four points:¹

• Time. "What did Alice do last spring?" asks about a period, and text that merely sounds similar does not answer it.
• Connected facts. Knowing "Alice works at Google" and "Google is in Mountain View" should allow an answer that was never stored directly.
• Consolidation. A coding assistant that remembers "the user prefers functional programming" should consolidate it into an observation and weigh it when it makes recommendations.
• Point of view. The same information means different things to memory banks with different personalities. A memory bank is the store of one user or agent.

What is Hindsight?

Hindsight is a memory system for AI agents. It does not store conversations: it extracts what was said into typed facts and then builds further knowledge on them.¹ The original text is kept as a source document, but recall and reflect search the memories, not the documents.⁵

Apps reach Hindsight's three operations through SDKs, MCP or plain HTTP.¹ All three work on one memory bank, which is isolated from the others and stored in a Postgres database with the pgvector extension.¹

The figure runs the example through all three operations:

One detail is easy to miss: the bank already held the observation "Alice works at Microsoft". After retain, a background worker updates that observation and adds no second one, and both facts stay as its sources.¹

A bank holds five types of memory. The first two are extracted from what the agent sends; the other three are built from them.¹

Entities and the knowledge graph

A knowledge graph is a network of real-world entities and the relationships between them: any object, place or person can be a node, and an edge defines the relationship between nodes.⁷ Hindsight links each fact to the entities it mentions (people, organizations, places, products and concepts) and to the other facts that share them.¹⁵

These links make a question answerable from two facts that were never stored together.¹ Microsoft Research writes that the majority of RAG (retrieval-augmented generation) approaches search by vector similarity, and that baseline RAG "struggles to connect the dots" when answering a question requires traversing separate pieces of information through their shared attributes.⁸ Hindsight's graph search returns facts that are not similar to the query in meaning or wording but are connected to it through the graph.²

The documentation shows this with three stored facts and one question. A search by similarity retrieves only the facts about Alice; the graph leads from Alice to Project Atlas, then to Kubernetes and the outage.³

Example: three linked facts answer one question

The question and the three stored facts of the documentation's example. The name beside a line is the entity that links the two parts. The numbers match the notes.

1. The fact about AliceA search by similarity retrieves the facts about Alice only.³
2. Linked through Project AtlasFacts that mention the same entity are linked⁵, so the graph leads from the first fact to this one.³
3. Linked through KubernetesThis fact names the outage, where the documentation's path through the graph ends.³

One real memory bank

The graph of one memory bank in Hindsight's own interface: 172 memories and 5,364 links between them, coloured by the kind of link (semantic, temporal, entity or causal).¹

Hindsight builds this graph when the agent stores something with retain(). The figure follows two texts about the same person:⁵

Two things happen to entities and facts on the way in. First, different spellings become one entity: "Alice", "Alice Chen" and "Alice C." are matched by the similarity of the names, supported by which entities appear together and when. This can go wrong: on a bank with a lot of history, a short new name that resembles an existing entity, and appears alongside entities that the existing one is already associated with, can be absorbed into it.⁵

Second, the facts are connected by four kinds of links:⁵

Hindsight also tracks two times: when an event happened and when the bank was told about it. A question about 2024 then finds an event that was reported in 2025. A general fact such as a preference has no time of its own.⁵

Recall and reflect

The two reading operations differ in what they return: recall returns memories, and reflect returns an answer.¹

Recall: four searches, one ranking

No single search method handles every kind of question, so recall runs four searches in parallel: by meaning, by exact words, through the entity graph and by time. The search by time runs only when the query names a date.²

The four result lists are merged by rank, not by score, so a memory that several searches agree on ranks higher. A cross-encoder then reads the query and each of the top candidates, 300 by default, as a pair. Three small multipliers adjust its score: how recent the memory is (at most ±10%), how close it is to a time window in the query (±10%) and, for observations, how much evidence supports it (±5%).²

Memories are taken from the top until max_tokens is used up, 4,096 tokens by default.²

Reflect: an answer with its sources

Reflect runs an agent loop over the bank. The agent decides what it needs and calls its own tools, for up to ten rounds, and it cannot answer before it has retrieved something.¹

The loop reads the most refined knowledge first: mental models and knowledge pages, then observations, then raw facts through recall, and the source chunk or document when it needs the exact wording.¹ An observation is marked stale when new facts are still waiting to be consolidated; the agent then verifies it against the facts.⁶ Only memories that were actually retrieved can be cited.¹

How a bank answers depends on three settings. They shape reflect only: recall returns the same memories whoever is asking.¹

Speed of the three operations

Hindsight does the slow work when it writes, so that reading is fast: facts are extracted, embedded and linked during retain and not during a search. The comparison table in the documentation gives these typical latencies:⁹

Extracting facts is structured work, so retain does not need a large model: the documentation recommends the model gpt-oss-20b.⁹

Hindsight, RAG and use cases

Traditional RAG retrieves the documents that are similar to a query. The documentation compares the two like this:³

The documentation recommends RAG for questions over a static set of documents and for search without time requirements. It recommends Hindsight for AI assistants with persistent memory, for applications that track entities, for a consistent disposition and for questions about time such as "last month".³

For coding agents the overview says "18 harnesses, one install" and shows Claude Code, Cursor and Codex among them.¹ For reflect it gives disposition presets for customer support, code review, legal analysis, therapy or coaching, and research assistance.⁶

The bottom line

Hindsight gives an AI agent a memory that is built from facts, entities and links, not from saved conversations. Retain writes it, recall searches it in up to four ways, and reflect answers from it with sources. For questions over a static set of documents, its own documentation recommends plain RAG.

Sources

1. Overview Hindsight docs
2. Recall: How Hindsight Retrieves Memories Hindsight docs
3. RAG vs Memory Hindsight docs
4. LICENSE Hindsight repository on GitHub
5. Retain: How Hindsight Stores Memories Hindsight docs
6. Reflect: Agentic Reasoning with Disposition Hindsight docs
7. What is a knowledge graph? IBM
8. GraphRAG: Unlocking LLM discovery on narrative private data Microsoft Research
9. Performance Hindsight docs

Written by Rahmatullah Musawi with the help of an AI assistant.