Hindsight's Agent Memory Model Is Better Than the Benchmark Page Selling It
Vectorize's agent memory server ships retain, recall and reflect over four distinct memory types. The architecture is worth copying. The leaderboard that sells it names no competitors.
Every agent memory project opens by telling you it is not RAG. Hindsight's README does it too, and then does something almost none of them do: it publishes a taxonomy specific enough that you can check the claim yourself.
Four memory types, and the two that matter are the first two. World facts are "objective facts received from external sources," and the documentation's example is "Alice works at Google." Experience facts are "the agent's own actions and interactions," example given: "I recommended Python to Bob." Same store, same retrieval path, permanently different provenance.
That single distinction is the most useful idea in the repo, and it is worth more than the benchmark scores on the front page.
Why the split is the whole point
Here is the bug that eats agent memory systems, and I have shipped it myself. Your agent makes a recommendation. The recommendation goes into the conversation. The conversation gets summarized into memory. Three sessions later the agent recalls that memory and treats it as a fact about the world, because by then the sentence "Python is the right choice for Bob" has lost every marker of who said it and on what basis.
The agent is now citing itself as a source. It will defend that position, and it will defend it more confidently each time it recalls it, because repetition in a memory store reads as corroboration.
Storing experience facts in a separate class, permanently labeled as things the agent did rather than things that are true, is a structural fix for that. Not a prompt instruction telling the model to be careful. A type distinction the retrieval layer can act on. If you build nothing else from this repo, build that.
The other two types are more conventional. Observations are "automatically synthesized knowledge," created during retain operations when facts get analyzed and consolidated, with evidence tracking pointing back at the facts that produced them. Mental models are "pre-computed, curated summaries for common queries," and those require a separate creation or refresh operation rather than appearing on their own. That last detail matters operationally: mental models are a cache you maintain, not a thing that happens.
The mechanism, not the pitch
Hindsight has been near the top of the Trendshift daily board all afternoon, reading number one at 3:21pm ET and number six an hour later, which is a fair reminder that Trendshift publishes momentum scores rather than star totals and that a rank is a snapshot with a short shelf life. The thing that kept moving it is release traffic: its own paperclip integration cut integrations/paperclip/v0.4.0 at 11:45 UTC today, with an obsidian integration release in the same minute. It is MIT licensed, with the LICENSE file reading "Copyright (c) 2025 Vectorize AI, Inc.", and the shields badge puts it around 30,000 stars.
Three operations carry the system. Retain stores information into a memory bank, extracting facts, entities, and temporal data along the way. Recall searches with four parallel strategies rather than one: semantic vector search, BM25 keyword matching, graph traversal over entity and temporal links, and time range filtering, with the results merged and reranked. Reflect performs reasoning over retrieved memories, guided by a bank's mission, directives, and disposition traits.
A memory bank is "a dedicated memory space for a specific agent or context," holding its own stored memories, entity relationships, search indices, and that mission and directive configuration. Which means the unit of isolation is the bank, and your multi-tenant story is a bank-per-tenant story.
The four-strategy recall is the part I would defend against a simpler design. Pure vector search fails on exact identifiers, proper nouns, and anything a user typed once with unusual phrasing, which is why BM25 is in there. Pure keyword fails on paraphrase. Neither handles "what did we decide last Tuesday," which is what the time range filter and the temporal graph links are for. Merging four retrievers and reranking costs latency, and it is the correct trade for a memory system where a miss is invisible and a false negative looks like amnesia.
Storage is PostgreSQL with pgvector, or Oracle AI Database 23ai for enterprise deployments with what the README calls full feature parity.
Put this into practice
The fastest honest evaluation takes one command and about twenty minutes.
docker run -it --pull always --name hindsight --restart unless-stopped \
-p 8888:8888 -p 9999:9999 \
-e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-v hindsight-data:/home/hindsight/.pg0 \
ghcr.io/vectorize-io/hindsight:latest
Then pip install hindsight-client -U and point it at localhost.
Do not start by loading a corpus. Start by testing the thing the architecture claims to fix. Create one bank. Retain two items: a world fact your agent learned from a document, and an experience fact describing a recommendation your agent made. Recall both. Look at what comes back and confirm that the provenance survived the round trip and that you can tell, programmatically and not by reading the text, which one your agent authored.
If that works, the second test is decay under scale, which I will come back to, because the vendor publishes the numbers for it and they are the most credible thing on the benchmark page.
Skip all of this if your agent is a workflow with a fixed set of steps. The project's own framing says it is overkill for simple automation, and running a Postgres instance to remember six things is a bad trade you will resent in three months.
Where the benchmark page falls down
The README says Hindsight reached state of the art on LongMemEval as of January 2026, with results independently reproduced by research collaborators at the Virginia Tech Sanghani Center and The Washington Post. Reproduction by named collaborators beats no reproduction at all, and it is not the same thing as an independent audit, because collaborators are chosen and audits are not.
The live benchmarks page is where I lost some confidence. It shows eight dataset results: LongMemEvalS at 94.6%, LoComo10 at 92%, PersonaMem32K at 86.6%, BEAM100K at 75%, BEAM1M at 73.9%, LifeBenchEN at 71.5%, BEAM500K at 71.1%, and BEAM10M at 64.1%. The page states that Hindsight leads every dataset.
It does not show a single competitor's score. There is no head-to-head table, no named rival system, and no date on the run. It describes the Agent Memory Benchmark as "the industry standard for evaluating memory and retrieval systems," which is a claim a vendor makes about a benchmark it reports its own results on. A page can say "leads every dataset" and show you only its own numbers, and those two things together tell you nothing you can act on.
What the page does contain, to Vectorize's credit, is the number a marketing team would have cut. The BEAM series scales the corpus, and the results sag as it grows: 75% at 100K, 71.1% at 500K, 73.9% at 1M, and 64.1% at 10M. That is a downward trend rather than a clean monotonic decline, and the 500K result sitting below the 1M one is itself worth asking about. Publishing roughly eleven points of loss between 100K and 10M is an honest disclosure about where the system stops being reliable, and it tells you more about production behavior than the 94.6% headline does. Read those four numbers and ignore the first one.
Two products, one name
The repo is MIT. The docs describe something else. Hindsight Cloud meters four separate token types: "Retain tokens - Consumed when storing new memories, Recall tokens - Consumed when searching/retrieving memories, Reflect tokens - Consumed when generating AI-powered insights, Mental Model tokens - Consumed when creating, refreshing, or retrieving mental models," running on a credit system the introductory documentation describes without publishing a rate.
So four of the operations you would design your agent around are separately billable on the managed path, and the docs I could reach do not tell you what any of them cost. That is not a trap, and Vectorize is not hiding the open-source core. It does mean that "Hindsight" names two things, and that the architecture decisions you make while evaluating the free Docker image are the same decisions that determine your bill if you later move to the cloud version. An agent that calls reflect on every turn is a design choice on one path and a line item on the other.
Decide which product you are adopting before you write the integration, not after.
What I would actually do
Run the Docker image this weekend and test the world-fact versus experience-fact distinction against your own data. That is twenty minutes and it answers the only question that matters: whether the taxonomy holds up when your agent's output goes back into its own memory.
If it does, you have two options and both are reasonable. Adopt Hindsight and accept operating a Postgres instance beside your agent. Or take the idea, which is free, and put a provenance field on whatever store you already run, because the insight is not patented and the fix is a column.
The benchmark page will not help you choose. Your own two-fact test will.
Sources: vectorize-io/hindsight · Hindsight docs · Hindsight benchmarks · Trendshift daily board
Medium metadata
Title: Hindsight's Agent Memory Model Is Better Than the Benchmark Page Selling It Subtitle: Vectorize's agent memory server ships retain, recall and reflect over four distinct memory types. The architecture is worth copying. Tags: AI Agents, Open Source, Machine Learning, Software Engineering, Databases Canonical: publish on fervorai.dev first, import to Medium from that URL.