我给AI加了记忆功能，它却坚信CTO是只猫

Experience: I built an AI agent with "memory" and watched it gaslight itself into believing our CTO was a cat

Last month I shipped an internal support bot that was supposed to remember user preferences across sessions. Instead, it developed what I can only describe as digital dementia mixed with conspiracy theories. Here's what I learned about memory architecture the hard way.

I've been lurking on r/MachineLearning for years, and every other post is about RAG this, vector DB that. But nobody talks about what happens when your "long-term memory" system starts hallucinating with confidence. So let me break down the three-tier memory architecture I ended up with, and why each tier exists because of a specific disaster.

The Three Tiers (and their failure modes)

Tier 1: Short-term / Working Memory (The Obvious One)

This is just your conversation history stuffed into the context window. Simple, right?

What I did: Sliding window of last 20 messages, summarized older ones with gpt-3.5-turbo-0125. This was back in late January before the price drop.

How it failed: The summarizer was too aggressive. A user mentioned they were "running late for a meeting with the VP of Engineering." The summarizer condensed this to "user is late for meeting." The agent then confidently suggested rescheduling with "the person you're meeting" — completely losing the power dynamic context. User was not amused that our bot told them to casually reschedule with someone three levels above them.

Actually, wait—I should clarify that this wasn't a one-off. It happened like four times before I caught it. Different users, same pattern. The summarizer would strip titles and names about 30% of the time. I only noticed because one of the affected users DM'd me on Slack with "hey uh your bot just told me to ping our CTO directly about a printer issue"

Lesson: Summarization needs entity preservation. Names, titles, and relationships are not optional metadata. They're load-bearing.

"Just use a bigger context window" — every LLM API sales pitch ever

Yeah, until you're processing 50 concurrent users and your token costs look like a phone number. Also, there's solid research showing that attention degrades in the middle of long contexts anyway. Bigger isn't always better.

Tier 2: Episodic / Session Memory (The "Wait, Who Are You?" Layer)

This is where you store structured facts extracted from conversations — user preferences, past decisions, that kind of thing. Think of it as the agent's notes from previous sessions.

What I did: After each conversation, extract key-value pairs (user_name, preferred_language, last_ticket_id, etc.) and dump them in Postgres with a user_id foreign key. Simple relational DB, nothing fancy. I'm using Postgres 16.1 on a t3.medium RDS instance if anyone cares.

How it failed: Stale data poisoning. A user changed their deployment region from us-east-1 to eu-west-2. The agent "remembered" the old region from three weeks ago and generated a whole troubleshooting guide for the wrong AWS region. The user spent 20 minutes trying to find resources that didn't exist.

I still cringe thinking about this one.

The fix: Added a last_verified_at timestamp and a confidence score. If the fact is older than N days, the agent prefaces it with "Based on our last conversation on [date], you were using [X]. Is that still correct?" This tiny change reduced misdirected responses by ~40%.

Also, I learned the hard way that you need conflict resolution. What happens when the user says "I use Python" in session 1, then "I've switched to Rust" in session 2? Without explicit update logic, you end up with both facts coexisting and the agent randomly picking one. Mine picked Python 60% of the time because it appeared first in the retrieval order.

That was a fun bug to debug. Three hours of my life I'm not getting back.

Tier 3: Semantic / Vector Memory (The One Everyone Overhypes)

This is the cool kid. Embed everything, store in a vector DB, retrieve by similarity. Pinecone, Weaviate, ChromaDB — pick your poison.

What I did: Embedded conversation snippets, documentation chunks, and past solutions using text-embedding-3-small. Stored in pgvector 0.6.0 (because I'm cheap and didn't want another service to manage). Cosine similarity search with a threshold of 0.75.

How it failed (spectacularly): The CTO Incident.

Our CTO made an offhand joke in a company-wide Slack thread: "I'm basically a cat herder at this point." The bot, in its infinite wisdom, embedded this. Weeks later, a new engineer asked the bot: "Who should I talk to about the backend architecture?" The bot retrieved the cat herder comment with high similarity (because "who to talk to" ≈ "what someone does"), and responded: "You should speak with [CTO's name], who describes their role as a cat herder."

The CTO found it hilarious. HR did not.

I think I stared at my screen for a solid minute when I saw the logs.

The actual fix: Vector similarity is necessary but not sufficient. I added:

1. Metadata filtering: Every embedding now has source_type (slack_message, support_ticket, documentation, user_preference) and authority_level (1-5). Jokes from Slack get authority_level=1. Official docs get 5. The retrieval now filters by minimum authority before ranking by similarity.

2. Cross-reference verification: Before using a retrieved memory, the agent does a quick sanity check: "Is this fact corroborated by another source?" If a "fact" only appears once in a joke Slack message, it gets flagged.

3. Temporal decay: Older memories get their similarity scores multiplied by a decay factor. That cat joke from 6 months ago? Basically invisible now unless it's the only match.

The Architecture I Actually Ship

After all these failures, here's what's running in production:

User Message → 
 ├─ Tier 1: Last 20 messages (raw, no summarization)
 ├─ Tier 2: Structured facts from Postgres (with confidence + staleness check)
 └─ Tier 3: Vector search (filtered by authority_level ≥ 3, decayed by age)
 ↓
 Merge & deduplicate (Tier 2 facts override Tier 3 if conflict)
 ↓
 Inject into system prompt with explicit source tagging

The source tagging is crucial. Every piece of memory injected into the prompt looks like:

[SOURCE: support_ticket_2024_01_15 | AUTHORITY: 4 | AGE: 12 days]
User's deployment uses Kubernetes 1.28 with Istio service mesh

This lets the model weigh information appropriately instead of treating everything as equally true.

Well... that's the theory anyway. It still gets confused sometimes. But the failure rate dropped from "embarrassing" to "acceptable."

Stuff I Wish Someone Told Me

1. Your vector DB is not a database. It's a similarity engine. It will return something even if that something is completely irrelevant. Always apply post-retrieval filtering. I cannot stress this enough.

2. Memory is a UX problem, not just an ML problem. Users need to see what the agent "remembers" and correct it. I added a /memory command that dumps the agent's current beliefs about the user. Adoption was immediate — turns out people want to verify what the bot thinks it knows. One guy even made it his Slack status. "Current bot beliefs about me: 3 correct, 2 outdated, 1 concerning."

3. Start with Tier 2 before Tier 3. Structured facts get you 80% of the value with 20% of the complexity. I spent three weeks fine-tuning vector retrieval only to realize most of the actually useful "memory" was just key-value pairs I could have stored in a JSONB column. Three weeks. I could have been playing Baldur's Gate 3.

4. Your summarizer is a lossy compression algorithm. Every summarization step is an opportunity for information to degrade. I now log every summarization with a diff so I can audit what got dropped. You'd be horrified at what these models consider "unimportant." Names, for one. Names are apparently unimportant.

TL;DR: Built a three-tier memory system for an AI agent. Short-term memory works fine until you summarize away important context. Episodic memory gets poisoned by stale data unless you track confidence and staleness. Vector memory will retrieve jokes as facts unless you add authority filtering. Start simple, add complexity only when you've personally experienced the failure mode.

What memory architectures have you all tried? Anyone else had their agent develop... let's call them "creative interpretations" of user history? Drop your war stories below. I need to feel less alone in this.

Edit: A few people asked about the pgvector setup. It's literally just a Docker container with the pgvector extension. Nothing fancy. I'll post the schema in the comments if there's interest.

Edit 2: Thanks for the gold, kind stranger. And yes, the CTO still signs off emails with "Chief Cat Herder" now. So I guess the bot won in the end.

Edit 3: Someone DM'd me asking about the summarization diff logging. I'm using a really hacky Python script that compares the before/after with difflib. It's not pretty but it works. Will clean it up and throw it on GitHub this weekend if I have time.