Cole Medin (16 min) is a year into his personal AI second brain — an agent that builds a knowledge base as he works — and this one answers the question he says he keeps getting: how do you turn a second brain into a team brain?

What actually has to change

  • Scale: with many collaborators, information arrives from everywhere — Slack threads, GitHub repos, docs, wiki spaces.
  • An Obsidian vault of markdown stops working the moment it isn’t just you.
  • Retrieval has to stay accurate over thousands (or millions) of records, not a folder of notes.
  • Every answer needs source citation so you can check the agent isn’t inventing things.
  • And the thing people skip: security and permissioning. Nobody’s team brain gives everyone everything.

The architectural decision: don’t replace the personal agent

There are two paths — keep the agent personal and distribute only the knowledge base, or run one shared agent everyone hits from their own interface. Cole picks the first.

  • Everything you already built stays; the personal system remains the layer you interact with.
  • The team brain is the knowledge base you access, not a new agent.
  • You distribute policy and knowledge; each person keeps their agent’s personality and personal memory.
  • It scales best — distributed agents, individually customizable.

One table, many sources

  • A single documents table: title, text, URL, author, domain, metadata. No table-per-source sprawl.
  • Every connector has one job — fetch from the source, mold it into the document shape, write it.
  • That standard format is what keeps querying simple for every personal agent on top.

The MCP server is the connection point

  • Search documents, search code, fetch a full document, identify the caller — plus a “who knows” lookup that maps a topic back to the author in the company.
  • Attaching it changes nothing in your existing agent architecture; it’s just another tool.
  • Client config is a URL plus an authorization token. That’s the whole integration.

Permissioning: label at ingestion, enforce in the database

  • Each source gets a domain label (ops, marketing) written onto every record as it’s ingested; shared spaces can carry multiple labels.
  • An MCP tokens table: an admin creates the keys and hands them to people.
  • A principals table maps each key to the domains that person can see; leadership gets a special tag for everything.
  • At query time the database resolves identity from the key and filters rows before results ever reach the agent.
  • Live demo: as one user the same question returns “up to 20% off list on an annual contract”; as another, the knowledge base “doesn’t cover anything around” it — invisible, because no match means no access.

The retrieval strategy

  • Keyword search catches exact wording and IDs; semantic search catches related concepts with no shared keywords. Combined and weighted, they cover each other’s gaps.
  • Even after permission filtering, a large table still needs a handful of snippets, not the whole set.
  • Oracle AI Database plus its LangChain integration provides embeddings and both search modes in-database — no external embedding service in the demo stack. (The video is a paid collaboration with Oracle; Cole says the strategy applies regardless of tooling.)

The most important takeaway here, no matter how you build this system, is you need the gate to sit in the database. You cannot have this second brain, the personal part of the system, responsible for the security in any way.