Martin Keen compares two ways to give a language model information outside its training: retrieve selected passages, or supply whole documents. His eleven-minute IBM Technology walkthrough makes the choice depend on the task and the size of the collection. A bounded comparison between documents creates different demands from searching a company’s entire knowledge base.

Decide how evidence enters the prompt (00:00–03:49)

  • Retrieval-augmented generation, or RAG, answers using documents retrieved at question time. Keen illustrates a pipeline that breaks source documents into chunks before a user asks anything.
  • An embedding model turns those chunks into numerical representations called vectors. A vector database stores them so the application can search for passages related to a question.
  • The retrieved chunks join the user’s question in the model’s context window, the input space it can use to produce an answer.
  • Long context takes a more direct route. The application supplies the documents themselves and asks the model to find or combine the relevant information.
  • Larger windows make this practical for more bounded collections. Keen asks whether the simpler architecture can replace the retrieval machinery in those cases.

Long context removes a retrieval pipeline (03:56–06:01)

  • Keen lists the parts a RAG application may need: a chunking strategy, an embedding model, a vector database, and a reranker that sorts candidate results.
  • Those representations also need to stay synchronized with the source documents. Each component adds maintenance work and another place for the system to fail.
  • Directly sending a manageable document set to the model removes that separate indexing and search path from the example.
  • Retrieval also creates a way to miss evidence silently. A relevant passage can exist in the collection yet never reach the model because the search returns other chunks.
  • Supplying the full source removes that particular filtering failure. Whether the model actually uses the supplied evidence is a separate question later in the lesson.

Whole-document comparisons need broad coverage (06:08–07:28)

  • Keen’s example compares product requirements with release notes. The question asks which security requirements were omitted from the release.
  • A similarity search can find passages discussing security and requirements. Finding matching snippets does not by itself expose what is missing between the two documents.
  • Keen argues for giving the model both documents in full so it can compare their coverage. The task requires reasoning across the collection, beyond locating one matching passage.
  • This example explains his preference for long context on bounded, global-reading tasks such as analyzing a contract or summarizing a book.

Repeated reading and attention still cost something (07:35–09:42)

  • Sending a large manual with every question makes the model process substantial input repeatedly. RAG pays to index the source, then supplies smaller retrieved portions to each request.
  • Prompt caching, reusing previously computed input state, can offset repeated processing for static content. Keen notes that frequently changing material reduces that opportunity.
  • Fitting information into a window also differs from reliably using it. He describes models missing a paragraph buried in a very large prompt or inventing details from nearby text.
  • Retrieval can focus the model’s input. His example supplies five relevant chunks instead of surrounding them with an entire large document.

Match the architecture to the collection (09:49–10:51)

  • Enterprise collections can exceed even very large context windows. A retrieval layer filters the material down to something the model can accept.
  • Keen recommends long context when the source set is bounded and the question needs broad reasoning over it. He retains retrieval for navigating a much larger knowledge base.
  • His comparison keeps two questions distinct: can the application supply all the evidence, and can the model use it effectively?

Caveat: This is Keen’s architectural comparison, not a benchmark; his vector-search examples do not cover every retrieval design or prove that full-context answers are more accurate.