Right context, wrong assumption: what Morgan Stanley learned about prompting at scale

Morgan Stanley manages roughly $4 trillion in client assets and employs around 16,000 financial advisors. In 2023, it partnered with OpenAI to deploy a GPT-4-powered internal assistant called the AI @ Morgan Stanley Assistant, designed to help advisors retrieve information from the firm's library of over 100,000 research documents, compliance briefs, and investment guidelines. The volume of internal knowledge was not the problem. The problem was figuring out what to actually give the model in each query.

Financial advisors do not ask uniform questions. One might need a specific fund's current yield alongside its ESG classification. Another needs the compliance rules for recommending that fund to a particular client tier. The initial instinct, shared by many teams deploying retrieval-augmented generation (RAG) systems, was to retrieve more documents and pass them all in. More coverage, the reasoning went, meant fewer gaps in the answer. That instinct turned out to be wrong in a costly and instructive way.

What Morgan Stanley actually did

The engineering team, working with OpenAI, moved away from bulk document retrieval toward what they described as precision retrieval: identifying not just relevant documents but the specific sections and metadata that matched the intent of the query. This required prompt architects to be explicit about query decomposition, breaking a single advisor question into its component parts before retrieval even began.

A question like "What are the current tax-efficient income options for a high-net-worth client in the 37% bracket?" decomposes into at least three distinct retrieval tasks: current product eligibility, tax treatment by bracket, and any compliance constraints on the advisor's specific license category. Passing the full question to a retrieval system and hoping for a coherent chunk was not the solution. The prompt had to frame each sub-intent separately.

The mechanics involved a few specific choices. First, the system prompt given to the model was stripped of generic instructions ("you are a helpful financial assistant") in favor of role-specific framing that matched the actual decision the advisor was making at that moment. Second, retrieved context was ranked not by semantic similarity alone but by recency and document authority, because a 2019 product brief is not equivalent to a 2025 compliance update even if they discuss the same fund. Third, the model was explicitly instructed on what to do when the retrieved context was insufficient: say so and indicate which type of document was missing, rather than synthesize an answer from adjacent material.

This last point deserves attention. One of the more common prompt designprompt designPrompt engineering is the practice of designing and refining text inputs to guide large language models toward accurate, relevant, and reliable outputs.View full definition → failures is leaving the model no exit. When context is thin, a model without explicit instructions will use what it has. That behavior looks like helpfulness but produces confident-sounding answers built on incomplete evidence.

The results

Morgan Stanley has not published granular performance metrics from this deployment as of mid-2026, so specific accuracy improvement figures should be treated cautiously. What the firm has stated publicly is that the assistant reached adoption among a significant share of its advisor population within months of launch, with advisors describing it as replacing what had previously been 20-to-40-minute manual research tasks.

The more telling signal came from the failure modes that did not materialize. In financial services, the risk of a model hallucinating a fund characteristic or misquoting a compliance rule is not abstract. Morgan Stanley's deployment has not generated the kind of public compliance incident that would follow from systematic model confabulationconfabulationA hallucination is when an AI model generates output that is fluent and confident but factually wrong, fabricated, or unsupported by its source data.View full definition →. That is not accidental. The context design choices, specifically the instruction to flag retrieval gaps rather than fill them, reduced the surface area for invented content.

For reference, Google's internal studies on RAG systems (published through Google DeepMind research) have consistently shown that context windowcontext windowThe context window is the maximum amount of text (measured in tokens) a language model can process at once, including both the input prompt and the generated output.View full definition → stuffing, adding more retrieved material without ranking or filtering, can degrade answer quality by introducing noise that pulls the model's attention away from the most relevant passage. Morgan Stanley's approach aligns with those findings in practice.

What transfers to your work

The Morgan Stanley case is not primarily a story about enterprise AI infrastructure. The principles apply at the individual prompt level, which is where most professionals actually operate.

The first transferable discipline is decomposing your question before you write the prompt. If your question has more than one decision embedded in it, the model will prioritize one and underserve the others. Write a sentence describing exactly what decision you are trying to make, then check whether your prompt actually addresses that decision or something adjacent to it.

The second discipline is choosing context deliberately. If you are pasting in a long email thread, a document, or meeting notes, identify which three or four sentences actually bear on your question and include those rather than the full block. This is not about saving tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →. It is about signal-to-noise ratio. A model reading a 2,000-word email to answer a question about the Q3 budget variance will spend its attention on material that has nothing to do with that variance.

The third is giving the model a clean failure path. Add a line to your prompt that says something like: "If you don't have enough information to answer this accurately, tell me what's missing rather than guessing." Models do not do this by default. They are trained to be helpful, which in practice means they will produce an answer whether or not the answer is justified by the context you provided.

Where your situation differs from Morgan Stanley's: they had engineers building retrieval pipelines and ranking algorithms. You probably do not. The manual equivalent is simple: read your own prompt before you send it and ask whether the context you included actually contains the answer, or whether you are hoping the model will infer it from surrounding material. That single habit eliminates a large share of poor outputs.

The discipline is not about writing longer prompts. It is about knowing why each piece of context is in the prompt at all. If you cannot state a reason, cut it.