The word "summarization" has become a catch-all for what AI does to documents. Feed a paper into a tool and it returns a summary: the main argument, key findings, a few bullet points. This is useful for deciding whether a source is worth reading in full. What it does not do is tell you why that source matters to your particular research question, which of its claims support or complicate your developing argument, or how it sits relative to three other sources you added yesterday that address a similar point from a different angle.
That is annotation. And the difference between a summary and an annotation is not a matter of length or format. It is a matter of perspective. A summary is produced from the source outward: it describes what the document contains. An annotation is produced from the researcher's question inward: it describes how this document serves, complicates, or frustrates a specific inquiry.
When we were building Pilgrim's approach to AI-assisted reading, distinguishing these two tasks cleanly was one of the core design decisions. Getting this wrong produces tools that feel useful for individual documents but do not help when the work shifts to synthesis across a research corpus.
What Summarization Actually Does Well
Let us be precise about where AI summarization is genuinely valuable, because it is good at some things and the good cases matter.
Triage is the primary use. A researcher working from a 60-item reading list does not have time to read every document in full before deciding how to allocate reading time. AI summaries help here: they give you enough information about a source to decide whether it belongs in the project at all, and if so, where in your reading priority it sits. This is real value. Triage is one of the most time-intensive parts of early-stage research, and compressing it makes the overall process faster.
Comprehension support is the secondary use. Dense technical papers, documents in a second language, or material in an unfamiliar field all become more accessible with a plain-language summary alongside them. The summary does not replace reading but reduces the activation energy needed to engage.
What summarization does not do: it does not situate the source in relation to your argument. It does not compare the source's claims against other sources you are already tracking. It does not flag where the source diverges from a position you were about to commit to. The model producing the summary has no access to your project context. It knows only the document in front of it.
What Annotation Requires
A good annotation requires four pieces of context that a summarization model does not have.
First, your research question. Not the general topic, but the specific question you are trying to answer. "What is the impact of zoning policy changes on residential supply in mid-sized American cities" produces a different annotation of the same document than "what explains variance in housing construction rates across comparable regulatory environments." The claim that matters, the evidence worth extracting, the framing worth noting: these are all determined by which question you are asking.
Second, your current working hypothesis. If you have already formed a tentative position, a good annotation tells you whether this source supports it, complicates it, or introduces a consideration you had not accounted for. Without knowing your hypothesis, the annotating system cannot tell you whether a finding is confirming or disconfirming from your perspective.
Third, your source context. If Source A has already established a claim and Source B is making a similar claim from a different methodological base, noting that overlap is useful. If Source C contradicts Source A on a point you consider foundational, flagging that tension immediately is more valuable than any summary of Source C on its own terms.
Fourth, your drafting context. If you are mid-draft and working on a section that needs empirical support for a specific claim, the annotation that helps you is one that tells you what this source can provide that section, not a general account of what the source argues overall.
None of these are available to a model that receives only the document.
Why Tools Conflate the Two
The reasons are mostly architectural. Summarization is a stateless task: one document in, one summary out. No state is required beyond the current document, which makes it straightforward to build. Annotation is a stateful task: you need the document, the research question, the working hypothesis, and ideally the contents of the other sources in the project. Maintaining and querying that state requires a different architecture and a different interaction model.
Most early AI reading tools were built around the stateless summarization model because it was the path of least architectural resistance. The limitation did not matter much when the tools were primarily used for single-document reading. It becomes significant when researchers expect the tools to support the more complex task of building an argument across a corpus.
There is also a user-facing reason. Summaries are immediately evaluable: you can check whether they are accurate by comparing them to the source. Annotations are more difficult to evaluate, because a good annotation depends on context that only the researcher holds. "This source complicates your hypothesis because..." requires that the researcher agree about what the hypothesis is. This makes annotation harder to demo and harder to measure quality against, which creates less pressure to build it well.
What We Built Instead
Pilgrim's annotation engine works from the research project context, not just the document. When you add a source to a project, the system has access to the questions you have defined for the project, the claims you have extracted from other sources already in the project, and the sections you are currently developing in draft. The annotation it produces for the new source is oriented toward those project elements, not toward a general description of the source's contents.
In practice, this means that two researchers adding the same paper to two different Pilgrim projects would receive different annotations, because the relevance of any given claim in the paper depends on what they are trying to argue. A claim about replication methodology is peripherally relevant to a researcher studying outcomes and central to a researcher studying research design. The document is the same; the annotations are not.
We are not suggesting that AI-generated annotations eliminate the need to read sources. An annotation is a scaffold, not a substitute. Researchers who rely entirely on annotations without reading critically are making a different kind of error. What project-aware annotation does is make reading more efficient: it directs your attention to the parts of a source that are most consequential for your specific inquiry and flags tensions with your existing source base that would be easy to miss when reading in isolation.
A Note on What This Does Not Solve
The distinction between annotation and summarization matters most for researchers who have clearly defined their questions and have a developing argument to annotate against. For researchers at the very beginning of a project, when the question is still forming and there is no hypothesis to support or complicate, the project-context approach has less to anchor to. Summaries are more useful in that early, exploratory phase precisely because they make no assumptions about what you are looking for.
The practical implication: annotation and summarization are complements, not substitutes. Triage and early exploration benefit from summarization. Active hypothesis development and argument construction benefit from annotation. Tools that offer only one of the two are useful for a fraction of the research workflow. Building both in the same system, and being clear about which is doing which job, is what makes the overall process work.