Checking an AI summary means verifying what it says, what it leaves out, and whether its sources support the wording. In financial research, a correct number attached to the wrong reporting period can be as misleading as an invented figure. A summary that turns management’s expectation into a confirmed result creates a different problem, even if every number survives inspection.
Use summaries to organize reading, not replace evidence. Within a workflow for extracting information from financial news with AI, verification should sit between the generated text and any research note, alert, or investment decision. The working rule is straightforward: every decision-relevant statement needs traceable support, and every calculation needs a reproducible method.
Separate Errors From Unsupported Claims
An error contradicts the evidence. An unsupported claim goes beyond it. Suppose an earnings release reports revenue growth of 6%, but the summary says 9%. That is an error. If the summary says growth proves the company is gaining market share, yet supplies no market data, that is an unsupported interpretation. The interpretation might be true. The release alone does not establish it.
Also distinguish both problems from an omission. A summary could reproduce revenue growth accurately while leaving out a reduction in annual guidance. Nothing in the text is false, but the resulting impression is incomplete.
Confident wording does not resolve these distinctions. Generative AI can produce false content and fabricated supporting citations, risks covered in the NIST Generative AI Profile’s discussion of confabulation. Treat fluent prose as presentation, not proof. A polished sentence still needs a receipt.
For review purposes, assign each material statement one status: supported, contradicted, not established, or interpretation. Keep “not established” separate from “false.” Otherwise, the checking process starts making claims it cannot defend either.
Confirm the Source Before Checking the Summary
Start with the material the system actually received. Save the document title, publisher, publication time, retrieval time, URL, and reporting period. If the model processed an attachment or extracted text, retain that version too. A link to the right document does not establish that its tables, footnotes, or final pages reached the model.
Open citations yourself. Confirm that each destination exists, concerns the correct company or event, and contains the claimed evidence. A search result snippet is not a substitute for the underlying passage. Neither is a citation that lands on a relevant document but fails to support the sentence beside it.
Apply the same checks to the publisher and document’s authenticity. The process for verifying financial news sources helps distinguish an original announcement from a repost, commentary, or an unconfirmed report. Several articles repeating one announcement should not be counted as several independent confirmations.
Match the Evidence to the Claim
Choose evidence that can answer the question being checked. Use the company’s published financial statements for reported results, its announcement for stated guidance, and the relevant official release for an economic decision. For a claimed market reaction, require timestamped market observations rather than an issuer’s description of its prospects.
For U.S. company research, Form 10-K provides business, risk, and annual financial information. However, filing a document does not turn it into an SEC endorsement. The SEC’s explanation of how to read a 10-K makes clear that the company prepares the report and the SEC does not vouch for its accuracy.
Keep that distinction in the summary. “The company reported” establishes who made the statement. “The regulator confirmed” says something else entirely.
Break the Summary Into Checkable Claims
Review individual assertions rather than approving a paragraph because its general message sounds reasonable. One sentence can contain several claims about a result, a comparison, and its cause.
Take the hypothetical sentence: “Quarterly sales rose 12%, beat expectations, and benefited from stronger European demand.” Check the sales figure, quarter, comparison period, expectations benchmark, and explanation separately. Finding the 12% figure does not validate the rest.
| Claim type | What to verify | Action if support is missing |
|---|---|---|
| Reported fact | Exact passage, entity, date, and scope | Remove it or mark it unverified |
| Calculated figure | Inputs, units, formula, and rounding | Recalculate independently |
| Quotation | Original wording, speaker, and surrounding context | Correct it or remove quotation marks |
| Comparison | Matching periods, definitions, and benchmark | State the mismatch or omit the comparison |
| Interpretation | Evidence and assumptions connecting facts to the inference | Label it as analysis, not a reported result |
Record a page, section, table row, or transcript timestamp beside each material claim. This small step makes later corrections much faster than searching an entire document again.
Check Numbers, Labels, and Comparisons Together
A number without its label is not fully checked. Verify the currency, unit, period, accounting basis, and whether the figure describes the entire business or one segment. Check negative signs and table headings directly, especially when reviewing extracted PDF text.
Recalculate derived figures outside the language model. Revenue rising from $800 million to $880 million represents 10% growth: the $80 million increase divided by the $800 million starting value. Do not accept a generated percentage because both input figures are correct.
Distinguish percentages from percentage points. A margin increasing from 20% to 22% rises by 2 percentage points, or 10% relative to its starting level. Calling it simply a “2% increase” introduces ambiguity.
Preserve the Accounting Basis
Keep GAAP and non-GAAP labels attached to the figures they describe. If a release presents adjusted earnings per share, do not shorten that to earnings per share without qualification. Check the adjustments and corresponding GAAP measure before writing a broader statement about profitability.
Non-GAAP measures are not necessarily comparable across companies, even when their names resemble each other. The SEC staff’s non-GAAP financial measures guidance addresses misleading labels and comparability, making it relevant evidence when checking how a summary presents adjusted results.
Apply the same discipline to “beat expectations.” Identify the estimates provider, observation time, number of contributors where available, and whether the estimate uses the same accounting basis as the reported result. Without a matching benchmark, leave the claim unresolved.
Check Meaning, Attribution, and Missing Qualifications
Read around the supporting passage, not just the sentence containing a matching phrase. Preserve words such as “may,” “expects,” “subject to,” and “excluding.” Removing one can turn a conditional statement into an unconditional one.
Watch the speaker too. An analyst asking whether demand will recover is not management confirming a recovery. An executive discussing a possible acquisition is not announcing a signed transaction. These distinctions deserve particular attention when using language models to analyze earnings calls, where questions and answers need separate attribution.
Check quotations character by character against the original material. If the wording is condensed or rewritten, present it as a paraphrase. Do not allow quotation marks to imply a verbatim statement that nobody made.
Then perform an omission review. Ask which excluded fact could change the reader’s interpretation: weaker guidance, cash outflows, dilution, a one-time gain, or a condition attached to an agreement. A short summary need not reproduce everything. It should preserve the facts that could reverse its apparent message.
Verify Dates and Market Context
Separate the event time, publication time, reporting period, and review time. A newly published article may discuss an old event. A current document may contain both historical results and forward-looking estimates. Replace vague references such as “today” or “last quarter” with exact dates or named periods in records intended for later use.
For market observations, specify the instrument, trading session, timestamp, and comparison point. “Shares rose 4%” could refer to an intraday move, a closing change, or after-hours trading. Those are different observations, not interchangeable ways to describe the same result.
Check the feed’s timing before accepting “current” prices. The distinction between real-time and delayed market data matters when a summary places price movements beside an announcement. Record the time zone as well, especially when combining documents and prices from different markets.
Do not infer causation from sequence alone. If a stock declined after an announcement, that supports a chronological description. Claiming it declined because of one sentence in the announcement requires more evidence. Preserve that uncertainty rather than writing a cleaner story than the records justify.
A Worked Example: Repairing a Misleading Earnings Summary
Consider this hypothetical source package:
A company reports quarterly revenue of $1.2 billion, up 8% from the prior year. Adjusted diluted earnings per share are $1.40; GAAP diluted earnings per share are $0.92. Management keeps its annual revenue guidance unchanged and says customer approvals may delay some orders.
The generated summary reads: “The company delivered 8% profit growth, earned $1.40 per share, and raised its outlook as demand accelerated.”
The first problem is the metric: 8% describes revenue growth, not profit growth. The second is an absent accounting label: $1.40 is adjusted diluted earnings per share. The third is a direct contradiction: guidance was unchanged, not raised. The final claim about accelerating demand is not established by the supplied evidence.
A corrected version would read: “Quarterly revenue increased 8% from the prior year to $1.2 billion. Adjusted diluted earnings per share were $1.40, compared with GAAP diluted earnings per share of $0.92. Management maintained annual revenue guidance and warned that customer approvals could delay some orders.”
This revision does not decide whether the shares are attractive. It restores the reported facts and preserves the qualification. Any investment interpretation should appear separately, with its assumptions exposed.
Use AI to Assist Verification, Not Approve Itself
Give the checking stage a narrower task than the drafting stage. Rather than asking “Is this summary accurate?”, require evidence for each claim. A practical instruction is:
Review this summary against the supplied documents only. For each material claim, provide the supporting passage and its location. Classify the claim as supported, contradicted, not established, or interpretation. Check figures, units, periods, accounting labels, attribution, and omitted qualifications. Do not fill gaps using assumed background knowledge.
Then inspect the returned passages yourself. A second answer is another item to review, not independent confirmation simply because it agrees with the first.
Where possible, separate mechanical checks from judgment. Use a calculator for arithmetic, document search for quoted wording, and direct table inspection for figures. Reserve human review for attribution, ambiguity, omitted context, and whether the evidence warrants the inference.
Set a Release Gate and Keep an Audit Trail
Review the claims with the greatest decision impact first: guidance changes, transaction status, earnings, liquidity, regulatory actions, and market prices. A cosmetic wording issue should not consume the time needed to check whether a deal was proposed or completed.
Use three release outcomes: approve, revise, or hold. Approve when material statements are supported and interpretations are labeled. Revise when a correction resolves the problem. Hold when missing or conflicting evidence could change the message. Urgency is not evidence.
A checked summary should still not become the sole basis for an investment decision. The joint SEC, NASAA, and FINRA investor alert on AI-generated investment information warns that outputs can be faulty even when the inputs are accurate, and calls for checking underlying sources.
Retain the original summary, corrected version, supporting documents, claim statuses, reviewer, and review time. For automated workflows, also record the model version and instructions used. This is a practical research record, not a universal statement of regulatory retention duties.
When testing a summarization process, track material errors, unsupported claims, missing qualifications, and review time separately. Test it on documents outside the set used to refine its instructions, including corrections, conflicting reports, and incomplete inputs.
The release standard should be traceability: another reviewer can locate the evidence, reproduce the arithmetic, and distinguish reported facts from analysis without having to trust the model’s confidence.