Agently Docs

English

中文

English

中文

Appearance

Document Review Assistant Playbook ​

Languages: English · 中文

Document review is a good stress test for an agent system. A single prompt can produce a plausible comment, but a business system needs issue records, evidence links, suggested actions, and a way to review or approve the result.

Better Shape ​

text
document input
  -> classify review scope
  -> extract evidence / clauses / references
  -> create structured issue records
  -> check business rules or model-judge criteria
  -> generate review report artifact
  -> human confirmation or workflow continuation

Start with Structured Issue Records ​

python
result = (
    agent
    .input(document_text)
    .output({
        "review_summary": (str, "short summary of review result", True),
        "issues": [
            {
                "severity": (str, "high / medium / low", True),
                "clause_ref": (str, "where the issue appears", True),
                "finding": (str, "what is wrong or risky", True),
                "evidence": (str, "quoted or summarized evidence", True),
                "suggested_fix": (str, "recommended change", True),
            }
        ],
    })
    .get_result()
)

data = result.get_data()

Downstream systems should consume the structured issue list, not scrape a natural-language report.

Use Workspace for Evidence and Artifacts ​

Long documents and review reports should not live entirely in prompt or execution state.

ItemRecommended place
Original documentWorkspace artifact or external document ref
Extracted evidenceWorkspace observation / evidence record
Issue listStructured result + optional Workspace record
Draft reportWorkspace artifact
Human decisionWorkspace decision / checkpoint

Execution state can keep refs and summaries.

Add TriggerFlow When Review Has Stages ​

Use TriggerFlow when the task needs:

  • section-level fan-out,
  • specialist review roles,
  • approval or manual correction,
  • report rendering after issue extraction,
  • resume after external input,
  • runtime stream for visible progress.

For a short one-off review, a single AgentExecution may be enough.

Product Events ​

For a UI or IM workflow, expose stable events:

json
{
  "event_type": "review_section_done",
  "task_id": "doc-review-241",
  "stage": "risk_review",
  "status": "running",
  "summary": "3 high-risk clauses found",
  "artifact_ref": "workspace://artifacts/review-draft.md"
}

Do not expose raw chunk names or parser paths as the frontend contract.

Quality Checks ​

RiskCheck
Missing required fieldOutput schema with ensure=True
Unsupported conclusionEvidence ref or quoted support required
Wrong severityDeterministic rule for known thresholds, or model judge with explicit rubric
Hallucinated clauseClause refs must map to document spans or evidence records
Unsafe auto-sendHuman approval before external write or final delivery

Avoid ​

  • Asking the model to "review the document" and saving only the prose answer.
  • Putting the entire document, every intermediate result, and final report into execution state.
  • Letting the report text be the only source of issue records.
  • Skipping evidence refs for high-risk findings.
  • Treating a human-facing comment as an internal command.

Continue Reading ​