AI Agent Development and Interview Guide: 04-Agent Architecture and LangGraph Orchestration

1. First Decision: Do You Actually Need an Agent?

Conditions favoring deterministic functions or hard-coded workflows:

• Steps follow a fixed deterministic sequence;
• Input parameters and field schemas are well-defined upfront;
• Operational rules can be exhaustively enumerated;
• Error consequences are severe with no human escalation gates;
• Strict SLAs exist for p99 latency and bit-exact reproducibility.

Conditions favoring Agent architectures:

• User goals are open-ended and execution paths cannot be fully predetermined;
• A large surface of external tools exists requiring contextual selection;
• Intermediate tool outputs dynamically alter downstream execution plans;
• The workflow inherently requires searching, trial-and-error, reflection, or follow-up clarification;
• Max step limits, permission boundaries, and termination criteria can be enforced deterministically.

🔥 P0 High-Frequency Essential: What is the difference between an Agent and a Workflow? A Workflow's control flow is primarily hard-coded by developers at compile-time; an Agent delegates runtime execution steps or tool selection to model reasoning. Production systems rarely force a binary choice, but rather embed bounded Agent decision nodes inside explicit deterministic state graphs, keeping high-risk mutations and core business rules strictly deterministic.

2. ReAct, Plan-and-Execute, and Router

2.1 Router

A single model classification node routes incoming requests to deterministic downstream handler pipelines. It is minimal, low-latency, highly testable, and ideal for triaging "Knowledge Base / Order Lookup / Human Escalation" traffic.

2.2 ReAct Loop

The model dynamically selects actions conditioned on current state, executes tools, observes environment feedback, and decides subsequent actions:

Input -> Decide Action -> Invoke Tool -> Observe -> Replanning -> Complete

Advantages: High flexibility; Disadvantages: Susceptible to infinite loops, redundant tool calls, and runaway token costs. You must strictly configure maximum step thresholds, tool IAM policies, and error handling strategies.

2.3 Plan-and-Execute

Generates a multi-step plan upfront before executing sequentially; execution outcomes can dynamically trigger replanning. Suited for long-horizon tasks, but plans can become stale and incur additional model roundtrips.

2.4 Selection Recommendations

3. State as the Single Source of Truth

Never allow application state to exist solely as unstructured conversational chat strings. Enforce rigid TypedDict or Pydantic schemas:

from typing import Literal, TypedDict


class AgentState(TypedDict, total=False):
    run_id: str
    user_id: str
    goal: str
    route: Literal["knowledge", "ticket", "human"]
    retrieved_evidence: list[dict]
    tool_calls: list[dict]
    tool_results: list[dict]
    step_count: int
    max_steps: int
    final_answer: str
    error_code: str | None

State design principles:

• Persist raw structured data objects rather than pre-formatted strings;
• Explicitly partition user inputs, model proposals, verified facts, and tool outputs;
• Mask sensitive credentials and never persist raw API keys in state stores;
• Favor immutable append operations or explicit field-level mutation reducers;
• Architect serialization, schema version migrations, and TTL cleanup for long-running workflows.

LangGraph officially defines nodes as "Python functions that accept current state and return state updates," and advises handling errors as first-class control flow. Refer to Thinking in LangGraph.

🔥 P0 High-Frequency Essential: Why is explicit state essential? Explicit state makes control flow observable, persistable, resumable, and testable; it cleanly separates ground-truth verified facts from generated model text, enabling deterministic state recovery during human approvals and failure retries. Relying solely on unstructured conversational history makes consistency impossible.

4. A Minimal State Graph

The following implementation demonstrates control flow mechanics independently of specific LLM provider SDKs.

from typing import Literal, TypedDict

from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import END, START, StateGraph


class SupportState(TypedDict, total=False):
    query: str
    route: Literal["knowledge", "ticket", "human"]
    evidence: list[str]
    ticket_id: str
    answer: str


def classify(state: SupportState) -> dict:
    query = state["query"].lower()
    if "refund" in query or "outage" in query:
        return {"route": "ticket"}
    if "human" in query or "agent" in query:
        return {"route": "human"}
    return {"route": "knowledge"}


def route_after_classify(state: SupportState) -> str:
    return state["route"]


def retrieve(state: SupportState) -> dict:
    # In production, query Retriever and return structured Evidence with source provenance.
    return {"evidence": ["[S1] Standard Policy: Check knowledge base first for common inquiries."]}


def answer(state: SupportState) -> dict:
    evidence = "
".join(state.get("evidence", []))
    return {"answer": f"Answered based on the following materials:
{evidence}"}


def create_ticket(state: SupportState) -> dict:
    # Production implementation requires authentication, idempotency keys, and audit logging.
    return {"ticket_id": "T-001", "answer": "Ticket T-001 has been created successfully."}


def handoff(state: SupportState) -> dict:
    return {"answer": "Transferred to human support representative."}


builder = StateGraph(SupportState)
builder.add_node("classify", classify)
builder.add_node("retrieve", retrieve)
builder.add_node("answer", answer)
builder.add_node("create_ticket", create_ticket)
builder.add_node("handoff", handoff)

builder.add_edge(START, "classify")
builder.add_conditional_edges(
    "classify",
    route_after_classify,
    {
        "knowledge": "retrieve",
        "ticket": "create_ticket",
        "human": "handoff",
    },
)
builder.add_edge("retrieve", "answer")
builder.add_edge("answer", END)
builder.add_edge("create_ticket", END)
builder.add_edge("handoff", END)

graph = builder.compile(checkpointer=InMemorySaver())

config = {"configurable": {"thread_id": "demo-thread-1"}}
result = graph.invoke({"query": "How many days of annual leave do I get?"}, config=config)
print(result["answer"])

This pedagogical example utilizes an in-memory checkpointer; production deployments require persistent backends. LangGraph Persistence documentation highlights that Checkpointing enables human-in-the-loop workflows, conversational memory, time-travel debugging, and fault tolerance; refer to Persistence.

5. What Model Decision Nodes Should Return

Never allow router nodes to emit unstructured free text. Enforce strict output schemas:

from typing import Literal
from pydantic import BaseModel, Field


class NextAction(BaseModel):
    action: Literal["search", "create_ticket", "ask_user", "finish"]
    tool_name: str | None = None
    tool_arguments: dict = Field(default_factory=dict)
    user_question: str | None = None
    final_answer: str | None = None

The model merely proposes the next action; the application runtime must validate:

• Whether tool_name exists in the authorized tool registry;
• Whether arguments conform to the parameter schema;
• Whether the authenticated user possesses execution permissions;
• Whether execution exceeds step limits and token budgets;
• Whether write mutations require human authorization;
• Whether the current state graph allows transitioning to the proposed action.

6. Termination Conditions and Loop Guards

Every Agent Run must enforce:

• Maximum execution steps;
• Maximum model invocations;
• Maximum tool invocations;
• Maximum token budget and monetary cost;
• Global end-to-end deadline timeout;
• Action deduplication loop detection;
• User cancellation signals;
• Unambiguous terminal success and failure states.

def guard_limits(state: AgentState) -> Literal["continue", "stop"]:
    if state["step_count"] >= state["max_steps"]:
        return "stop"

    calls = state.get("tool_calls", [])
    if len(calls) >= 2 and calls[-1] == calls[-2]:
        return "stop"

    return "continue"

Identical tool names do not necessarily imply loops (parameters and intent must be normalized and compared), but basic heuristic checks catch the majority of infinite looping patterns.

🔥 P0 High-Frequency Essential: How do you prevent Agent infinite loops? Enforce hard limits on steps, wall-clock time, tokens, and monetary cost; detect repeated state/action cycles; return structured error payloads from failed tools; define explicit termination criteria for zero-progress iterations; escalate high-risk or low-confidence states to human operators; and log step rationales in Traces. Never rely solely on prompt instructions saying "do not loop."

7. Treating Errors as First-Class Control Flow

LangGraph documentation categorizes errors into: transient network failures, model-recoverable errors, user-actionable errors, and unrecoverable runtime exceptions. Corresponding strategies:

Never delegate high-stakes exception handling entirely to LLM discretion. Financial, deletion, escalation, and external dispatch risks must be guarded by deterministic code.

8. Checkpoints and Fault Recovery

Checkpoints capture state snapshots at each graph superstep, enabling long-horizon workflows to:

• Resume execution seamlessly across worker process restarts;
• Pause and await asynchronous human authorization;
• Retry execution directly from the failed sub-step;
• Replay production error execution trajectories locally;
• Fork alternative execution branches from historical checkpoints.

However, resuming execution may re-run node functions; side-effects must therefore be strictly idempotent. LangGraph Interrupt documentation specifically warns: resumed nodes re-execute from the start of the node, requiring side-effects prior to interrupt to be strictly idempotent. Refer to Interrupts.

9. Human-in-the-Loop

from langgraph.types import Command, interrupt


def approve_ticket(state: SupportState) -> dict:
    decision = interrupt({
        "question": "Approve ticket creation?",
        "proposed_title": state["query"][:80],
    })
    return {"approved": bool(decision)}


# Execution pauses upon encountering interrupt on initial run; must resume using the identical thread_id.
# graph.invoke(Command(resume=True), config=config)

Approval payloads must provide operators with complete context to evaluate risk: action name, target entity, sanitized parameters, and projected blast radius, rather than a bare "proceed (y/n)?" prompt.

🔥 P0 High-Frequency Essential: Which actions require Human-in-the-Loop (HITL)? Irreversible mutations, high-value financial transactions, outbound external communications, sensitive PII access, privilege escalations, arbitrary code execution, and actions generated with low model confidence. Escalation thresholds are dictated by enterprise risk policies; avoid blocking on every read-only tool, and never allow the model to autonomously bypass required human gates.

10. The Boundary of Determinism

Deconstruct complex tasks into two distinct domains:

Well-Suited for Model Discretion

• Parsing open-ended user intent;
• Unstructured entity and parameter extraction;
• Candidate tool selection from authorized registries;
• Contextual text summarization and synthesis;
• Proposing open-ended operational plans.

Must Be Enforced by Deterministic Code

• Authentication and authorization verification;
• Financial and quantity safety limits;
• State machine transition validity checks;
• Parameter type constraints and business validation rules;
• Mandatory human approval triggers;
• Retry backoff, timeouts, and cost circuit breakers;
• Compliance logging and immutable audit trails.

This explicit architectural boundary separates production-grade enterprise Agents from brittle toy demos.

11. Single Agent Before Multi-Agent

If a single state graph with dedicated task nodes can solve the problem, never prematurely introduce multi-agent "role-playing." Multi-agent architectures incur:

• Multiplied model roundtrips and compounded latency;
• Context loss across inter-agent conversational boundaries;
• Ambiguous task ownership and accountability;
• Complex distributed deadlocks and un-debuggable evaluations;
• Exponentially harder IAM and permission governance.

Reserve multi-agent patterns strictly for scenarios with clear domain boundaries, parallel sub-task execution, or isolated tool/security privilege domains. Chapter 6 covers this in depth.

12. Chapter Exercises

Exercise A: Tri-Route Customer Support Graph

Implement three execution paths: Knowledge Q&A, Ticket Creation, and Human Escalation. Enforce structured router outputs, require human approval on ticket creation, and write unit tests for each branch.

Exercise B: Fault Recovery & Idempotency

Simulate a network timeout during the initial create_ticket execution. Prove that resuming from the checkpoint does not generate duplicate tickets, and verify checkpoint state integrity.

Exercise C: Loop Red-Teaming Guard

Construct a simulated tool that persistently returns "please retry." Ensure the state graph terminates within max step limits and emits an actionable diagnostic error rather than draining token budgets.

13. High-Frequency Interview Q&A

🔥 P0: How do you design an Agent state schema?

Partition state into user inputs, planned tasks, verified facts, tool requests, tool results, approval records, error logs, and final answers; enforce typed schemas (Pydantic/TypedDict); preserve raw payloads; implement serialization and schema migration strategies; exclude raw credentials; and define clear mutation rules (append vs. overwrite).

🔥 P0: What is the difference between Checkpoint and Long-Term Memory?

A Checkpoint captures the execution snapshot of a specific run/thread for fault recovery and time-travel debugging; Long-Term Memory persists knowledge and facts across multiple distinct conversation threads for future retrieval. They differ in lifecycle, scope, and eviction policies; never treat all historical execution checkpoints as user memory.

🔥 P0: How do you evaluate an Agent's execution trajectory?

Evaluate beyond final answer correctness: inspect tool selection precision, argument validity, total step count, duplicate/redundant invocations, permission violations, and proper termination/escalation timing. Benchmark against labeled ground-truth trajectories, allowed action sets, and token cost metrics.

⭐ P1: How do you choose between ReAct and Plan-and-Execute?

Use bounded ReAct for short, dynamic tool exploration; use Plan-and-Execute with checkpoints for long-horizon tasks requiring upfront decomposition and dynamic replanning. Fixed procedural tasks should never be forced into Agent loops. Base decisions on task horizon, path uncertainty, tool registry size, error blast radius, and latency budgets.

⭐ P1: What challenges arise when migrating state graph topologies?

Historical checkpoints may reference deleted or renamed nodes, and state field schemas may experience breaking type changes. You must design state versioning, migration adapters, backward compatibility layers, canary deployments, and graceful drain policies for in-flight threads.

14. Chapter Completion Criteria

• Able to articulate when to avoid Agent architectures in favor of deterministic workflows;
• Able to implement state graphs with conditional routing and cycle controls;
• Enforce max step limits, error taxonomy classification, checkpointing, and human-in-the-loop gates;
• Ensure write tools and mutations remain strictly idempotent across failure recoveries;
• Able to explain the rationale of every execution step from distributed Traces;
• Able to draw a rigorous boundary between model reasoning and deterministic code governance.