HiP-HOPS for agent graphs¶

HiP-HOPS — Hierarchically Performed Hazard Origin and Propagation Studies — is a compositional safety analysis. Its central claim is that if you annotate each component with how it can fail locally, the system-level fault trees can be generated by traversing the connections, rather than drawn by an engineer.

That matters for agentic systems more than for most, because agent architectures change weekly. A hand-drawn fault tree is out of date the first time someone adds a node; a synthesised one is regenerated by re-running the analysis.

The phases, and where each lives here¶

HiP-HOPS phase	Here	Module
Model the system as components with ports and connections	Read from `graph.get_graph()`: nodes → components, edges → connections, conditional edges → an explicit router component	`architecture.model`
— (agent graphs need this; classical HiP-HOPS assumes it)	Unroll feedback loops and close them with a feedback-cut component	`architecture.acyclic`
Annotate each component with local failure logic (IF-FMEA): output deviations as expressions over input deviations and internal basic events	A library of agent archetypes: LLM agent, tool/executor, router, aggregator, transform, boundary, feedback cut	`faulttree.failure`
Synthesise fault trees by traversing connections and substituting local logic	Backward traversal from a boundary hazard, memoised so shared sub-trees stay shared	`faulttree.synthesis`
Minimal cut sets, quantification, FMEA	MOCUS bottom-up with absorption; MCUB quantification with optional imprecise intervals; FMEA derived from the trees	`faulttree.analysis`
(beyond classical HiP-HOPS) exact probability and diagnosis	The tree converted to a Bayesian network	`bayes`
(beyond classical HiP-HOPS) measured basic events	HIP-LLM posterior under an operational profile	`reliability`

Deviations, and the notation¶

A deviation is a failure class occurring at a component’s port, written class-component.port:

O-coder.out        the calculator produced no output
VS-aggregator.out  the judge delivered a wrong-but-plausible answer

Local failure logic is a set of Boolean expressions, one per output deviation, over input deviations and the component’s own basic events. For an LLM agent:

O-out  = BE-EMPTY   OR BE-CTX     OR O-in
VC-out = BE-FORMAT  OR BE-TRUNC   OR VC-in
VS-out = BE-HALLUC  OR BE-NONDET  OR VS-in

Read the last line: an agent delivers a subtly wrong answer if it hallucinates, if non-determinism took it somewhere wrong, or if it was given a subtly wrong answer and passed it on. That last term is what makes the analysis compositional — and it is also why a chain of agents is so much less reliable than any one of them.

Three modelling decisions¶

Conditional edges become components¶

In LangGraph a router is not a node; it is a callable passed to add_conditional_edges. It is nevertheless real software with its own failure modes — a regular expression that matches the wrong branch, or no branch at all. The extractor materialises it as a ROUTER component sitting between the deciding node and its successors.

router = model.components["generator::router"]
router.branches      # ['coder', 'error', 'end']

Its source is not the node’s source. LangGraph keeps the routing function separately and the drawable graph does not carry it, so attributing the node’s code to the router would also give the router the node’s resources — putting it in the wrong common-cause group. Pass it explicitly:

extract_architecture(graph, node_functions={"generator::router": route_fn})

Feedback loops are unrolled, never deleted¶

Fault trees are acyclic; ReAct loops are not. The tempting fix — cut the back edge — is unsafe, because it deletes the tool’s contribution from the tree and therefore understates risk.

Instead, make_acyclic unrolls each loop to a stated depth and closes it with a feedback-cut component carrying the loop’s deviations to the system boundary:

acyclic, report = make_acyclic(model, unroll=2)
print(report.summary())

1 feedback loop(s) found; unrolled to depth 1 and closed with 1 feedback-cut
component(s).
  loop: coder -> generator -> generator::router
  back edge cut: coder -> generator
  Deleting a back edge outright would remove the feedback path's contribution
  from the fault tree and understate risk; the feedback-cut component preserves
  it.
  unroll=1: a single pass through the loop body is modelled. Increase unroll to
  expose iteration-dependent effects such as prompt growth.

The report goes into the Markdown output, so the loop handling is a stated assumption rather than a hidden one.

Shared resources become common-cause groups¶

Two components that call the same model snapshot do not fail independently. The extractor reads model ids out of the node source (and out of the live objects, when you pass globals_ns=globals()) and groups components by shared resource:

model.common_cause_groups()

{('llm', 'Qwen/Qwen2.5-Math-1.5B-Instruct'): ['aggregator', 'cot_agent', 'react_agent'],
 ('runtime', 'cuda:0'): ['aggregator', 'cot_agent', 'react_agent']}

Each group gets a CCF- basic event that appears in the tree of every member. This is what turns an apparently redundant architecture into one with an order-1 cut set, and it is the finding most worth having.

Declare resources the source does not name:

LangGraphExtractor().with_resources(
    critic={"llm": "gpt-4o-2024-11-20"},
    drafter={"llm": "gpt-4o-2024-11-20"},   # same snapshot: now a CCF group
)

Hazards¶

A hazard is a top event at the system boundary. The default set is derived from the architecture:

Id	Severity	Meaning
`H1`	major	No answer delivered
`H2`	critical	Incorrect answer delivered and accepted as correct
`H3`	minor	Malformed answer delivered
`H4`	minor	Answer too late / budget exhausted
`H5-<tool>`	catastrophic	Unsafe execution of model-authored code in a tool

H5 appears only where a tool’s source contains eval or exec. In the ReAct example it does, and it is an order-1 cut set:

[catastrophic] H5-coder  BE-coder-UNSAFE (coder)

Supply your own hazards when the boundary is somewhere else:

from hiphopsllm import Hazard, FClass

study = AgenticReliabilityStudy(
    graph,
    hazards=[Hazard(id="HX", name="PII leaves the system", severity="critical",
                    component="__end__", port="in", fclass=FClass.COMMISSION)],
)