Capstone: A Staged, Gated Production Rollout

Both outcomes, not just the happy one

Reviewed changeassumed complete -- Git/NetDevOps, Lesson 1
select_canary(nr)Lesson 2
run_stage(canary)apply + validate -- Lesson 3
evaluate_gate(result, threshold)Lesson 3
PASS -> run_stage(rest) | FAIL -> rollback_stage(canary)Lesson 4
report

This capstone runs twice, for real, on purpose

A capstone that only shows a successful rollout would overstate what this module actually verified. Both real fixture scenarios from Lessons 3-4 are run here, end to end, producing two genuinely different reports.

Quick checkConcept check

Why does this capstone execute two separate scenarios instead of just demonstrating a successful rollout?

It doesn't -- only the successful path is shownDemonstrating only the success path would overstate what was actually verified -- this module's own discipline requires both the pass-and-expand and fail-and-rollback paths to be genuinely executed, not just describedThe failure path is included for entertainment value onlyTwo runs are required by nornir_napalm's API

The full orchestrator

run_rollout(nr, threshold) -- real, composed from Lessons 2-4

def run_rollout(nr, threshold=1.0):
  canary, rest = select_canary(nr)

  canary_result = run_stage(canary)
  gate = evaluate_gate(canary_result, threshold=threshold)

  report = {
      "canary": canary_result,
      "gate": gate,
      "expanded": {},
      "rolled_back": False,
  }

  if gate["decision"] == "PASS":
      rest_result = run_stage(rest)
      report["expanded"] = rest_result
  else:
      rollback_agg = rollback_stage(canary)
      report["rolled_back"] = not rollback_agg.failed

  return report

Every function here is exactly what Lessons 2-4 already verified -- select_canary, run_stage, evaluate_gate, rollback_stage. This orchestrator introduces no new mechanism, only the if/else connecting them.

Quick checkIdentify the correct code

Which line in run_rollout() is the actual 'gate' this entire module is named after?

canary, rest = select_canary(nr)if gate["decision"] == "PASS":report = {"canary": canary_result, ...}rest_result = run_stage(rest)

Real, verified run — Scenario A: canary passes, rollout expands

real, executed

nr = load_inventory("inventory/hosts_pass.yaml")
report = run_rollout(nr, threshold=1.0)
print(json.dumps(report, indent=2))

The exact fleet and fixtures from Lesson 3's PASS-path example.

output — real, verified

{
"canary": {
  "SW1": { "applied": true, "validated": true, "detail": "description='mgmt-uplink-verified'" }
},
"gate": { "decision": "PASS", "pass_rate": 1.0, "passed": 1, "total": 1 },
"expanded": {
  "SW2": { "applied": true, "validated": true, "detail": "description='mgmt-uplink-verified'" },
  "SW3": { "applied": true, "validated": true, "detail": "description='mgmt-uplink-verified'" },
  "SW4": { "applied": true, "validated": true, "detail": "description='mgmt-uplink-verified'" },
  "SW5": { "applied": true, "validated": true, "detail": "description='mgmt-uplink-verified'" }
},
"rolled_back": false
}

Real, verified output. The canary passed, the gate opened, and the change genuinely expanded to all four remaining hosts -- no rollback needed.

Quick checkPredict output

In Scenario A's real, verified report, why does 'expanded' contain four hosts (SW2-SW5) and not five?

This is a bug -- all five hosts should appear in 'expanded'SW1 was the canary, already handled in the 'canary' field -- 'expanded' is specifically the rest-of-fleet stage, run only after the canary's gate passed, so it correctly contains only the remaining four hostsOne host was silently skipped due to an errorexpanded always shows exactly 80% of the fleet

Real, verified run — Scenario B: canary fails, rollout aborts and rolls back

real, executed

nr = load_inventory("inventory/hosts_fail.yaml")
report = run_rollout(nr, threshold=1.0)
print(json.dumps(report, indent=2))

A genuinely different fixture set -- the canary's commit succeeds but its validation fails, exactly as Lessons 3-4 verified in isolation. Note: no fixture files exist for SW2-SW5 in this scenario -- if expansion were ever mistakenly triggered, it would fail loudly rather than silently succeed.

output — real, verified

{
"canary": {
  "SW1": { "applied": true, "validated": false, "detail": "description='mgmt-uplink'" }
},
"gate": { "decision": "FAIL", "pass_rate": 0.0, "passed": 0, "total": 1 },
"expanded": {},
"rolled_back": true
}

Real, verified output. The canary's change didn't take effect, the gate correctly aborted, 'expanded' is genuinely empty (the rest of the fleet was never touched -- confirmed by the absence of any error, since no fixtures exist for those hosts in this scenario), and rollback succeeded.

Quick checkScenario

How does this lesson prove that SW2-SW5 were genuinely never touched in Scenario B, rather than just assuming it from reading the code?

It doesn't -- this is only inferred from the control flowNo fixture files exist for SW2-SW5 in the fail-path scenario at all -- if run_stage(rest) had been mistakenly called, NAPALM's mock driver would have raised a real, visible error. The real run completing without that error is direct proof, not just code-readingSW2-SW5 have fixtures, but they were configured to always report unchangedThe rest of the fleet is checked manually after each run

What this capstone never claims

Mock/offline execution -- always labeled as such

Every result on this page came from NAPALM's real mock driver -- the same mechanism verified in NAPALM for Network Engineers (Phase 3.5) and reused in every capstone since. This is genuinely executed, local, offline automation logic (Category A). It is never a real production rollout, real hardware verification, or live fleet validation -- no such infrastructure was used or is claimed anywhere in this module.

Quick checkScenario

How should a learner honestly describe what this capstone proves?

"A real production rollout was tested successfully.""The staged-rollout mechanism -- canary selection, apply-then-validate staging, threshold gating, and automatic rollback -- was genuinely verified end to end, on both outcomes, against NAPALM's real mock driver. No real device or production fleet was used.""This module proves the technique works on real hardware.""Only the successful path was actually tested; the failure path is theoretical."

Key takeaways

This capstone combines every piece of the module -- canary selection, apply-then-validate staging, threshold gating, and rollback -- into one orchestrator that introduces no new mechanism of its own beyond the if/else connecting them.
Real, verified Scenario A: canary passes (100% rate), rollout genuinely expands to the remaining four hosts, no rollback.
Real, verified Scenario B: canary's commit succeeds but validation fails, the gate correctly aborts, the rest of the fleet is provably untouched, and rollback genuinely succeeds.
Both outcomes were executed for real -- this module never demonstrates only the success path.
Every result here is Category A (genuine, local, mock-driver execution) -- never presented as a real production rollout, real hardware verification, or live fleet validation.

What you'll learn

Both outcomes, not just the happy one

The full orchestrator

Real, verified run — Scenario A: canary passes, rollout expands

Real, verified run — Scenario B: canary fails, rollout aborts and rolls back

What this capstone never claims