Stage Gates: Validate, Then Expand — or Abort

Applying and validating, composed as one stage

Reused pieces, a new composition

napalm_configure() and napalm_get() (Phase 3.6) are unchanged. What's new is composing them into one stage: apply the change, then immediately read state back to confirm it actually took effect -- not assumed from a successful commit alone.

run_stage(nr_subset) -- real, verified

def _check_host(task):
  task.run(task=napalm_configure, configuration=(
      "interface GigabitEthernet0/1\n"
      " description mgmt-uplink-verified\n"
  ))
  task.run(task=napalm_get, getters=["get_facts", "get_interfaces"])

def run_stage(nr_subset):
  stage_result = {}
  agg = nr_subset.run(task=_check_host)

  for hostname, multi_result in agg.items():
      if multi_result.failed:
          stage_result[hostname] = {
              "applied": False, "validated": False,
              "detail": str(multi_result[0].exception),
          }
          continue

      configure_result = multi_result[1]
      get_result = multi_result[2]
      interfaces = get_result.result["get_interfaces"]
      iface = get_result.result["get_facts"]["interface_list"][0]
      actual_description = interfaces[iface]["description"]

      validated = actual_description == "mgmt-uplink-verified"
      stage_result[hostname] = {
          "applied": not configure_result.failed,
          "validated": validated,
          "detail": f"description='{actual_description}'",
      }
  return stage_result

task.run() composes two sub-tasks (Phase 3.4's own composition mechanism) -- napalm_configure applies the change, napalm_get reads it back. applied and validated are tracked separately: a commit can succeed while the change still doesn't show up as expected.

Quick checkConcept check

Why does run_stage() track applied and validated as two separate booleans, instead of one combined success flag?

It's redundant -- they always have the same valueA configure task can genuinely succeed (applied=True) while the resulting state still doesn't match what was intended (validated=False) -- these are two different real outcomes, and this lesson's fail-path demo below shows exactly that gapapplied is only used for logging, never for decisionsvalidated is always computed before applied

Real, verified: the PASS path

real, executed against a canary that succeeds

canary_result = run_stage(canary)
print(canary_result)

The canary's config was applied and, on reading state back, genuinely matched the target.

output — real, verified

{'SW1': {'applied': True, 'validated': True, 'detail': "description='mgmt-uplink-verified'"}}

Real, verified output. Both applied and validated are True -- the canary is genuinely healthy.

evaluate_gate(): a real, structured decision

evaluate_gate(result, threshold) -- real, verified

def evaluate_gate(result, threshold=1.0):
  total = len(result)
  passed = sum(1 for r in result.values() if r["applied"] and r["validated"])
  pass_rate = passed / total if total else 0.0
  decision = "PASS" if pass_rate >= threshold else "FAIL"
  return {"decision": decision, "pass_rate": pass_rate, "passed": passed, "total": total}

threshold is the minimum fraction of the stage that must pass for the gate to allow expansion -- a real, explicit success/failure criterion, not an implicit 'did it error or not.'

output — real, verified (PASS path)

{'decision': 'PASS', 'pass_rate': 1.0, 'passed': 1, 'total': 1}

Real, verified output. 100% of the canary passed, meeting the default threshold of 1.0 -- the gate allows expansion.

Quick checkIdentify the correct code

Which correctly describes evaluate_gate()'s real decision rule?

It returns PASS as long as no exception was raised, regardless of validatedIt returns PASS only when the fraction of hosts with BOTH applied=True and validated=True meets or exceeds threshold -- a structured, explicit criterion, not an implicit error checkIt always requires 100% regardless of the threshold argumentIt ignores the applied field entirely

Real, verified: the FAIL path — not just described, executed

Both outcomes actually run, not just the success path

A second, separate fixture scenario was built specifically so this lesson could show a genuine failure -- not a hypothetical. The canary's config commit succeeds, but the interface description silently didn't change (a real, plausible production failure mode).

real, executed against a canary that fails validation

nr = load_inventory("inventory/hosts_fail.yaml")
canary, rest = select_canary(nr)
canary_result = run_stage(canary)
print(canary_result)

gate = evaluate_gate(canary_result, threshold=1.0)
print(gate)

A genuinely different fixture set from the PASS-path example above -- the commit succeeds, but the post-check read-back shows the OLD description.

output — real, verified (FAIL path)

{'SW1': {'applied': True, 'validated': False, 'detail': "description='mgmt-uplink'"}}
{'decision': 'FAIL', 'pass_rate': 0.0, 'passed': 0, 'total': 1}

Real, verified output. applied=True (the commit succeeded) but validated=False (the description never actually changed) -- exactly the gap this lesson's first Callout named. The gate correctly returns FAIL.

Quick checkPredict output

In the real, verified FAIL-path output above, why is applied=True even though the gate still returns FAIL?

This is a bug -- applied should be False whenever the gate failsapplied reflects only whether the commit itself succeeded (it did); validated reflects whether the resulting state actually matched the target (it didn't) -- the gate correctly uses BOTH, so a 'successful but ineffective' change is still caughtapplied and gate decision are unrelated fieldsThe fixture data is inconsistent

Quick checkScenario

A larger canary stage has 10 hosts; 9 validate successfully and 1 doesn't. With threshold=1.0 (the default used throughout this lesson), what does evaluate_gate() return?

PASS, since 90% is a reasonable success rateFAIL -- pass_rate would be 0.9, which is less than threshold=1.0. A lower threshold (e.g. 0.9) would need to be explicitly chosen by the caller for this same result to PASSPASS, since only one host failedevaluate_gate() cannot handle more than one host

Key takeaways

run_stage() composes napalm_configure() and napalm_get() (both unchanged from Phase 3.6) into one apply-then-validate stage, tracking applied and validated separately.
evaluate_gate(result, threshold) is a real, structured pass/fail decision -- verified to require both applied and validated, compared against an explicit threshold.
Both outcomes were genuinely executed with real, different fixture data: a canary that passes (100% rate, PASS) and one that fails validation despite a successful commit (0% rate, FAIL) -- not just the success path demonstrated.
The FAIL-path result (applied=True, validated=False) is exactly the real production failure mode -- a commit that 'succeeds' without producing the intended state -- that this lesson's two-boolean design exists to catch.

What you'll learn

Applying and validating, composed as one stage

Real, verified: the PASS path

evaluate_gate(): a real, structured decision

Real, verified: the FAIL path — not just described, executed