NetAuto Academy

Multi-vendor Automation

Per-Device Failure Handling and a Composed Config Workflow

Advanced15 min read

What you'll learn

  • That one device failing never blocks collecting the rest of the fleet's results
  • A real, verified, honest look at inconsistent exception behavior across real drivers
  • napalm_configure as a real composition of load/diff/commit-or-discard
  • Nornir's own global dry_run flag, verified end to end
🎓 Teach This Lesson

Listen to a guided explanation of this lesson.

Spoken narration isn't supported in this browser.

One device's failure, not the fleet's

Not a new pattern -- the same one, reconfirmed

Phase 3.4 already established that one host's failure never blocks Nornir from collecting the others' results. Lesson 3's real 4-host run reconfirmed this across a mixed-vendor-shaped fleet. This lesson goes one level deeper: what do real failures actually look like across genuinely different vendor drivers?

Real, verified failures — and a genuine inconsistency

real drivers, real unreachable addresses -- ios, eos, junos
nr = InitNornir(
  inventory={"plugin": "SimpleInventory", "options": {"host_file": "inventory/hosts_real.yaml", ...}},
  runner={"plugin": "threaded", "options": {"num_workers": 3}},
)
nr.inventory.defaults.username = os.environ.get("NAPALM_USERNAME", "")
nr.inventory.defaults.password = os.environ.get("NAPALM_PASSWORD", "")
agg = nr.run(task=napalm_get, getters=["get_facts"])

Three hosts, platform: ios / eos / junos respectively, pointed at addresses nothing responds on in this environment -- a genuine, reproducible failure, not a hypothetical.

output — real, verified, 21.08s elapsed for all 3 concurrently
IOS1   failed: True   napalm.base.exceptions.ConnectionException('Cannot connect to 10.10.10.211')
EOS1   failed: True   pyeapi.eapilib.ConnectionError('Socket error during eAPI connection: ...')
JUNOS1 failed: True   napalm.base.exceptions.ConnectionException(None)

Real, verified output. Three genuinely different exception shapes for the same conceptual failure (nothing responds).

Observed in this verification environment -- not claimed as a universal NAPALM guarantee

IOS's failure is NAPALM's own normalized ConnectionException with a real message. EOS's failure is pyeapi.eapilib.ConnectionError -- confirmed via issubclass() to be a DIFFERENT class from Python's builtin ConnectionError, and not one of NAPALM's own napalm.base.exceptions classes at all. Junos's failure is NAPALM's normalized ConnectionException, but with an empty (None) message. This was observed in this verification environment against unreachable addresses -- not claimed as guaranteed identical behavior on every platform or against every real device.

Quick checkConcept check
Based on the real, verified output above, does NAPALM normalize connection failures identically across every driver?
Quick checkScenario
A learner writes except ConnectionError: to catch a NAPALM connection failure, assuming it works the same on every driver. What real, verified risk does this lesson identify?

napalm_configure: a real composition, not a new mechanism

napalm_configure's real source, installed nornir_napalm 0.6.0
def napalm_configure(task, dry_run=None, filename=None, configuration=None,
                    replace=False, commit_message=None, revert_in=None):
  device = task.host.get_connection(CONNECTION_NAME, task.nornir.config)

  if replace:
      device.load_replace_candidate(filename=filename, config=configuration)
  else:
      device.load_merge_candidate(filename=filename, config=configuration)
  diff = device.compare_config()

  dry_run = task.is_dry_run(dry_run)

  if not dry_run and diff:
      device.commit_config(...)
  else:
      device.discard_config()
  return Result(host=task.host, diff=diff, changed=len(diff) > 0)

Verified directly by reading the installed package's real source -- this is exactly Phase 3.5's own load -> compare_config -> commit-or-discard sequence, composed into one Nornir task call.

Quick checkConcept check
What does napalm_configure's real source reveal about its relationship to Phase 3.5's candidate-config lessons?

dry_run: rigorously verified, both branches

verified two ways: removing the fixture the OTHER branch needs, to prove which path actually ran
# Test 1: dry_run=True, with commit_config's fixture file REMOVED.
#   Result: still succeeds -- proves discard_config() was called, not commit_config().
# Test 2: dry_run=False, with discard_config's fixture file REMOVED.
#   Result: still succeeds -- proves commit_config() was called, not discard_config().
# Test 3: InitNornir(dry_run=True), no per-task override, commit_config's fixture REMOVED.
#   Result: still succeeds -- proves the global flag alone drives the same branch.

Not just read from source -- each branch was proven by deliberately removing the fixture the OTHER branch would need, and confirming the call still succeeded.

output — real, verified across all three tests
dry_run=True,  commit_config.1 removed: failed=False, changed=True   (discard path confirmed)
dry_run=False, discard_config.1 removed: failed=False, changed=True   (commit path confirmed)
InitNornir(dry_run=True), no override, commit_config.1 removed: failed=False, changed=True

Real, verified output. Nornir's own global InitNornir(dry_run=True) flag correctly drives napalm_configure's branch even with no per-task dry_run= argument -- task.is_dry_run() falls back to it exactly as its real source states.

The same principle, now with its own real Nornir mechanism

Reliable Automation (Phase 2.8) taught dry-run by hand. NAPALM (Phase 3.5) taught it as compare_config()-then-decide. This lesson adds one more real piece: Nornir itself has a global dry_run flag (InitNornir(dry_run=True)) that every dry-run-aware task, including napalm_configure, automatically respects -- verified directly, not assumed.

Quick checkPredict output
InitNornir(dry_run=True) is set, and napalm_configure() is called with no dry_run argument at all. Based on the rigorously verified tests above, what happens?

Key takeaways

  • One device's failure never blocks Nornir's collection of the fleet's other results -- reconfirmed, not a new pattern.
  • A real, verified, honest finding: NAPALM's exception normalization has a gap -- EOS raises pyeapi.eapilib.ConnectionError, a genuinely different class from IOS/Junos's napalm.base.exceptions.ConnectionException. Observed in this environment, not claimed universal.
  • napalm_configure() is a real, verified composition of Phase 3.5's own load/compare_config/commit-or-discard sequence -- this module's core theme made concrete.
  • Nornir's own InitNornir(dry_run=True) global flag was rigorously verified (via deliberate fixture removal, not just source reading) to correctly drive napalm_configure's branch with no per-task override.