NetAuto Academy

Nornir for Network Engineers

Built-In Concurrency and Composing Custom Tasks

Advanced14 min read

What you'll learn

  • That nr.run() is concurrent by default, via a bounded thread pool
  • How this maps directly onto Phase 3.3's own ThreadPoolExecutor lesson
  • A real, measured speedup across the 5-device fleet
  • How to compose one task from another with task.run(), and how sub-task failure actually surfaces
🎓 Teach This Lesson

Listen to a guided explanation of this lesson.

Spoken narration isn't supported in this browser.

Concurrent by default

Not a new concept

Phase 3.3 built concurrency by hand with ThreadPoolExecutor. Nornir's nr.run() already runs every host concurrently, every time, using the exact same idea -- a bounded thread pool -- without writing ThreadPoolExecutor code at all.

the real, verified default runner
import inspect
from nornir.plugins.runners import ThreadedRunner

print(inspect.signature(ThreadedRunner.__init__))

Verified directly against the installed package.

output — real, verified
(self, num_workers: int = 20)

Nornir's default runner (plugin="threaded") is a bounded thread pool of 20 workers -- the same bounded-concurrency idea Phase 3.3's ThreadPoolExecutor(max_workers=...) lesson already covered by hand.

Quick checkConcept check
What does Nornir's default 'threaded' runner actually do?

A real, measured comparison

verify_concurrency_comparison.py
import time
from nornir import InitNornir
from nornir.core.task import Task, Result

def slow_task(task: Task) -> Result:
  time.sleep(0.3)
  return Result(host=task.host, result="done")

for workers in (1, 20):
  nr = InitNornir(
      runner={"plugin": "threaded", "options": {"num_workers": workers}},
      inventory={"plugin": "SimpleInventory", "options": {
          "host_file": "inventory/hosts.yaml",
          "group_file": "inventory/groups.yaml",
          "defaults_file": "inventory/defaults.yaml",
      }},
  )
  start = time.time()
  nr.run(task=slow_task)
  elapsed = time.time() - start
  print(f"num_workers={workers}: {elapsed:.2f}s")

5 hosts, each task simulating 0.3s of work -- num_workers=1 forces true serial execution; num_workers=20 is Nornir's real default.

output — real, measured
num_workers=1: 1.50s
num_workers=20: 0.30s

Real, measured output -- not invented. 1.50s = 5 x 0.3s, exactly serial. 0.30s = all 5 hosts run at once, exactly concurrent. A 4.97x speedup, measured directly against this module's own 5-device fleet.

Quick checkPredict output
Why does num_workers=1 take almost exactly 5 x as long as num_workers=20 in this measured comparison?

Composing one task from another

A task function can call task.run() to invoke another task function as a sub-task -- useful for building a larger check (like VLAN validation) from a smaller one (like fetching VLANs).

get_vlans.py — the smaller task
def get_vlans(task: Task) -> Result:
  hostname = task.host.name
  try:
      vlans = simulate_connection(hostname)
  except (ConnectionError, PermissionError) as e:
      return Result(host=task.host, result=str(e), failed=True)
  return Result(host=task.host, result=vlans, failed=False)

simulate_connection() is the same Netmiko-shaped simulation pattern established in Your First Network Automation -- not a new mock framework, just a function returning realistic success/failure outcomes without a real device.

Quick checkIdentify the correct code
Which correctly invokes get_vlans as a sub-task from inside another task function?

A genuine discovery: how a failing sub-task actually surfaces

Not what a first attempt assumes

A first, reasonable-looking attempt checks sub_result.failed after calling task.run(). That check is never reached for a failing sub-task -- task.run() raises immediately instead. This was found by inspecting Task.run()'s real source, not assumed.

inspect.getsource(Task.run) — real, verified excerpt
def run(self, task: Callable[..., Result], **kwargs: Any) -> MultiResult:
  ...
  r = self._start_children_task(task, **kwargs)
  if r.failed and self.host.close_connections_on_error:
      raise NornirSubTaskError(...)
  return self.results

Verified directly from the installed package's source. If the sub-task fails, task.run() raises NornirSubTaskError -- it does not return a normal Result you check with .failed afterward.

validate_vlans.py — the corrected version
from nornir.core.exceptions import NornirSubTaskError

def validate_vlans(task: Task) -> Result:
  try:
      sub_result = task.run(task=get_vlans)
  except NornirSubTaskError as e:
      return Result(host=task.host, result=e.result.result, failed=True)

  required = REQUIRED_VLANS[task.host.name]
  actual = sub_result[0].result
  missing = [v for v in required if v not in actual]
  if missing:
      return Result(host=task.host, result=f"missing VLANs: {[v['id'] for v in missing]}", failed=False)
  return Result(host=task.host, result="already compliant", failed=False)

try/except NornirSubTaskError is required, not optional. e.result.result recovers the original failure message from get_vlans -- SW4/SW5's real failures, still readable after the exception.

output — real, verified against all 5 hosts
SW1: failed=False result='already compliant'
SW2: failed=False result='missing VLANs: [10]'
SW3: failed=False result='missing VLANs: [30]'
SW4: failed=True result='SW4 did not respond'
SW5: failed=True result='SW5 rejected the credentials'

Real, verified output. SW4 and SW5's original failure messages (from get_vlans, raised via simulate_connection) survive intact through the caught NornirSubTaskError.

Quick checkScenario
A composed task calls task.run(task=get_vlans) without a try/except around it, and get_vlans fails for one host. What actually happens, based on Task.run()'s real, verified source?

Key takeaways

  • nr.run() is concurrent by default via a bounded thread pool (ThreadedRunner, num_workers=20) — the exact same bounded-concurrency idea Phase 3.3 built by hand with ThreadPoolExecutor.
  • Measured directly across the 5-device fleet: num_workers=1 → 1.50s (serial); num_workers=20 → 0.30s (concurrent) — a real 4.97x speedup, not an invented number.
  • task.run(task=sub_task) composes one task from another.
  • A failing sub-task makes task.run() raise NornirSubTaskError, not return a checkable failed Result — verified directly from source, and required to be caught explicitly.