Nornir for Network Engineers
Built-In Concurrency and Composing Custom Tasks
What you'll learn
- That nr.run() is concurrent by default, via a bounded thread pool
- How this maps directly onto Phase 3.3's own ThreadPoolExecutor lesson
- A real, measured speedup across the 5-device fleet
- How to compose one task from another with task.run(), and how sub-task failure actually surfaces
Listen to a guided explanation of this lesson.
Spoken narration isn't supported in this browser.
Concurrent by default
Not a new concept
Phase 3.3 built concurrency by hand with ThreadPoolExecutor. Nornir's nr.run() already runs every host concurrently, every time, using the exact same idea -- a bounded thread pool -- without writing ThreadPoolExecutor code at all.
import inspect
from nornir.plugins.runners import ThreadedRunner
print(inspect.signature(ThreadedRunner.__init__))Verified directly against the installed package.
(self, num_workers: int = 20)Nornir's default runner (plugin="threaded") is a bounded thread pool of 20 workers -- the same bounded-concurrency idea Phase 3.3's ThreadPoolExecutor(max_workers=...) lesson already covered by hand.
A real, measured comparison
import time
from nornir import InitNornir
from nornir.core.task import Task, Result
def slow_task(task: Task) -> Result:
time.sleep(0.3)
return Result(host=task.host, result="done")
for workers in (1, 20):
nr = InitNornir(
runner={"plugin": "threaded", "options": {"num_workers": workers}},
inventory={"plugin": "SimpleInventory", "options": {
"host_file": "inventory/hosts.yaml",
"group_file": "inventory/groups.yaml",
"defaults_file": "inventory/defaults.yaml",
}},
)
start = time.time()
nr.run(task=slow_task)
elapsed = time.time() - start
print(f"num_workers={workers}: {elapsed:.2f}s")5 hosts, each task simulating 0.3s of work -- num_workers=1 forces true serial execution; num_workers=20 is Nornir's real default.
num_workers=1: 1.50s
num_workers=20: 0.30sReal, measured output -- not invented. 1.50s = 5 x 0.3s, exactly serial. 0.30s = all 5 hosts run at once, exactly concurrent. A 4.97x speedup, measured directly against this module's own 5-device fleet.
Composing one task from another
A task function can call task.run() to invoke another task function as a sub-task -- useful for building a larger check (like VLAN validation) from a smaller one (like fetching VLANs).
def get_vlans(task: Task) -> Result:
hostname = task.host.name
try:
vlans = simulate_connection(hostname)
except (ConnectionError, PermissionError) as e:
return Result(host=task.host, result=str(e), failed=True)
return Result(host=task.host, result=vlans, failed=False)simulate_connection() is the same Netmiko-shaped simulation pattern established in Your First Network Automation -- not a new mock framework, just a function returning realistic success/failure outcomes without a real device.
A genuine discovery: how a failing sub-task actually surfaces
Not what a first attempt assumes
A first, reasonable-looking attempt checks sub_result.failed after calling task.run(). That check is never reached for a failing sub-task -- task.run() raises immediately instead. This was found by inspecting Task.run()'s real source, not assumed.
def run(self, task: Callable[..., Result], **kwargs: Any) -> MultiResult:
...
r = self._start_children_task(task, **kwargs)
if r.failed and self.host.close_connections_on_error:
raise NornirSubTaskError(...)
return self.resultsVerified directly from the installed package's source. If the sub-task fails, task.run() raises NornirSubTaskError -- it does not return a normal Result you check with .failed afterward.
from nornir.core.exceptions import NornirSubTaskError
def validate_vlans(task: Task) -> Result:
try:
sub_result = task.run(task=get_vlans)
except NornirSubTaskError as e:
return Result(host=task.host, result=e.result.result, failed=True)
required = REQUIRED_VLANS[task.host.name]
actual = sub_result[0].result
missing = [v for v in required if v not in actual]
if missing:
return Result(host=task.host, result=f"missing VLANs: {[v['id'] for v in missing]}", failed=False)
return Result(host=task.host, result="already compliant", failed=False)try/except NornirSubTaskError is required, not optional. e.result.result recovers the original failure message from get_vlans -- SW4/SW5's real failures, still readable after the exception.
SW1: failed=False result='already compliant'
SW2: failed=False result='missing VLANs: [10]'
SW3: failed=False result='missing VLANs: [30]'
SW4: failed=True result='SW4 did not respond'
SW5: failed=True result='SW5 rejected the credentials'Real, verified output. SW4 and SW5's original failure messages (from get_vlans, raised via simulate_connection) survive intact through the caught NornirSubTaskError.
Key takeaways
- nr.run() is concurrent by default via a bounded thread pool (ThreadedRunner, num_workers=20) — the exact same bounded-concurrency idea Phase 3.3 built by hand with ThreadPoolExecutor.
- Measured directly across the 5-device fleet: num_workers=1 → 1.50s (serial); num_workers=20 → 0.30s (concurrent) — a real 4.97x speedup, not an invented number.
- task.run(task=sub_task) composes one task from another.
- A failing sub-task makes task.run() raise NornirSubTaskError, not return a checkable failed Result — verified directly from source, and required to be caught explicitly.