Project 2: Multi-Vendor Configuration Compliance Report

The business problem

Scenario

The fleet spans three vendors. A required baseline setting -- in this project, a central NTP server -- is supposed to be present on every device, but nobody has a fast way to check that across all of them at once, especially since each vendor's configuration syntax looks different. Worse, the fleet itself isn't a fixed list anymore -- devices get added and decommissioned in NetBox, and a static hosts.yaml would silently drift out of date the moment that happens.

Requirements: the device list must be read live from NetBox, not a local file; the compliance check must run identically regardless of vendor; the report must clearly identify which specific devices are missing the required line, not just an aggregate pass/fail count.

Quick checkConcept check

Why does this project read the fleet from NetBox rather than a static hosts.yaml file, unlike most of this curriculum's earlier examples?

hosts.yaml files are deprecated and no longer supported by NornirThe requirement is specifically that the compliance check always reflects NetBox's current device list -- a static file would silently go stale the moment a device is added or removed in NetBoxNAPALM cannot connect to devices listed in a YAML fileIt's not actually necessary -- this is an arbitrary choice

Skills reused

NetBoxInventory2 (dynamic inventory)NetBox / Source of Truth
use_platform_napalm_driverresolves NetBox's platform field into a driver name
napalm_get() across a mixed fleetMulti-vendor Automation
New here: a compliance rule evaluated identically across all three

Not re-taught here

How NetBoxInventory2 resolves platforms, how nornir_napalm selects a driver from inventory, and how NAPALM's mock driver works are all assumed knowledge from NetBox / Source of Truth, Multi-vendor Automation, and NAPALM for Network Engineers.

A freshly-built NetBox-shaped mock, not a reused one

Rebuilt for this project, not copy-pasted

No source file from an earlier module's own verification lives in this repository -- each phase's Python artifacts are scratch work, not committed content. This project's mock NetBox server was built fresh, using only Python's standard library (http.server), and freshly verified end to end. It reproduces exactly the two endpoints NetBoxInventory2 actually calls when use_platform_napalm_driver=True -- /api/dcim/platforms/ and /api/dcim/devices/ -- at the same real, documented shape NetBox / Source of Truth already established. No new dependency was added to build it.

mock_netbox_server.py

import json
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import urlparse

PLATFORMS = [
  {"id": 1, "slug": "cisco_ios", "name": "Cisco IOS", "napalm_driver": "mock"},
  {"id": 2, "slug": "arista_eos", "name": "Arista EOS", "napalm_driver": "mock"},
  {"id": 3, "slug": "junos", "name": "Juniper Junos", "napalm_driver": "mock"},
]

DEVICES = [
  {"id": 101, "name": "CORE-SW1", "platform": {"id": 1, "slug": "cisco_ios", "name": "Cisco IOS"}, "site": {"id": 1, "slug": "hq"}, "primary_ip": None},
  {"id": 102, "name": "DIST-SW1", "platform": {"id": 2, "slug": "arista_eos", "name": "Arista EOS"}, "site": {"id": 1, "slug": "hq"}, "primary_ip": None},
  {"id": 103, "name": "EDGE-RTR1", "platform": {"id": 3, "slug": "junos", "name": "Juniper Junos"}, "site": {"id": 1, "slug": "hq"}, "primary_ip": None},
]


class NetBoxMockHandler(BaseHTTPRequestHandler):
  def _send_json(self, payload):
      body = json.dumps(payload).encode("utf-8")
      self.send_response(200)
      self.send_header("Content-Type", "application/json")
      self.send_header("Content-Length", str(len(body)))
      self.end_headers()
      self.wfile.write(body)

  def do_GET(self):
      path = urlparse(self.path).path
      if path == "/api/dcim/platforms/":
          self._send_json({"count": len(PLATFORMS), "next": None, "previous": None, "results": PLATFORMS})
      elif path == "/api/dcim/devices/":
          self._send_json({"count": len(DEVICES), "next": None, "previous": None, "results": DEVICES})
      else:
          self.send_response(404)
          self.end_headers()

  def log_message(self, format, *args):
      pass


def run_server(port=8000):
  return ThreadingHTTPServer(("127.0.0.1", port), NetBoxMockHandler)

Stdlib-only (http.server), zero new dependencies. Three devices, three distinct vendor platform slugs -- the deliberate multi-vendor shape this report needs.

Quick checkIdentify the correct code

Why does this mock server only need to implement /api/dcim/platforms/ and /api/dcim/devices/, when real NetBox exposes dozens of endpoints (interfaces, IP addresses, sites, and more)?

Those are simply the two easiest endpoints to buildReading NetBoxInventory2's own source shows it only calls those two endpoints when use_platform_napalm_driver=True -- a mock only needs to answer the calls the code under test actually makesNetBox itself only has two endpointsThe other endpoints don't exist yet in real NetBox

The compliance script

compliance_lib.py

from nornir import InitNornir
from nornir.core.inventory import ConnectionOptions
from nornir_napalm.plugins.tasks import napalm_get

REQUIRED_NTP_SERVER = "ntp server 10.10.10.1"


def load_inventory():
  nr = InitNornir(
      inventory={
          "plugin": "NetBoxInventory2",
          "options": {
              "nb_url": "http://127.0.0.1:8000",
              "nb_token": "fake-token-for-mock-server",
              "use_platform_napalm_driver": True,
              "group_file": "inventory/groups.yaml",
              "defaults_file": "inventory/defaults.yaml",
          },
      },
  )
  # NetBox supplies identity and platform only (NetBox / Source of Truth's
  # own finding) -- the mock fixture path each host's NAPALM connection
  # needs is local test wiring, not something real NetBox would store.
  for host in nr.inventory.hosts.values():
      host.connection_options["napalm"] = ConnectionOptions(
          extras={"optional_args": {"path": f"fixtures/{host.name}"}}
      )
  return nr


def _pull_config(task):
  task.run(task=napalm_get, getters=["get_config"])


def evaluate_compliance(nr):
  agg = nr.run(task=_pull_config)
  report = {}
  for hostname, multi_result in agg.items():
      running_config = multi_result[1].result["get_config"]["running"]
      host = nr.inventory.hosts[hostname]
      vendor_slug = host.data.get("platform", {}).get("slug", "unknown")
      compliant = REQUIRED_NTP_SERVER in running_config
      report[hostname] = {
          "vendor": vendor_slug,
          "driver": host.platform,
          "compliant": compliant,
          "detail": "required NTP server present" if compliant else f"missing required line: '{REQUIRED_NTP_SERVER}'",
      }
  return report


def summarize(report):
  total = len(report)
  compliant = sum(1 for r in report.values() if r["compliant"])
  return {
      "total": total,
      "compliant": compliant,
      "non_compliant": total - compliant,
      "compliance_rate": compliant / total if total else 0.0,
  }

The real, verified compliance evaluator. Note host.data['platform']['slug'] -- NetBox's own vendor identity -- reported separately from host.platform (the resolved NAPALM driver, 'mock' for all three here).

Quick checkConcept check

In the report, CORE-SW1's "vendor" shows cisco_ios but its "driver" shows mock. Why are these two different values, and is that a bug?

It's a bug -- they should always matchNot a bug: "vendor" is NetBox's own record of what the device actually is (host.data), while "driver" is which NAPALM driver actually opened the connection (host.platform) -- here, every vendor is tested through NAPALM's mock driver rather than a vendor-specific onevendor and driver are the same field, just named differently by mistakeThis means the connection to CORE-SW1 failed

Real, verified output — both outcomes, not just the happy path

terminal output — real, verified run

--- Inventory loaded from mock NetBox ---
CORE-SW1: platform=mock
DIST-SW1: platform=mock
EDGE-RTR1: platform=mock

--- Compliance evaluation (required: 'ntp server 10.10.10.1') ---
CORE-SW1 [vendor=cisco_ios, driver=mock]: COMPLIANT -- required NTP server present
DIST-SW1 [vendor=arista_eos, driver=mock]: COMPLIANT -- required NTP server present
EDGE-RTR1 [vendor=junos, driver=mock]: NON-COMPLIANT -- missing required line: 'ntp server 10.10.10.1'

--- Summary ---
{'total': 3, 'compliant': 2, 'non_compliant': 1, 'compliance_rate': 0.6666666666666666}

Real, verified. Two of three devices carry the required NTP line; EDGE-RTR1's fixture deliberately omits it, and the report correctly identifies exactly that device as non-compliant -- both outcomes genuinely produced, not just the clean-sweep case.

Quick checkPredict output

If EDGE-RTR1's fixture had also included the required NTP line, what would the summary's compliance_rate have shown?

Still 0.667 -- the rate doesn't reflect fixture content1.0 -- with all three devices carrying the required line, compliant would equal total, per summarize()'s own compliant/total calculation0.0 -- adding a line always triggers a failureThe script would raise an error

What this does and doesn't prove

Never claimed here

This ran against a freshly-built, stdlib-only NetBox-shaped mock and NAPALM's real mock driver -- never real NetBox, never a real device. The compliance logic itself (comparing a required line against real getter output) is genuinely verified; whether real production devices actually carry this NTP line is a separate, untested question.

How to present this in your portfolio

Portfolio guidance

Problem statement: "A required security/operational baseline needed checking across a mixed-vendor fleet whose membership itself changes over time." Architecture: NetBox as dynamic source of truth, NAPALM's normalized getters for vendor-neutral state, one compliance rule applied identically to all three. Evidence: the real output above, showing one genuine non-compliant finding, is stronger portfolio evidence than an all-green report -- it proves the check actually discriminates. Limitations to state honestly: one compliance rule, three mock devices, no real NetBox or hardware involved. Extension ideas: multiple compliance rules evaluated per device (not just one line), or a report that also flags devices NetBox lists with no matching platform record.

Key takeaways

The business problem: a required baseline setting, checked across a mixed-vendor fleet whose membership itself changes in NetBox over time.
The new integration: NetBoxInventory2's dynamic, multi-vendor inventory feeding directly into napalm_get()'s normalized config retrieval.
A freshly-built, stdlib-only NetBox mock (not reused from any earlier phase) implementing exactly the two endpoints NetBoxInventory2 actually calls.
Real, verified run: two compliant devices and one genuine non-compliant finding, both outcomes actually produced -- not only the clean-sweep case.
Mock NetBox and mock NAPALM driver only -- never claimed as evidence about real NetBox or real hardware.

What you'll learn