Retry Policies, Not Blind Retry

Not every failure deserves a second try

"Connecting to a Device with Netmiko" already taught you two different exceptions for two different problems: a timeout (no response — a network-path problem) and an authentication failure (a response, but rejected credentials). This lesson is about what to actually do with that distinction: a timeout is often worth trying again — the device might have been briefly busy. An authentication failure never is — the same wrong password will be wrong again, instantly, every time you retry it.

Blind retry is not reliability

Retrying everything, unconditionally, forever, isn't more reliable than not retrying at all — it just means a script that will never succeed takes much longer to fail, and a script retrying the right thing has no limit on how long it might hang. Reliable retry is bounded (a maximum number of attempts) and selective (only for failures that might actually resolve).

A bounded, selective retry wrapper

retry.py

import time


def call_with_retry(connect_fn, device, max_attempts=3, delay=2):
  """
  Calls connect_fn(device), retrying on a timeout up to max_attempts
  times. Never retries a permission/authentication failure -- retrying
  the same wrong credentials wastes time and won't change the outcome.
  """
  for attempt in range(1, max_attempts + 1):
      try:
          return connect_fn(device)
      except TimeoutError:
          if attempt == max_attempts:
              raise
          print(f"  Attempt {attempt} timed out — retrying in {delay}s...")
          time.sleep(delay)
      except PermissionError:
          raise

max_attempts bounds how many times this will ever try — never infinite. The except TimeoutError branch waits and loops again, but only if attempts remain; the except PermissionError branch re-raises immediately, on the very first failure, no retry at all.

Quick checkConcept check

In call_with_retry, why does the except PermissionError branch re-raise immediately instead of retrying?

Because PermissionError can't be caught more than onceBecause an authentication failure is a permanent problem — the same wrong credentials will be rejected identically on every attempt, so retrying wastes time without changing the outcomeBecause retrying PermissionError would cause an infinite loopThere's no real reason — it's an arbitrary choice

Verifying it without a device

Retry logic is pure control flow — it's fully testable with fake functions standing in for a real connection, no device required:

test_retry.py

call_count = {"n": 0}

def flaky_connect(device):
  call_count["n"] += 1
  if call_count["n"] < 3:
      raise TimeoutError("simulated timeout")
  return "connected"

result = call_with_retry(flaky_connect, "SW1", max_attempts=3, delay=0)
print(result)               # connected
print(call_count["n"])      # 3 -- it took three attempts

flaky_connect fails twice, then succeeds — a realistic stand-in for a device that was briefly slow. This test was actually run while writing this lesson: it prints 'connected' after exactly 3 calls, confirming the retry loop keeps going until it succeeds or runs out of attempts.

Quick checkPredict output

Given a fake connect function that raises TimeoutError every single time, no matter how many times it's called, what does call_with_retry(that_function, device, max_attempts=3, delay=0) do?

Returns None after 3 attemptsRetries forever, since TimeoutError is always retriedRaises TimeoutError after exactly 3 attemptsRaises TimeoutError immediately, without retrying at all

Quick checkIdentify the correct code

Which condition correctly ensures the retry loop gives up after the final attempt, instead of retrying one time too many or one time too few?

if attempt > max_attempts: raiseif attempt == max_attempts: raiseif attempt == 1: raiseif delay == 0: raise

Connecting this back to Netmiko

The real connection function from "Connecting to a Device with Netmiko" already distinguishes NetmikoTimeoutException from NetmikoAuthenticationException — this lesson's pattern applies directly: wrap the ConnectHandler(...) call in this same bounded loop, retrying only the timeout branch. Lesson 5 does exactly that, folding this retry pattern into the real connection function without changing what it promises to callers — still a device in, a state or a ConnectionError out.

Quick checkScenario

A device times out on the first connection attempt, then connects successfully on the second. From the perspective of code that called connect_and_check() (or call_with_retry), what does it see?

Two separate error reports, one per attemptA single successful result — the retry happened transparently, and the caller only sees the final outcomeAn exception, even though the second attempt succeededThe caller has to manually call the function twice itself

Key takeaways

Reliable retry is bounded (a maximum attempt count) and selective (only for failures that might actually resolve) — never infinite, never blind.
A timeout is worth retrying; an authentication failure never is — the same wrong credentials fail identically every time.
call_with_retry(connect_fn, device, max_attempts, delay) wraps any connection function with this policy without changing what it returns on success.
Retry logic is pure control flow, fully testable with fake functions that fail a controlled number of times — no device needed.
This same pattern gets folded directly into the real Netmiko connection function in Lesson 5, wrapping only the timeout path.

What you'll learn

Not every failure deserves a second try

A bounded, selective retry wrapper

Verifying it without a device

Connecting this back to Netmiko