How to retry a failed operation is a design question every network-facing piece of code eventually has to answer. The textbook technique is exponential backoff — doubling the wait time on each retry — but it isn’t automatically the right answer everywhere. This post works through what problem exponential backoff actually solves, then looks at three real retry paths in this app’s own code where fixed intervals were chosen instead, and why.
The Problem Exponential Backoff Solves
Note: exponential backoff is a retry strategy where the wait time between attempts grows exponentially — 1s, 2s, 4s, 8s, and so on — instead of staying constant.
base = 1
for attempt in range(6):
delay = base * (2 ** attempt)
print(f"attempt {attempt}: wait {delay}s")
# attempt 0: wait 1s
# attempt 1: wait 2s
# attempt 2: wait 4s
# attempt 3: wait 8s
# attempt 4: wait 16s
# attempt 5: wait 32s
Exponential backoff earns its keep in one specific situation: many independent clients retrying against the same shared resource at once. When a server starts failing under load, and every client retries at a fixed interval, the retries arrive in synchronized waves that keep hitting the already-struggling server at the same cadence — a pattern often called a “thundering herd.” Spacing each client’s retries out exponentially means those waves drift apart over time, giving the server room to recover. That’s why most HTTP client libraries and cloud SDKs ship exponential backoff by default.
The flip side is that this only pays off when many independent clients are actually contending for the same resource. Where that condition doesn’t hold, exponential backoff adds complexity without buying anything. Here are three retry sites in this codebase where it was deliberately skipped.
Example 1: A Single Fixed Retry for HTTP Checks
This app checks a site’s HTTP status before and after WordPress updates, rolling back if things got worse. The core of that logic is maintenance_agent.py::_http_status_check_stable():
def _http_status_check_stable(url, timeout=15, retry_delay=3, basic_auth=None):
status = _http_status_check(url, timeout=timeout, basic_auth=basic_auth)
if status == 0:
try:
import time as _time
_time.sleep(retry_delay)
except Exception:
pass
status = _http_status_check(url, timeout=timeout, basic_auth=basic_auth)
return status
Only status == 0 (a failure before the response even reaches the server — DNS blip, TLS handshake failure, a momentary connection drop) triggers a retry, and it’s exactly one retry after a flat 3-second wait. A 5xx response isn’t retried at all — once a response actually comes back, the code treats that as a genuine server-side problem rather than a transient network hiccup.
There’s no exponential growth anywhere in this function, and the reason is in a comment elsewhere in the same file: this function gets called from up to five different call sites for a single site during one maintenance run — a baseline check, a post-core-update check, post-rollback checks, and a check after each individual plugin update. On a site with twenty plugins, that call count adds up fast. If each of those calls retried multiple times with growing delays, the total time a single site’s maintenance run could take would become unpredictable, which matters a lot for an unattended scheduled run where per-site duration needs to stay roughly bounded. Telling apart “a momentary blip” from “a genuinely down server” doesn’t require patiently waiting longer and longer — one fixed retry is enough.
Example 2: Polling for a File Lock at a Fixed Interval
Two processes in this app — the GUI web server and a background maintenance run launched as a separate process — can both try to write the same config file (sites_*.json etc.) at once. core/file_lock.py::FileLock prevents that collision:
def acquire(self) -> None:
deadline = time.monotonic() + self.timeout
while True:
if self._try_acquire_once():
return
if time.monotonic() >= deadline:
raise FileLockTimeout(
f"Failed to acquire lock within {self.timeout}s: {self.lock_path}"
)
time.sleep(self.poll_interval)
The defaults are timeout=10.0 and poll_interval=0.1 — it polls every 0.1 seconds until either the lock is acquired or ten seconds pass, with no growth in the interval at all.
Here again, exponential backoff wouldn’t fit the actual contention pattern. This lock is contended by, at most, two processes belonging to the same app — not an unbounded number of independent clients — so there’s no thundering herd to prevent in the first place. And since the wait is local disk I/O rather than load on a remote server, the cost of polling frequently is just a bit of wasted CPU wake-ups, not risk to a shared resource. Given that, polling at a short fixed interval means the lock gets picked up close to the moment it’s released; a growing interval would risk sitting through an unnecessarily long gap right when the lock happens to free up.
Example 3: Retrying by Changing Approach, Not by Waiting
core/ssh_utils.py has a different kind of retry when fetching the plugin list:
res = c.run(
f"{wp_with_plugins} plugin list --update=available --format=json",
hide=True, warn=True, encoding='utf-8'
)
if not (res.ok and res.stdout.strip()):
# fall back: retry with all plugins skipped
res = c.run(
f"{wp_safe} plugin list --update=available --format=json",
hide=True, warn=True, encoding='utf-8'
)
The first attempt runs WP-CLI without --skip-plugins, so any plugin’s own update-detection hook can fire. If that fails, the fallback switches immediately — with no delay at all — to a safer command that skips every plugin. This isn’t the kind of failure that waiting resolves; the suspected cause is a specific plugin interfering with the command itself, a structural problem rather than a transient one. So the retry axis here isn’t “how long to wait” but “which approach to use.”
Takeaway
Exponential backoff earns its complexity when many independent clients might hammer the same resource at once, spreading out the resulting wave of retries so an overloaded server gets room to recover — a solid default for calls to external APIs and cloud services. But when the retry count is inherently small (one HTTP recheck), when contention is limited to a couple of processes on the same machine (a local file lock), or when the failure isn’t the kind that time alone fixes (switching strategy instead of retrying the same command), fixed intervals — or no delay at all — are simpler and don’t leave anything on the table. The question worth asking before reaching for backoff isn’t “how should the wait time grow,” but “what’s actually contending for what, and would waiting longer even help.”