Write code that tries to SSH into several sites in parallel and you’ll quickly run into two questions: how many connections should run at once, and in what order should the results come back? In Python, the foundation for both is the standard library’s queue module, and the concurrent.futures.ThreadPoolExecutor built on top of it. This article looks at how the two relate, and at a property that’s easy to overlook: the order tasks are submitted in is not the same as the order they finish in.
What queue.Queue actually is
Note:
queue.Queueis a FIFO (first-in, first-out) data structure for safely passing items between threads. Conceptually it’s no different fromappend/popon a list — the difference is that its internal locking makes it safe for multiple threads to touch at the same time.
The two core operations are put() (add an item) and get() (remove one). When the queue is empty, get() blocks the calling thread until something arrives (with an optional timeout). That “wait if empty” behavior is what makes the classic producer-consumer pattern easy to write: a producer thread can put() items as fast as it likes without worrying about how quickly the consumer keeps up.
queue.Queue also offers maxsize (a cap that keeps an overeager producer from running away) and task_done()/join() (a way to block until everything that’s been submitted has actually been processed). The standard library also ships a lighter variant without those extras, queue.SimpleQueue — and that’s the one ThreadPoolExecutor actually uses internally.
There’s a queue inside ThreadPoolExecutor too
ThreadPoolExecutor spins up a fixed number of worker threads and lets you hand work to them, and the mechanism it uses to hand that work off is exactly the queue described above. Looking at CPython’s implementation (concurrent/futures/thread.py), ThreadPoolExecutor.__init__ contains this line:
self._work_queue = queue.SimpleQueue()
Every call to executor.submit(fn, *args) puts a (function, arguments) pair onto _work_queue. Each running worker thread, meanwhile, loops forever calling get() on that same queue — pulling off whatever’s next, running it, and going back for more. In other words, ThreadPoolExecutor is a producer-consumer pattern built on queue.Queue (or, more precisely, the lightweight SimpleQueue), wrapped around a fixed pool of worker threads. The fact that it doesn’t use maxsize or task_done()/join() is a sign that SimpleQueue is all it needs.
A real example: checking plugin updates across sites
Our own maintenance tool uses ThreadPoolExecutor for exactly this shape of problem — SSHing into multiple WordPress sites in parallel to check for pending plugin updates (site_manager_web.py::aggregate_pending_updates). Stripped to the essentials, it looks like this:
with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as ex:
future_to_site = {
ex.submit(_fetch_pending_plugins_for_site, s): s for s in all_sites
}
for fut in concurrent.futures.as_completed(future_to_site):
r = fut.result()
# aggregate r...
max_workers is taken from the API request body, defaulting to 8 and clamped to a maximum of 16. When there are more sites than that, each ex.submit() call lands its task on the internal work queue described above, and whichever worker thread is free next pulls it off and runs a single SSH + WP-CLI call for that site. Up to this point, tasks are picked up roughly in submission order — the order the sites were loaded in — because the underlying queue is FIFO.
Submission order and completion order are two different things
What happens after a task starts is a different story. SSH response times vary from site to site depending on server load, network path, and how many plugins are installed, so it’s completely normal for a site that started later to finish sooner. That’s exactly why the code above uses concurrent.futures.as_completed(future_to_site) — it hands back each Future as soon as it finishes, in completion order, not submission order. A single slow site no longer blocks the results of the sites that finished ahead of it.
Contrast this with ThreadPoolExecutor.map(), which offers a different guarantee: results come back in the same order they were submitted in. To honor that guarantee, map() has to hold back the second result until the first one is ready, no matter how much faster the second task actually finished. If the first site in the list happens to have an unusually slow connection, iterating over map()‘s results stalls right there even though later sites are already done. For a job like aggregate_pending_updates, which waits for every site before returning a combined response, this distinction doesn’t change the final wait — but for any use case that wants to act on results as they arrive, map()‘s ordering guarantee works against you. The submit() + as_completed() combination is the explicit choice for exactly that situation: keep submission order, but let results come back in whatever order they actually finish.
When order does matter
There are also cases where you genuinely want results back in the same order as the inputs — say, fetching several pages of content in parallel and then stitching them back together into one document in sequence. There, map()‘s ordering guarantee is exactly what you want. Whether work runs in parallel and what order its results come back in are two separate design decisions, and ThreadPoolExecutor lets you express either combination just by choosing between submit()+as_completed() and map(). The queue underneath is the same FIFO queue.SimpleQueue either way — it’s the API you call on top of it that decides whether order is preserved.
Summary
queue.Queue (and its lighter sibling queue.SimpleQueue) is a FIFO structure built for safely passing work between threads, and ThreadPoolExecutor is that mechanism wrapped around a pool of worker threads. Tasks are picked up in roughly the order they’re submitted, but because each task takes a different amount of time to run, the order they finish in isn’t guaranteed to match. concurrent.futures.as_completed() embraces that reality by returning results in completion order, which is a genuinely different design choice from the ordering guarantee map() provides. Keeping that distinction straight makes both reading and writing concurrent code a lot less surprising.