Async / Await

async/await overlaps I/O waits on one thread—concurrency without threads.

Python · Advanced · ~5 min

In plain English

While one task is waiting for the network, let another one run. Not more workers — one worker that stops twiddling its thumbs.

Why it's worth your time

Almost every AI application is IO-bound on model calls, and this is the difference between 40 sequential requests and 40 concurrent ones.

If you remember three things

Concurrency for waiting, not parallelism for computing
One blocking call freezes the entire event loop
asyncio.gather is how you actually get concurrency

Overview

An async def defines a coroutine that runs on a single-threaded event loop. await suspends the coroutine at an I/O point and hands control back to the loop, which runs whichever coroutine is ready. This overlaps many I/O waits concurrently without threads—but gives no speedup for CPU-bound work.

In an interview

Async lets one thread juggle many I/O-bound tasks. An async def is a coroutine; await marks a point where it suspends and returns control to the event loop, which then runs other ready coroutines. asyncio.gather launches many at once so their waits overlap instead of running one after another. It shines for network, disk, and DB calls, but does nothing for CPU-bound work—there you need multiprocessing.

Production defaults

Use for: IO-bound work: HTTP, database, model calls. Not for CPU-bound work
Fan out: asyncio.gather(*tasks) — awaiting in a loop is just sequential code with extra syntax
Blocking code: wrap it in asyncio.to_thread / run_in_executor, or it stalls everything
Limit: a semaphore to cap concurrent calls; unbounded fan-out will get you rate-limited

What breaks

Async code is no faster — You awaited each call in a loop. That's sequential. Use gather.
Everything freezes intermittently — A blocking call in the event loop — requests, time.sleep, a heavy CPU function. Move it off the loop.

Watch it explained

AsyncIO, await, and async - Concurrency in Python — Socratica, 9:12