Understanding async and await in Python
A beginner's guide to asynchronous programming in Python with async and await. Learn how the asyncio library enables cooperative multitasking to build high-performance, I/O-bound applications.
Modern applications often spend a lot of time waiting for things—waiting for a network request to complete, a database query to return, or a file to be read. In traditional synchronous programming, this waiting blocks the entire program. Asynchronous programming is a paradigm that allows your program to do other useful work while it's waiting.
In Python, the modern way to write asynchronous code is with the asyncio library and the async and await keywords, which were introduced in Python 3.5.
The Problem: Synchronous (Blocking) I/O
Let's imagine a simple program that needs to download two web pages.
import requests
import time
def download_site(url):
requests.get(url)
print(f"Downloaded {url}")
start_time = time.time()
download_site("https://www.example.com")
download_site("https://www.example.org")
duration = time.time() - start_time
print(f"Downloaded 2 sites in {duration} seconds")
This code works, but it's inefficient. It makes the first request, waits for it to complete, and only then makes the second request. If each request takes 1 second, the total time will be 2 seconds. The program is spending most of its time just waiting for the network.
The Solution: Asynchronous I/O with asyncio
asyncio allows us to write concurrent code using a single thread. It uses an event loop to manage and distribute the execution of different tasks. This is a form of cooperative multitasking.
To use asyncio, you need to understand two key concepts:
Coroutines: A coroutine is a function that can be paused and resumed. In modern Python, you create a coroutine by defining a function with
async def.The
awaitkeyword: Theawaitkeyword is used inside anasyncfunction to pause its execution and wait for the result of another coroutine. While it's waiting, the event loop is free to run other tasks.
Let's rewrite our download example asynchronously.
import asyncio
import aiohttp # An asynchronous HTTP client library
import time
async def download_site_async(session, url):
async with session.get(url) as response:
print(f"Downloaded {url}")
async def main():
async with aiohttp.ClientSession() as session:
# Create a list of tasks to run concurrently
tasks = [
download_site_async(session, "https://www.example.com"),
download_site_async(session, "https://www.example.org"),
]
# Wait for all tasks to complete
await asyncio.gather(*tasks)
start_time = time.time()
# In Python 3.7+, you can just run asyncio.run(main())
asyncio.get_event_loop().run_until_complete(main())
duration = time.time() - start_time
print(f"Downloaded 2 sites in {duration} seconds")
Now, the program starts the first download, and as soon as it hits the await call, it yields control back to the event loop. The event loop then starts the second download. Both requests are now happening concurrently. If each request takes 1 second, the total time will be just over 1 second, not 2.
async and await: The Rules
- A function defined with
async defis a coroutine. Calling it does not execute it; it just returns a coroutine object. - The
awaitkeyword can only be used inside anasync deffunction. - To run the top-level coroutine, you need to pass it to the
asyncioevent loop (e.g., usingasyncio.run()).
When to Use asyncio
asyncio is best suited for I/O-bound problems. These are problems where your program spends most of its time waiting for external resources, like:
- Network requests (e.g., calling APIs, scraping websites).
- Database connections.
- Reading and writing to files.
It is not well-suited for CPU-bound problems (e.g., heavy mathematical calculations), as it runs on a single thread. For CPU-bound tasks, you should use Python's multiprocessing module.
Conclusion
Asynchronous programming with async and await is a powerful paradigm for writing high-performance, concurrent applications in Python. By allowing your program to perform other work while waiting for I/O, you can build applications that are significantly faster and more efficient. While it takes some getting used to, asyncio is an essential tool for any Python developer building network services or other I/O-intensive applications.