Understanding async and await in Python

Modern applications often spend a lot of time waiting for things—waiting for a network request to complete, a database query to return, or a file to be read. In traditional synchronous programming, this waiting blocks the entire program. Asynchronous programming is a paradigm that allows your program to do other useful work while it's waiting.

In Python, the modern way to write asynchronous code is with the asyncio library and the async and await keywords, which were introduced in Python 3.5.

The Problem: Synchronous (Blocking) I/O

Let's imagine a simple program that needs to download two web pages.

import requests
import time

def download_site(url):
    requests.get(url)
    print(f"Downloaded {url}")

start_time = time.time()
download_site("https://www.example.com")
download_site("https://www.example.org")
duration = time.time() - start_time
print(f"Downloaded 2 sites in {duration} seconds")

This code works, but it's inefficient. It makes the first request, waits for it to complete, and only then makes the second request. If each request takes 1 second, the total time will be 2 seconds. The program is spending most of its time just waiting for the network.

The Solution: Asynchronous I/O with asyncio

asyncio allows us to write concurrent code using a single thread. It uses an event loop to manage and distribute the execution of different tasks. This is a form of cooperative multitasking.

To use asyncio, you need to understand two key concepts:

1. Coroutines: A coroutine is a function that can be paused and resumed. In modern Python, you create a coroutine by defining a function with async def.

2. The await keyword: The await keyword is used inside an async function to pause its execution and wait for the result of another coroutine. While it's waiting, the event loop is free to run other tasks.

Let's rewrite our download example asynchronously.

import asyncio
import aiohttp # An asynchronous HTTP client library
import time

async def download_site_async(session, url):
    async with session.get(url) as response:
        print(f"Downloaded {url}")

async def main():
    async with aiohttp.ClientSession() as session:
        # Create a list of tasks to run concurrently
        tasks = [
            download_site_async(session, "https://www.example.com"),
            download_site_async(session, "https://www.example.org"),
        ]
        # Wait for all tasks to complete
        await asyncio.gather(*tasks)

start_time = time.time()
# In Python 3.7+, you can just run asyncio.run(main())
asyncio.get_event_loop().run_until_complete(main())
duration = time.time() - start_time
print(f"Downloaded 2 sites in {duration} seconds")

Now, the program starts the first download, and as soon as it hits the await call, it yields control back to the event loop. The event loop then starts the second download. Both requests are now happening concurrently. If each request takes 1 second, the total time will be just over 1 second, not 2.

async and await: The Rules

• A function defined with async def is a coroutine. Calling it does not execute it; it just returns a coroutine object.
• The await keyword can only be used inside an async def function.
• To run the top-level coroutine, you need to pass it to the asyncio event loop (e.g., using asyncio.run()).

When to Use asyncio

asyncio is best suited for I/O-bound problems. These are problems where your program spends most of its time waiting for external resources, like:

• Network requests (e.g., calling APIs, scraping websites).
• Database connections.
• Reading and writing to files.

It is not well-suited for CPU-bound problems (e.g., heavy mathematical calculations), as it runs on a single thread. For CPU-bound tasks, you should use Python's multiprocessing module.

Conclusion

Asynchronous programming with async and await is a powerful paradigm for writing high-performance, concurrent applications in Python. By allowing your program to perform other work while waiting for I/O, you can build applications that are significantly faster and more efficient. While it takes some getting used to, asyncio is an essential tool for any Python developer building network services or other I/O-intensive applications.