Async Programming and Concurrency

Duration: 8 min

Learn to write concurrent code for handling multiple AI model inference requests efficiently.

Async/Await Basics

Async programming allows your code to handle multiple operations without blocking.

import asyncio

async def fetch_model_data(model_id):
    print(f"Fetching model {model_id}...")
    await asyncio.sleep(2)  # Simulate API call
    return f"Model {model_id} data"

async def main():
    results = await asyncio.gather(
        fetch_model_data(1),
        fetch_model_data(2),
        fetch_model_data(3)
    )
    print(results)

asyncio.run(main())

Try it in Google Colab:

Fetching model 1...
Fetching model 2...
Fetching model 3...
['Model 1 data', 'Model 2 data', 'Model 3 data']

Threading for Parallel Tasks

Use threading for I/O-bound operations like API calls or file processing.

import threading
import time

def process_batch(batch_id):
    print(f"Processing batch {batch_id}")
    time.sleep(1)
    print(f"Batch {batch_id} complete")

threads = []
for i in range(3):
    t = threading.Thread(target=process_batch, args=(i,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

Processing batch 0
Processing batch 1
Processing batch 2
Batch 0 complete
Batch 1 complete
Batch 2 complete

💡 Tip: Use async/await for I/O-bound operations and threading for CPU-bound tasks in your AI pipelines.