31.8x Speedup With One File Change

I tested a RAG ingestion pipeline and hit a bottleneck: one document took 50 seconds to process.

My CPU sat idle. The app wasn’t doing math or logic; it just waited for an HTTP request to finish before launching the next.

I rewrote the embedding module to run asynchronously. The runtime fell from 49.61 seconds to 1.56 seconds—no infrastructure changes.

Test Details

  • Model: Amazon Titan Text Embeddings V2 (AWS Bedrock)
  • Dataset: 33 text chunks
  • Region: us-east-1

Results

  • Sequential: 49.61 s
  • Concurrent: 1.56 s
  • Speedup: 31.8×

Why it works Sequential code sends a request and blocks until it returns, repeating for every chunk. With 33 chunks at 1.5 s each, you waste ~50 s.

Async code fires all requests together; total time equals the slowest single request.

Scaling impact

  • 31 chunks: 49.61 s → 1.56 s
  • 100 chunks: ~160 s → ~3 s
  • 500 chunks: ~800 s → ~5 s
  • 1,000 chunks: ~1,600 s → ~10 s

Tips for your pipeline

  • Look for idle time. Don’t add hardware if your code is just waiting on the network.
  • Respect AWS Bedrock rate limits. Use an asyncio.Semaphore to cap concurrent requests.
  • Prefer non-blocking libraries. Switch from boto3 to aioboto3 or wrap calls with asyncio.to_thread().

Changing a single file removed the bottleneck. Low effort. High impact.

Source: https://dev.to/edwardyun/318x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock-4l61

Optional learning community: https://t.me/GyaanSetuAi