How I Built AI Comparison Pages on a Budget

I needed to build comparison pages for my AI tools directory.

The math was scary. With 200 models, I faced nearly 20,000 possible pairs. Using Claude Haiku to generate content for every pair would cost too much if I ran it daily.

Here is how I solved it using logic and limits.

The Strategy

I focused on high-intent queries. Users want to know "Llama 3 vs Mistral." They want a decision, not a long essay.

I used these rules to control costs:

• Group models by their pipeline tag. • Pick only the top 4 models by download count in each group. • Set a hard limit on total pairs.

This reduced my pairs from 20,000 down to about 50. This keeps my costs low while covering the most popular models.

The Technical Setup

I built an ETL process that runs every night. It is idempotent. This means if the pair already exists in my database, the script skips it.

Most nights, the script runs in 3 seconds and costs $0 because it skips everything.

I use several tricks to keep it efficient:

• Prompt Caching: I use a shared Claude Haiku client. Since the system prompt stays the same, caching makes subsequent calls nearly free. • Lean Prompts: I truncate model summaries to 400 characters. This keeps the input small and fast. • Static Generation: I use Astro to turn the data into static JSON files. There are no slow database calls or edge function latencies for the user.

What Happens when IT fails?

AI can be unpredictable. Sometimes Claude returns bad JSON.

I wrote a fallback system. If the AI fails, the page does not crash. Instead, it shows a standard template. It tells the user to check individual model pages. I track these failures in my database so I can retry them later.

What I would do differently

If I started over today, I would change two things:

  1. Cross-pipeline comparisons: Comparing a text model to a vision model is more valuable than comparing two similar text models.
  2. Search-driven selection: Instead of using download counts, I would use actual user search logs to decide which pairs to build.

Building with constraints forces you to build smarter.

Source: https://dev.to/morinaga/how-i-built-pairwise-ai-model-compare-pages-with-claude-haiku-and-a-budget-cap-3hpm

Optional learning community: https://t.me/GyaanSetuAi