AI subscriptions cost $20/month not because of greed, but because training and running large language models at scale is ferociously expensive. Between GPU scarcity, inference compute, and razor-thin margins, cloud AI providers are burning through billions just to keep the lights on. Here's the infrastructure cost breakdown.
The Money Burning: OpenAI's Economics
OpenAI reported in 2024 that they were consuming approximately $17 billion per year in revenue, with inference costs forming a significant portion of operational expenses. In a high-growth AI company, burning cash on compute is the primary cost driver. Every API call, every ChatGPT Plus subscription, every token processed on your request adds GPU time to their bill.
Consider: ChatGPT Plus is $20/month. A power user who chats 10 hours a month at an average of 2,000 tokens per request (input + output) generates roughly 100,000 tokens per month, or about 3.3 million tokens per year. At OpenAI's reported API pricing ($3 per million input tokens, $6 per million output tokens), that's roughly $20–30 in pure inference cost per subscriber per month—before overhead, salaries, and data center rent. The math barely works unless you're running at scale with efficient utilization.
Anthropic, Google, and Meta face similar economics. Each of these companies is fielding billions in annual compute bills.
H100s and the GPU Cost Trap
Training a large language model requires cutting-edge GPUs. An H100 GPU costs $25,000–40,000 per unit. Training GPT-4 scale models required training on tens of thousands of these chips for weeks or months, costing hundreds of millions of dollars. That's a one-time sunk cost, but it sets a high bar: you need revenue at scale just to recover the training bill.
Inference—running trained models to answer user questions—is the ongoing cost. Inference requires GPU time too. A single H100 can serve roughly 100–300 requests per second, depending on model size and latency requirements. If OpenAI is running millions of concurrent ChatGPT users, they need massive clusters of GPUs, all consuming electricity, generating heat, requiring cooling, and needing continuous replacement as hardware degrades.
The chip shortage of 2021–2023 made this worse. Major AI labs competed for limited H100 availability. Prices spiked. Competitors hoarded chips. By 2024, the situation improved slightly, but cutting-edge GPUs remain scarce and expensive. This scarcity is part of why large language model inference remains so costly.
Inference Margins Are Razor-Thin
Here's the painful truth: inference on large models generates very little profit per request. If you're serving a 70B parameter model, each token generated requires matrix multiplications across billions of parameters. On an H100, that's fast (roughly 10–50 tokens per second depending on optimization), but it's not free. Electricity, cooling, hardware amortization, and labor all add up.
A user paying $20/month might consume $15–25 in compute costs alone. Add engineering salaries, data center leasing, customer support, payment processing, and risk reserves, and the margin per subscriber is thin. Scale is essential: OpenAI's $17B burn reflects massive revenue, but also massive cost.
Smaller models improve margins. A 1B or 7B parameter model runs faster and cheaper than a 70B model. But there's a quality trade-off: smaller models are weaker at reasoning, long context, and specialized knowledge. Cloud providers make a bet: run larger models and accept thinner margins, or shrink model size and frustrate users with lower quality. Most choose the former.
Token Pricing Has Risen 3–5x in Two Years
In 2021, when GPT-3 was new, OpenAI charged roughly $0.002 per 1,000 tokens. Today, the base rate for GPT-3.5-turbo is $0.50–1.50 per million input tokens (depending on context window), and output tokens cost 2–3x more. For GPT-4, input is $3–15 per million tokens, and output is $6–45 per million.
This looks like a price *cut* (from $2 per 1K tokens), but it's misleading. Those older prices were promotional. As demand skyrocketed and compute costs remained high, vendors raised prices closer to true marginal cost. The trend is clear: as more users adopt AI, as demand for inference compute outpaces supply, prices either hold flat (shrinking margins) or rise (maintaining margins, but frustrating users).
Claude's pricing has followed a similar pattern. Gemini's free tier exists partly because Google can subsidize it with other revenue. These pricing floors are not sustainable in a market where compute is expensive and demand is elastic.
The Constant Upgrade Treadmill
Users expect newer, smarter models. OpenAI released GPT-4, then 4o, then 4o Mini. Anthropic released Claude 3 Opus, then Sonnet, then Haiku. Each generation requires re-training, which costs hundreds of millions again. The old model stays online (lower price, lower cost), but the new model adds expense. Companies must balance backwards compatibility with innovation, and both add cost.
Meanwhile, competitors constantly pressure pricing. When Claude launched, it was comparable or better than GPT-4 for many tasks. OpenAI had to cut GPT-4 pricing and invest in GPT-4o to compete. This price war is good for users but devastating for margins. It's why cloud subscriptions are moving toward higher prices or usage tiers: vendors need revenue to fund the next model release.
The Bandwidth and Latency Problem
Serving a model with sub-500ms latency requires careful orchestration. Requests must be routed to the nearest GPU cluster, queued fairly, and executed with minimal overhead. If you're serving global traffic, you need redundancy: multiple data centers, cross-zone replication, failover logic. A millisecond slower inference means unhappy users; but faster inference means more GPUs idling during traffic valleys.
Bandwidth is also expensive. Serving a model to millions of concurrent users means pushing petabytes of data per day through network pipes. Major cloud providers negotiate bulk bandwidth rates, but it still adds up. A user in Tokyo hitting a server in Virginia incurs latency and bandwidth cost.
What Local Economics Look Like
On-device models have different costs. A 1B or 7B parameter model (GGUF format) is 2–20 GB of data downloaded once at setup. After that, inference runs on the user's device: their CPU/GPU, their electricity, their RAM. From the vendor's perspective, there's no per-request cost. A $2 one-time purchase covers the model license, code updates, and minimal server overhead (sync, optional web search, analytics).
The trade-off is quality: a 7B local model won't match a 70B cloud model on hard reasoning or specialized knowledge. But for everyday chat, summarization, code help, and writing, a well-optimized 7B or 2B model is often enough. The infrastructure cost advantage is enormous: local models amortize across the device's lifetime, not per-request.
Why Subscriptions Will Stay Expensive
For users who need the best-in-class reasoning and live web knowledge, cloud subscriptions are the rational choice. The quality gap justifies the cost. But that quality comes from compute-heavy models that vendors must run at scale, with margins barely covering the cost.
What will change is segmentation. Free tiers will remain limited—capped by hourly request limits or reduced model quality—to protect vendor margins. Premium tiers will rise in price. Some vendors will invest in cheaper, smaller models to serve cost-conscious users. And on-device AI will continue to improve, offering users a third option: lower quality but no recurring cost.
The economic reality is that cloud inference is fundamentally expensive. Until electricity becomes cheaper, GPUs become faster and more plentiful, or techniques for running models more efficiently emerge, subscriptions will remain in the $10–30/month range for meaningful access. That's not greed. That's the real cost of the metal, power, and labor needed to run a global AI service.
The Choice
Understanding the economics helps clarify the trade-off. Cloud AI is expensive because it's powerful, always-on, and constantly improving. But that expense compounds if you're paying monthly for years. One-time purchases shift the burden to your device, which means lower quality but genuine privacy and no recurring bill. A local AI app is a rational alternative if you accept a smaller model's limitations and value the economic advantage.
The infrastructure costs are real, and the economics of cloud AI are constrained by physics and supply chains. Understanding why subscriptions cost what they do helps you make an informed choice about which tool fits your needs and your budget. If the honest answer for your work is "I mostly need quick answers, drafting and summaries, and I would rather not rent that forever," the maths favours owning the tool outright. Get MyBenAI for a one-time $2 and run the model on hardware you have already paid for — or read the full cost-of-ownership comparison over five years before you decide.