The question is no longer whether AI is useful for your business. The question is whether you should pay a cloud provider per token or buy hardware once and pay for electricity. By the end, you will have the tables and the formula, using September 2026 API prices, ready for a conversation with a finance director.
What you need
- Your monthly query volume and a rough estimate of tokens per query (a factor-of-two error is fine for the first pass).
- The cloud price lists: OpenAI and Anthropic.
- A spreadsheet or Python. An AI assistant can also do the arithmetic if you paste the tables in.
Step 1: calculate your cloud bill
Providers charge per token, roughly per word processed. At low volume it looks trivial; at the scale a real business uses AI, the arithmetic changes.
A typical SME workload (100,000 queries a month at 500 input and 300 output tokens each) produces this bill with GPT-5, at 1.25 USD per million input tokens and 10 USD per million output tokens, treating 1 USD as 1 EUR, which favours the cloud:
| Cost component | GPT-5 (cloud) | Qwen3-8B (local) |
|---|---|---|
| Input tokens (50M/month) | EUR 63 | EUR 0 |
| Output tokens (30M/month) | EUR 300 | EUR 0 |
| Electricity (200 W, 8 h/day, EUR 0.27 per kWh) | EUR 0 | EUR 13 |
| Total monthly running cost | EUR 363 | EUR 13 |
| Annual running cost | EUR 4,356 | EUR 156 |
| One-time hardware (assumed: a GPU workstation) | EUR 0 | EUR 2,400 |
The API lines come straight from OpenAI’s published prices; the electricity line is 200 W × 8 h × 30 days = 48 kWh a month. Year one of the local option adds the one-time hardware, so its year-1 total is about EUR 2,560; from year two, about EUR 156 plus your own maintenance time. Add to both columns the costs only you can price: staff time for updates, any integration work, and on the cloud side any extra compliance paperwork.
An 8B model such as Qwen3-8B, served with Ollama, covers many everyday SME tasks: document summaries, internal Q&A, classification, drafting and data extraction. For frontier-level reasoning, a hybrid approach sends only the hard queries to the cloud; the saving is whatever share of your tokens you move local.
Step 2: calculate the total cost of ownership
Pricing calculators show the API cost. None of them show the full stack. List these items and get your own quotes or estimates for each:
Cloud costs that do not appear in the calculator:
- Data egress, if your provider or cloud platform charges for it.
- Higher rate-limit tiers, if your volume needs them.
- GDPR work: processing agreements, standard contractual clauses and transfer documentation.
- Switching cost if you ever change provider: re-writing prompts, integrations and tests.
Honest local costs:
- Hardware, amortised over three years. Price the exact machine you need; our hardware catalog has sourced prices by budget.
- Initial setup and integration, whether done in-house or by a provider.
- Electricity: watts × hours × your tariff. For example, a 300 W workstation running twelve hours a day uses about 1,314 kWh a year, about EUR 369 at EUR 0.28 per kWh.
- Model updates and maintenance time.
- Redundancy (a second machine or spare parts) if the system is critical.
Cloud TCO = (monthly API × 12) + compliance + egress
Local TCO year 1 = hardware + setup + (electricity × 12) + (maintenance × 12)
Local TCO year 2 = (electricity × 12) + (maintenance × 12)Step 3: find your break-even point
xychart-beta
title "Cumulative cost: cloud vs local (EUR, 24 months)"
x-axis ["M1", "M2", "M3", "M4", "M5", "M6", "M7", "M8", "M9", "M10", "M11", "M12", "M18", "M24"]
y-axis "Cumulative cost (EUR)" 0 --> 10000
line [363, 726, 1089, 1452, 1815, 2178, 2541, 2904, 3267, 3630, 3993, 4356, 6534, 8712]
line [2413, 2426, 2439, 2452, 2465, 2478, 2491, 2504, 2517, 2530, 2543, 2556, 2634, 2712]Months to break-even = (hardware + setup) / (monthly cloud cost − monthly local operations)
Example: EUR 2,400 / (EUR 363 − EUR 13) = 6.9 months| Queries per month | Cloud per month (GPT-5) | Local electricity per month | Break-even on EUR 2,400 of hardware |
|---|---|---|---|
| 10,000 | ~EUR 36 | EUR 13 | ~103 months: cloud wins at this volume |
| 30,000 | ~EUR 109 | EUR 13 | ~25 months |
| 50,000 | ~EUR 182 | EUR 13 | ~14 months |
| 100,000 | ~EUR 363 | EUR 13 | ~6.9 months |
| 250,000 | ~EUR 908 | EUR 39 (24 h/day) | ~2.8 months |
Same assumptions as step 1: 500 input and 300 output tokens per query, GPT-5 list prices, 1 USD treated as 1 EUR. If maintenance costs you, say, EUR 50 a month in staff time, subtract it too: at 100,000 queries the break-even moves from 6.9 to 8 months.
Conclusion: local AI is not always the answer. At a few tens of thousands of queries a month the cloud is cheaper, unless privacy, latency or regulation outweigh cost. Around 100,000 queries a month and above, local pays for itself in under a year on these assumptions. If your volume is small and sensitive, the EUR 250 board or the PC you already own is the low-barrier entry point.
Step 4: place your company
| Profile | Typical use | Where local usually fits |
|---|---|---|
| Solo practitioner or micro-business (1 to 3 people) | Research, proposals, client documents | Mostly subscriptions today; the main benefit of local is not sending client contracts to an API outside the EU, not cost |
| SME (10 to 50 people) | Knowledge base, HR, support triage, meeting notes | One workstation often covers the load; run step 3 with your real volume |
| Mid-market (50 to 500 people) | 100+ users, several pipelines | Several workstations or a small server; volume is usually high enough for the cost case to matter |
Beyond cost
Numbers get the finance director’s attention; these arguments close the decision.
- Latency. A cloud call includes a network round trip and sometimes a queue; a 7 to 8B model on a local GPU removes both. For live chat, voice assistants and inline annotation, measure time to first token on both options with your real prompts.
- Data sovereignty. What is processed locally never leaves your infrastructure: no international transfers (GDPR Articles 44 to 49), no sub-processor list, no explaining to clients and staff where their data goes. In healthcare, finance and legal it is a requirement, not a preference.
- Vendor independence. No OpenAI outage, AWS region failure or API deprecation notice touches your operations. Cloud prices have changed several times in 24 months; a model on your own machine does not reprice itself.
- Predictable costs. Cloud billing is variable by design. Local AI has a fixed operating cost after deployment, which simplifies budgeting.
Step 5: present it to the finance director
- Start with current spend. “What do we spend today on AI tools and API access across all teams?” Once subscriptions and API invoices are added up, the figure usually surprises.
- Present three scenarios. A: do nothing (cloud grows with usage). B: hybrid (local for volume and sensitive data, cloud for the hard cases). C: full local stack (maximum saving, upfront investment).
- Lead with break-even, not savings. “We recover the investment in seven months; after that, every month is margin.” Finance people distrust savings claims and trust break-even because it is checkable.
- Quantify the risk that disappears. Employee and customer data stops leaving the company, and with it the part of GDPR exposure tied to transfers.
- Propose a pilot. Thirty days, one use case, a fixed scope and budget agreed upfront, deliverable: a working integration and a 90-day ROI report with success criteria agreed in writing.
Next steps
- Run the calculation with your own tokens: Cloud vs local AI: calculate your break-even in 15 minutes.
- Choose the machine: Edge AI hardware guide 2026.
- Where agent ROI really comes from: AI agents for SME automation.
- Guided estimate: our ROI calculator produces a report you can share with your team.
Work with us
We can run this TCO analysis with your real token counts before recommending hardware. A 30-minute conversation is usually enough to tell whether local AI makes financial sense in your case: book a call or see how we work in consulting.