In a health system, AI spends most of its time:
- Documenting a visit into a structured note
- Condensing a chart before a consult
- Drafting a prior authorization letter
- Suggesting billing codes
- Reformatting a referral
- Answering a portal message
Thousands of these simple tasks a day in a mid-sized system. It’s transcription, summarization, and reformatting, which language models handled competently three generations ago.
However, a smaller set is more challenging. A differential when labs and imaging disagree, or a history buried in 200 pages of outside records. That work earns the more expensive AI.
…a routine visit note and the hardest diagnostic question can cost the same to answer.
Without deliberate model routing, a routine visit note and the hardest diagnostic question can cost the same to answer.
New to some of these terms?
- Token: The unit AI models read and write in. Ordinary prose runs about three-quarters of a word per token, but clinical text is denser, since drug names, lab values, and dosages fragment into several tokens each. Output tokens usually cost several times more than input tokens.
- Inference: The act of running a model to get an answer. Training builds the model; inference is what you pay for every day afterward.
- Model Routing: Sending each request to the cheapest model likely to handle it well, and escalating to a more expensive one when the cheap answer isn’t good enough.
- Open-weight model: A model whose trained values are published, so your organization can run it on hardware you control instead of paying a vendor per token. Open weights don’t mean open licences, and running the model yourself trades token costs for infrastructure costs.
- SLM: A model with far fewer parameters than a frontier model. Cheaper and faster to run, and competitive on narrow or well-defined work.
Rising Spend and Unfunded Controls
An unbudgeted AI bill can be the difference between a profitable quarter and a loss.
Healthcare organizations planning to increase AI budgets in 2026
Health system executives expecting capital investment to decline over the next two years
Hospitals planning to raise AI governance and safety budgets by two or more percentage points
Adoption can outrun the funding behind it. The clearest example on record is outside healthcare. Uber gave 5,000 engineers access to Claude Code in December. Usage went from 32% to 84% by March, at $500 to $2,000 per engineer per month. AI budget was gone four months into the year. Uber absorbed it.
