April 27, 2026
Managing Partner
Most fine-tuning proposals focus on the wrong economics: the training run is cheap, but the evaluation, data, retraining, privacy, and operating model determine whether the investment makes sense. This article gives executives a practical decision rule for when fine-tuning should be approved, when RAG and prompting remain superior, and why no proposal should move forward without a labeled evaluation set and measured baseline.
A team brings you a fine-tuning proposal. The deck is polished. The argument is plausible. The Year 1 ask is somewhere between $1.5M and $3M. You have thirty minutes to decide whether to approve it. The question that decides the right answer is whether the proposal would survive the unit economics. Most don’t, and the reason is structural: the cost components teams optimize for GPU hours, training tokens, LoRA rank are the smallest line items on the actual bill. The cost components that determine the answer, labeled evaluation sets, retraining cadence, and the operating model around the model usually appear on no slide.
This article presents the cost model we built to test the architecture decision, the production-usage data that grounded our framework, and the four-signal rule we now use with clients. Two of the signals are knockouts: present alone, each is sufficient to justify fine-tuning. The other two are reinforcing: present together, they justify it. All four sit downstream of an evaluation gate that we’d argue is the binding constraint on whether any of this works in practice.
Before we framed the decision, we wanted to know what the bill actually looked like across architectures. We built a unit-economics model that compares three production patterns at sustained query volumes from 100 to 1,000,000 per day.
The model compares three architectures. The first is pure prompting on a frontier model (Claude Sonnet 4.6), with a 50% prompt-cache hit rate, the simplest possible architecture. The second is RAG plus structured prompting Sonnet 4.6 again, with retrieval against a Pinecone serverless index of roughly 500,000 documents. The third is RAG plus a fine-tuned, smaller model Llama 3.3 8B trained with LoRA on Together AI, served at base-model rates, with the same retrieval layer underneath.
Pricing for all three was pulled from vendor documentation in April 2026 and held constant across scenarios. Loaded organizational costs assumed fully burdened ML and data engineering compensation, an evaluation infrastructure of the RAGAS, Promptfoo, or Patronus class, data curation at $8,000 per thousand examples, and an annual retraining cycle for the fine-tune scenario to address data drift.
The result, plotted on log-log axes:
Below approximately 45,000 queries per day, the three curves stack in the same order: pure prompting, cheapest; RAG, next; fine-tuning, most expensive. Between 1,000 and 10,000 queries per day, the volume profile most enterprise pilots actually live at the gap between the prompt-only path and the fine-tune path is roughly $1.1M over three years. That gap is almost entirely fixed organizational overhead.
Above 100,000 queries per day, the curves invert. At a million queries per day, fine-tuning’s three-year TCO is about $2.6M. The same workload on RAG with Sonnet 4.6 is $15.4M. Pure prompting at that volume is $10.5M. Here, the question is no longer whether to fine-tune; it is whether the team can afford not to.
The mechanics behind the inversion are visible at the per-query level:
A million queries on RAG plus Sonnet 4.6, even with caching applied to the system prompt, costs $12,720 in inference. The same million queries on a fine-tuned Llama 8B costs $522, a roughly 24x reduction. Two factors drive it: a smaller model with cheaper per-token rates, and a shorter prompt because the system prompt and few-shot examples have been absorbed into the model weights. At 1,000 queries per day, the savings amount to nothing in the context of a $1M+ TCO. At 1,000,000 queries per day, the savings compound to the dominant line item.
The break-even volume at which the fine-tune curve crosses the RAG curve falls at 44,719 queries per day under the workload profile we modeled. Different profiles move the number. Heavier output tokens push it lower; longer retrieval contexts push it higher. The model is published alongside this article; teams should reproduce it on their own profile rather than treating the 45K figure as universal.
What matters is not the specific number but the shape of the answer. There is a real crossover; it lives in a defensible volume band, and the per-query economics that drive it are mechanical, not philosophical.
The crossover analysis raises a more uncomfortable question: at the volumes where most enterprise teams operate, what is the fine-tuning bill actually composed of?
We decomposed the three-year TCO at three representative volumes. The result is the figure we now use to anchor the conversation with engineering teams.
At 10,000 queries per day, training compute is 0.0% of fine-tuning TCO. At 100,000 queries per day, training compute is 0.0% of fine-tuning TCO. At 1,000,000 queries per day, training compute is 0.0% of fine-tuning TCO. The actual training run on Together AI for our reference workload, 2,000 examples, 600 tokens per example, 3 epochs, costs $1.73. Even on OpenAI’s managed fine-tuning service, where GPT-4o training costs $25 per million tokens, the same job costs about $90. Either way, training is a rounding error against the rest of the bill.
What dominates is the operating model around the model. At 50,000 queries per day, the fine-tune architecture’s three-year TCO breaks down as approximately $1.52M in people, evaluation infrastructure, data curation, and annual retraining; about $30K in inference; about $35K in retrieval infrastructure; and the $2 in training. The line items that are nearly always omitted from fine-tuning proposals, labeled evaluation set construction, evaluation harness setup, retraining cadence, and data versioning discipline, account for 96% of the bill.
Three of those line items deserve specific attention because they are where teams most often underestimate.
The first is the evaluation harness. A serious fine-tuning operation runs a head-to-head test of the fine-tuned model against the RAG baseline on the same labeled query distribution after every retraining. This requires a labeled set with inter-rater agreement above 0.7 kappa, a stable judge prompt, and an automated harness that can rerun the comparison nightly. Teams that have already built one for prompt and retrieval design amortize the cost. Teams that haven’t are paying for it as part of the fine-tuning project, and they tend to underestimate the work by a factor of two or three.
The second is data drift. A fine-tuned model encodes a snapshot. When the underlying business logic changes, a new product taxonomy, a regulatory update, or a pricing schema revision, the model’s behavior drifts from what the data now requires, and the only fix is retraining. Most enterprise data drifts on a quarterly cadence. The retraining cycle is therefore not a one-time cost; it is a recurring obligation that the fine-tune curve in our model carries forward through years two and three.
The third is privacy posture. Research from CISPA, published at USENIX Security 2025, found that fine-tuning Pythia models on real email data increased the success rate of PII extraction attacks by over 50% relative to the pre-trained baseline. Fine-tuning on synthetic data still raised the rate by over 20%. This does not mean every fine-tune leaks; it means the privacy review for fine-tuning training data is a workstream of its own, and the cost of getting it wrong is not on a vendor pricing page.
Once you accept that fine-tuning is fundamentally a data and evaluation operation, the next question is what kinds of problems are worth running that operation for.
To answer the use-case question, we wanted production data, not opinion. We queried the HuggingFace API for the fifty most-downloaded models in each of ten pipeline categories and aggregated downloads by category. The categories span the workload spectrum: classification, named-entity recognition, embeddings, summarization, translation, fill-mask, question answering, structured text-to-text generation, zero-shot classification, and general open-ended text generation.
Across the pipelines we measured, narrow, specialized tasks accounted for 76.5% of cumulative top-50 downloads, 840 million compared with 259 million for general-purpose generation. The single most-deployed model on HuggingFace is sentence-transformers/all-MiniLM-L6-v2, a 22-million-parameter embedding model with 215 million downloads. The most-deployed classifier, cross-encoder/ms-marco-MiniLM-L6-v2, has 30 million.
This is data on what the open-source production layer actually deploys, not on what teams say they want. And what gets deployed concentrates on workloads where prompt engineering against a frontier model has a ceiling that smaller fine-tuned models can clear at one-to-two orders of magnitude lower cost. Bucher and Martini, in arXiv:2406.08660, ran a four-case-study comparison across sentiment analysis, stance classification, emotion detection, and political position classification. Smaller fine-tuned BERT-family models consistently and significantly outperformed zero-shot prompted ChatGPT-3.5, ChatGPT-4, and Claude Opus on F1. The result generalizes the cost-aware finding from earlier classification literature: at production scale, a fine-tuned encoder beats a prompted frontier model on cost, latency, and often accuracy for well-defined classification.
The pattern that emerges from the production layer is consistent with the framework that emerges from the cost model. Fine-tuning earns its keep on tasks where the desired behavior is narrow, stable, and high-volume, where the model needs to learn a specification rather than reason across a general one. It is much rarer at the top of the download distribution to find a fine-tuned model whose job description is “be generally smart about our company.” The teams that pitch that use case usually end up serving it with retrieval against the relevant documents, layered into a frontier model with a structured prompt.
A useful design pattern follows. The most successful production fine-tuning we see in client engagements is not user-facing. They are components of a fine-tuned classifier as the routing layer in a multi-step agent, a fine-tuned reranker on the retrieval layer, and a fine-tuned schema validator on structured outputs. User-facing reasoning still occurs on the frontier model. A fine-tune does not need to replace prompting; it needs to fit into the right place in the architecture.
The cost model and the production data converge on a framework with four signals. We present it as Santiago & Company’s synthesis from this evidence base, not as a derived law. Two of the signals are knockouts present alone; each is sufficient. Two are reinforcing; both together are sufficient. None of them excuses a proposal from the evaluation gate underneath them all.
The volume floor is the cleanest knockout, as the cost model directly proves it. Sustained throughput above approximately 45,000 queries per day on a stable task definition produces unit economics savings that compound beyond the organizational floor. The latency floor is the second knockout. There are workloads where the time-to-first-token of a frontier API plus retrieval, typically 200 to 500 milliseconds per query, per the Q1 2026 inference benchmarks, exceeds the production budget, and where the quality of Haiku 4.5 or GPT-5.4 Nano is not enough to close the gap. A fine-tuned, smaller model running on optimized inference infrastructure, such as Groq or Fireworks, can achieve sub-100ms p95. If that is the latency tier the workload requires, and the team can prove it cannot get there otherwise, the case is made.
The reinforcing signals are subtler. Behavior specificity is the requirement that outputs conform to a stable, narrow specification, a regulated format, a structured schema, a brand-specific tone, and that prompt-based approaches show a violation rate of greater than 5% in schema or format, even after retries. The Bucher and Martini result is the strongest empirical anchor: on tasks like that, fine-tuned smaller models consistently beat zero-shot frontier models on F1. Proprietary reasoning patterns are the requirement that the task involves applying judgment specific to the organization that cannot be expressed as retrievable facts, such as underwriting heuristics, internal taxonomy resolution, and code conventions in proprietary frameworks. The HuggingFace concentration analysis is the weight here: this is the category of work where the production layer has consistently rewarded fine-tuning.
A proposal that exhibits one knockout signal is sufficient. A proposal that exhibits both reinforcing signals is sufficient. A proposal with one reinforcing signal alone, say, “we need a specific tone,” is not, because tone alone is usually addressable with a careful system prompt and a few-shot example set. A proposal that exhibits no signals but argues from a general “we want our own model” intuition is the case; the cost model exists to flag.
The interaction between the volume floor and the reinforcing signals deserves an explicit rule. Below 45,000 queries per day, the unit economics work against fine-tuning, but the framework still permits approval if both reinforcing signals are present and the proposal can articulate why the non-economic case overrides the economic one. In practice, this is the regulated-domain case, financial services compliance writing, medical coding, and legal contract review, where format precision and proprietary judgment dominate calculations that pure economics misses. The proposal must show its work; it cannot lean on the framework to skip the analysis.
The choice of “one knockout or two reinforcing” rather than some other combination is a judgment call from the production patterns we observed, not a theorem we derived. A team adopting this framework should treat it as a starting structure that they refine against their own evaluation data over time.
The evaluation gate is what most failed fine-tuning projects skipped. A labeled evaluation set must exist before the project starts. The baseline performance of RAG plus structured prompting against that set must be measured. The target improvement and its quantified business value, in dollars or operational units, must be specified before any training run is approved. A 2026 Gartner I&O survey, summarized in industry analysis, found that AI projects with quantified upfront success metrics achieved roughly four times the production rate of those without. The quantified gate is not a consulting flourish; it is the mechanism by which a fine-tuning proposal stops being a wish and starts being a falsifiable claim.
A fine-tuning proposal that fails to answer six questions is incomplete and should be returned. The questions follow from the framework but stand on their own as a one-page intake instrument.
The first question is the volume forecast. What is the sustained query rate over the next twelve and twenty-four months, and does it cross the volume floor for the workload profile in question? A proposal that cannot supply a defensible forecast is asking for a fixed-cost commitment against a moving denominator.
The second question is the latency budget. What is the p95 latency target? What does the team currently measure on a frontier-API-plus-RAG baseline, and what is the gap? If there is no gap, the latency knockout does not apply.
The third question is the evaluation set. Does a labeled evaluation set exist? What is the inter-rater agreement? What is the RAG baseline’s F1 or task-specific score on it, and what is the target improvement? A proposal without baseline numbers is a proposal that has not done the prerequisite work.
The fourth question is the dollar value of the improvement. If the target improvement is real, what does it save in operational cost, error rate, or latency tier per period? This is what justifies the organizational spend; the answer “users will be happier” is insufficient.
The fifth question is the data drift cadence and the retraining plan. How often does the underlying data change in ways that affect the task, and what retraining frequency maintains quality? Annual is common; quarterly is realistic for some domains; monthly retraining usually means the fine-tuning itself is the wrong solution, and the team should reach for retrieval.
The sixth question is the privacy review of the training data. Has the training corpus been reviewed for PII? Has the retention and deletion policy been specified? Has the team accounted for the elevated extraction risk posed by the CISPA work documents? If the answer is “we’ll handle it later,” it gets handled in litigation.
A proposal that answers all six well will sometimes still recommend fine-tuning. That is the framework working as intended. The point is not to drive teams away from fine-tuning; it is to separate proposals that have done the analytical work from proposals that have not.
If only one question is asked of the team that brings the next proposal across the desk, it is the third one. Ask what the RAG baseline F1 is on the held-out evaluation set. If that number does not exist, the answer to the proposal is not yet.
Aerospace and defense companies will convert record demand into revenue only by managing qualified throughput. The scarce asset is qualified throughput, and it has to be managed node by node. The approved combination of site, process, workforce, and part at the individual production-node level.
Santiago & Company announced the public release of the AI Workforce Stability Architecture (AWSA) at the United Nations WSIS Forum 2026, introducing an open framework that helps governments monitor, prepare for, and respond to the economic and fiscal impacts of AI-driven labor displacement in service-export economies.
AI is moving on a 15-month clock; governments are moving on a five-year one. For service-export economies, the winners will not be those that forecast labor disruption most precisely, but those that pre-wire fiscal and workforce responses before the evidence becomes undeniable
SpaceX does not need to disrupt the wireless industry to reshape it. By turning satellite coverage into negotiating leverage, it is forcing carriers to rethink who owns the customer relationship when coverage comes from space.
