LLM Fine-tuning for Business – Custom LLM & Dedicated AI Model Development

Will fine-tuning fix your problem? The diagnosis I run before quoting

Fine-tuning changes how a model behaves: format, style, how it classifies, which terms it uses. It is a poor way to give a model knowledge about your company. So the first question is: does the model not know, or does it know and get it wrong?

A study from Technion and Google Research, „Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?” (Gekhman et al., EMNLP 2024), shows why. The authors tested what happens when training data contains facts the model did not already know. The model learned those examples markedly slower than ones consistent with its existing knowledge, and once it did learn them, its tendency to hallucinate went up. Their conclusion: models acquire knowledge mostly in pre-training, and fine-tuning teaches them to use it better. Price lists, product details and procedures belong in a RAG implementation, not in training.

Symptoms and what I test first

OpenAI's model optimization documentation (2026) lists the same reasons fine-tuning makes sense: more examples than fit in one request, shorter prompts at scale, and a smaller, cheaper model for a specific task. I cover getting structured answers without training in Structured outputs in production, and working with examples in prompts in Prompt engineering for business.

Fine-tuning in 2026 – what changed at the API providers

In 2026, fine-tuning through a major provider's API became a less reliable path. OpenAI is winding down its fine-tuning platform, so a model trained on a provider's platform should be treated as a dependency with an expiry date, and open weights as the option that stays under your control.

According to OpenAI's deprecations page (notice dated 7 May 2026), the wind-down runs in phases: from 7 May 2026 organisations that have never fine-tuned can't create training jobs; from 2 July 2026 organisations that haven't run inference on a fine-tuned model in the past 60 days lose access; and from 6 January 2027 active customers can't start new jobs either. Existing fine-tuned models keep working, but only until their base model is deprecated. OpenAI's model optimization documentation confirms that new users no longer have access to the platform.

Other providers still offer training, within limits. Google offers supervised tuning for selected Gemini models in Vertex AI, and in 2024 Anthropic made fine-tuning of Claude 3 Haiku available through Amazon Bedrock. The list of tunable models moves with each model's lifecycle, so I check the current documentation before every project.

For clients this boils down to one practical rule: the training set and the evaluation set are worth more than the trained model itself. I keep them in an open, provider-neutral format (JSONL) so that when the base model changes, retraining and comparing results takes days rather than a rebuild. How I compare providers and open weights is in How to choose an LLM in 2026.

Choosing a base model for fine-tuning – licence, size and language

I pick a base model on three criteria, in this order: a licence that allows your use, quality on your own data before training, and the smallest size that passes the test. Public leaderboards only give me a shortlist.

„Open” licences differ more than the word suggests

For Polish-language work, Bielik 11B v3 is described in a December 2025 technical report (arXiv): the authors built it on the Mistral 7B v0.2 architecture by depth up-scaling and claim it outperforms other Polish-specialised models and many models 2 to 6 times its size. That is the authors' claim, so I still test it against a multilingual model on the client's data.

How I compare candidates before training

1. I shortlist 2–3 models whose licence fits the planned use and whose size fits the target server.
2. I run the same test set with the best prompt on each. In my experience, the model that does best untrained usually stays ahead after training.
3. I check tokenisation on the client's texts: how many tokens a typical document takes. More tokens means longer training, slower answers and less content in the context window. This matters most for languages other than English.
4. Only then do I choose the size. I start with the smallest model that has a chance of passing, because every step up raises the server bill for as long as the model runs.

Running open models on your own hardware, including the difference between Ollama and vLLM, is covered in Ollama, vLLM and self-hosted LLMs.

SFT, DPO, GRPO or distillation – which training method I use

For most business projects I start with supervised fine-tuning (SFT) and a LoRA adapter. I add other methods when the data has a different shape: pairs of better and worse answers, an automatically checkable result, or a larger model's outputs to copy.

DPO was introduced by Rafailov et al. (NeurIPS 2023) as a method that replaces classic RLHF with a single training stage on preference pairs, with no separate reward model. Hugging Face's TRL library, which I use, has dedicated trainers for SFT, DPO, GRPO, reward modelling and distillation. OpenAI's documentation limited reinforcement fine-tuning to reasoning models and noted that expert graders must agree on the ideal output. That condition applies to any RL method: if two of your specialists would score the same answer differently, the model has nothing consistent to learn.

LoRA or full fine-tuning

LoRA trains small matrices attached to a frozen model, and QLoRA also stores the base model in 4 bits. The QLoRA authors (Dettmers et al., NeurIPS 2023) showed this makes it possible to fine-tune a 65B model on a single 48 GB GPU while matching full 16-bit fine-tuning quality in their tests. „LoRA Learns Less and Forgets Less” (Biderman et al., TMLR 2024) adds an important nuance: on code and maths, LoRA learned new skills noticeably less well than full fine-tuning but preserved the model's general abilities better. For business tasks that are about format and style rather than a new domain, that's a good trade.

Auditing training data – what I check before anything goes into training

A fine-tuned model reproduces patterns from its data, including errors, contradictions and personal data. Before I build the training file, I go through a checklist covering where the data comes from, its quality and its risks.

Checklist

• Origin and legal basis: where the examples come from (helpdesk, email, documents) and whether the company may use them for training. With personal data you need a GDPR legal basis and often a DPIA.
• Personal data in the text: I remove or replace names, addresses, phone numbers, national ID numbers and bank account numbers before any text goes into training.
• Label consistency: on a sample, I check whether two people would label an example the same way. If not, we agree the rules first and train second.
• Freshness: old prices, discontinued products and superseded procedures come out, or the model will learn them as the norm.
• Duplicates and leakage: I remove near-duplicates and make sure similar cases don't land in both the training and test sets. I split the test set by time or by customer, not at random.
• Edge-case coverage: rare but important situations (a complaint threatening legal action, an out-of-scope question) need their own examples, including examples of refusing.
• Synthetic data: when I top up the set with examples generated by another model, I check the provider's terms. OpenAI's Terms of Use, for instance, prohibit using output to develop models that compete with OpenAI; whether that affects a given project is a question for the client's lawyer.
• Format matching the model: examples go into the chat template the chosen base model uses; a different template at training and inference time is a common cause of poor results.

Why personal data is a bigger problem in training than in RAG

Models can memorise parts of their training data. Carlini et al. (USENIX Security 2021) extracted hundreds of verbatim training sequences from GPT-2, including names, phone numbers and email addresses, some of which appeared in only a single document in the training data. In RAG you delete a specific chunk from the index; you can't simply pull data out of model weights, you have to retrain on a cleaned set. So I anonymise before training, not after. Practical rules for personal data in AI projects are in GDPR and AI in practice, and getting data in order before any rollout in Data readiness.

Evaluating a fine-tuned model – three models, one test set

Whether training worked is decided by a comparison on a test set frozen before training. I always compare three variants: the base model with the best prompt, the fine-tuned model, and a large cloud model with the same prompt as the reference point for quality and cost.

A falling loss curve doesn't tell you the model is better at your task. It only says the model reproduces the training set more closely, which with too much training means overfitting. So I make decisions on task metrics, not training curves.

Metrics by task type

I agree acceptance thresholds with the client before training, for example: the fine-tuned model must at least match the large cloud model on the primary metric and do no worse on the regression set. The client gets a report with results for all three variants, including when the untrained option wins. I describe the LLM-judge method in LLM evaluation with LLM-as-a-judge, and wiring tests into the release process in How to test AI applications.

What fine-tuning costs and when your own model pays off

My fine-tuning projects start from €3,000, but whether it pays off depends on the cost of running the model, not the one-off training. Training a LoRA adapter is usually the smallest line item; data preparation, evaluation and a server that runs around the clock cost the most.

What goes into the project price

• Data preparation: export, anonymisation, cleaning and labelling. The more scattered and inconsistent the data, the more work.
• Evaluation: building the test set, calibrating the scoring and comparing the three variants.
• Training and iterations: the number of rounds depends on how quickly the model clears the thresholds.
• Deployment: inference server, API, authentication, monitoring and documentation for your team.

How I work out the break-even point

1. Current monthly API cost: number of requests times average input and output tokens times the model's price. First I check how much caching, batch mode and a cheaper model can cut it, because the fair comparison is with the best version of the API setup, not the current one.
2. Cost of your own model: GPU for every hour the model must be available (for customer-facing work, usually 24/7), plus time for maintenance, monitoring and retraining.
3. Compare both at today's volume and at next year's. A GPU server costs the same busy or idle, so at low traffic the API almost always wins.
4. Add what doesn't show up on the invoice: a requirement that data stays in-house, and independence from provider model retirements.

One more thing changes the economics: several LoRA adapters can share one base model. The vLLM documentation describes selecting the adapter per request, so one server can run ticket classification, invoice extraction and reply drafting, each with its own adapter. The API cost levers I always compare against are in AI API cost optimization.

Deploying and maintaining a fine-tuned model

I expose the trained model through an OpenAI-compatible API, so the application can switch to it by changing the endpoint and model name. After launch the model needs monitoring and a retraining plan, because inputs, the company's offer and the available base models all change.

What the deployment includes

• Inference server: vLLM or TGI; the LoRA adapter loaded on top of the base model or merged into it, depending on how many adapters share a server.
• Quantisation for inference: lower weight precision lets the model fit a cheaper GPU; I check its effect on quality with the same test set.
• Versioning: each model version is tied to its training set, configuration and evaluation result, so rolling back takes minutes.
• Gradual rollout: a new version gets a share of traffic first, and full traffic only after its results are compared with the previous one.
• Security: authentication, rate limits and logs, without storing personal data where it isn't needed.

When I retrain

• The output format, category list or company voice changes.
• A weekly sample of production answers, scored with the same rubric as the evaluation, shows quality slipping.
• A clearly better base model appears. A LoRA adapter only works with the model it was trained on, so a new base means retraining on the same dataset and a fresh comparison.

Metrics, tracing and alerts for LLM applications are covered in Monitoring AI in production. When the model is part of a larger system with a backend, queues and integrations, that falls under AI systems programming.

Fine-tuning, the EU AI Act and licences – when you become a model provider

A typical LoRA fine-tune does not make a company a provider of a general-purpose AI model under the EU AI Act. Obligations still follow from what the model is used for and from the base model's licence.

The European Commission's guidelines for providers of general-purpose AI models (18 July 2025) set a simple threshold: a company that modifies such a model becomes its provider if the compute used for the modification exceeds one third of the compute used to train the original. Training an adapter on a few thousand examples is nowhere near that. It doesn't remove obligations tied to the use case: a model that screens job candidates or assesses creditworthiness falls under the high-risk requirements whether or not it was fine-tuned.

What I hand over at the end of a project

• The adapter weights or merged model (for open models).
• The training set, evaluation set and training scripts, so training can be repeated without me.
• The evaluation report with results for every variant compared.
• A note on the base model's licence obligations, e.g. the „Built with Llama” notice and naming rule for Llama models.

Company obligations under the AI Act more broadly are covered in The EU AI Act in practice.

What to prepare before a fine-tuning project – a checklist

Projects move fastest when the company can show the task through examples and name someone to judge the results before we start. Without that, the first weeks go on agreeing what a good answer looks like.

1. A task description and 20–30 reference answers written or approved by a specialist.
2. Access to historical data (helpdesk, CRM or document archive export) and a note on which of it is current.
3. Your current prompt and examples where the model fails, if you already use AI.
4. A volume estimate: requests per day now and in a year, plus the required response time.
5. Data and hosting requirements: can data go to the cloud, or must the model run on your own servers?
6. Someone on your side with time to review test results across iterations.

If you're not yet sure your data and processes are ready for this, an AI readiness audit is a good first step. When the model should not only answer but also act in your systems, that's the scope of AI agents for business. The wider picture, including choosing processes and costs, is on the AI & automation page, and the technical details of QLoRA training with code are in Fine-tuning an LLM on company data.