Ask what went wrong in the last bad answer
A fictional Thai furniture importer wants an assistant to answer product questions. It gets a chair's finish wrong, overlooks a delivery exception and writes in a tone the team dislikes. Those look like one AI quality problem, but they may need different repairs.
If the correct finish never reached the model, improve the information supply. If the exception was present but the assistant missed it, inspect document structure and the test case. If the facts are right but the output repeatedly ignores a stable format, improve examples and instructions before considering training.
Start with the smallest method that passes your real cases. The answer can be a combination: retrieve current specifications, include the relevant customer context and use a model trained for a consistent output style. It can also be ordinary application logic when the question is an exact stock lookup.
What each option actually changes
| Option | What it supplies | Useful starting situation | Failure to test |
|---|---|---|---|
| Long context | A large supplied brief in the current request | A small, bounded collection needed together | Relevant detail gets lost among distractions |
| Retrieval | Selected source passages found for the question | A larger or changing knowledge collection | Search misses the right passage or its exception |
| Fine-tuning | Changed model behaviour learned from examples | A stable, repeated behaviour still fails after simpler fixes | Training examples teach the wrong pattern |
OpenAI's model optimisation guide distinguishes evaluation, prompting and fine-tuning as parts of improving model behaviour. It does not establish that fine-tuning is the next step for every business. Model optimisation, checked 20 October 2026.
The table is a decision aid, not a benchmark. Actual accuracy, latency and cost depend on your documents, questions, model, configuration and review requirements. Test those variables in your own account.
A bigger brief can be the simplest useful system
If staff answer questions from a short approved handbook and a single product sheet, supplying both may be easier to operate than building search. You can see exactly what information the model received. There are fewer moving parts to maintain, and a policy update can replace a clear input bundle.
The cost appears when every request carries information it does not need, or when conflicting versions accumulate. A large advertised capacity does not tell you whether the model reliably notices a small exception buried in the middle. Test questions that depend on details near different parts of the supplied material.
Remove duplicated headers, irrelevant navigation and superseded sections before expanding capacity. Better source preparation often makes the task easier without changing the model. Keep document boundaries and titles, because an isolated paragraph can lose the product or policy it belongs to.
Use a fixed source bundle for a fair comparison. If one approach receives clean documents and another receives a messy export, the result tells you about preparation as much as model capability.
Retrieval saves context by making a new decision
Retrieval first decides which passages to give the model. That can keep a large collection manageable, but the answer cannot use an exception that search failed to return. Inspect retrieval separately from writing quality.
OpenAI documents semantic search and attribute filters in its retrieval system. Retrieval guide. For a catalogue, filters can narrow the search to an approved product family or document status. Exact identifiers may need keyword or database lookup rather than relying solely on semantic similarity.
In the fictional importer, CH-204 and CH-240 could be different chairs. A search result that sounds right may be commercially wrong. Test model codes, Thai product names, English aliases and abbreviated customer wording. Preserve the exact identifier through the workflow.
Measure two things: whether search returned the necessary evidence and whether the assistant used it correctly. If evidence was absent, fix retrieval or the source. If evidence was present and the answer still contradicted it, investigate the answer step. This separation prevents endless prompt editing around a search failure.
Fine-tuning should have a stable target
Availability checked 20 October 2026: OpenAI is winding down its self-serve fine-tuning platform. New organisations have been unable to create training jobs since 7 May 2026. Since 2 July, organisations that have not run inference on a fine-tuned model in the past 60 days cannot create new jobs either. Eligible active customers lose new-job creation on 6 January 2027; inference continues only until the underlying base model retires. Check the dated OpenAI availability notice before planning work. The comparison here describes a general technique, not an available OpenAI purchase path for a new customer.
Consider fine-tuning when you can describe a repeated behaviour, provide high-quality examples and show that simpler instruction changes have not met the requirement. Consistent classification or a specialised output format can be candidates. A constantly changing price list is a poor reason to start.
Training examples need review. If half the team marks a delivery complaint as logistics and half marks it as customer service, a model cannot invent the business's preferred boundary. Agree on labels, remove contradictory examples and keep a separate evaluation set.
Include training preparation, reruns and ongoing evaluation in the cost comparison. A cheaper per-request model can still be more expensive to operate if its training workflow needs frequent repair. Do not assume a tuned model can reliably recall every source fact or provide a valid citation from memory.
Keep changing facts outside the trained behaviour where practical. Training can shape how the assistant handles a product question while current stock and specifications still come from approved sources.
Run a comparison you can explain
Information strategy trial Business question set: same cases for every approach Source snapshot: same approved versions Baseline: simple prompt plus relevant documents Alternative: retrieval with visible selected passages Fine-tuning candidate: only if a stable behaviour gap remains For each case record: Necessary evidence supplied? Answer supported and exceptions respected? Correct product and policy version? Human correction minutes? Measured latency and actual usage cost? Failure cause and operational owner?
Choose cases that could change the decision: a normal product question, an exact code, a cross-document comparison, an exception and an unanswerable request. A small test is a starting diagnosis, not proof across the entire catalogue. Expand it around the failures you discover.
Prefer the method your team can keep accurate. If the simple brief passes and updates cleanly, keep it. If retrieval fixes a real information problem, add it. Fine-tuning earns its place when a specific remaining behaviour improves on held-out work enough to justify its maintenance.
Sources and dates
Sources checked on 20 October 2026. Worked examples are teaching examples unless explicitly attributed. Provider documentation describes product behaviour; the operating recommendations are DLVX editorial judgement.
- OpenAI model optimisationEvaluation, prompting and fine-tuning are distinct improvement methods. Checked 2026-10-20.
- OpenAI retrievalSemantic search and attribute filters; comparisons and recommendations are editorial, not performance claims. Checked 2026-10-20.
- OpenAI self-serve fine-tuning availability noticeEligibility restrictions effective May and July 2026, new-job cutoff 6 January 2027, and inference tied to base-model retirement. Checked 2026-10-20.