Your chatbot remembers the old refund policy

Uploading the new policy does not guarantee the next answer will use it. Reliable customer answers need a rule for which version applies to this customer, plus a way to prove that the assistant selected it.

By DLVX · 5 min read · Published 20 October 2026

Read the worked guideCheck the sources

Your chatbot remembers the old refund policy

  1. Identify booking

    Find the event and terms relevant to this customer.

  2. Select version

    Use applicability, not the most recent upload.

  3. Draft with evidence

    Keep a source version attached to the answer.

  4. Recheck before send

    Refresh facts whose validity changed while the draft waited.

A teaching sequence. This diagram does not run an AI tool or connect to your business.
On this page

Newer is not always the applicable answer

A fictional Phuket tour operator changes its cancellation terms on 1 November. A guest booked on 20 October and asks about cancelling on 4 November. Which policy applies? The answer depends on the operator's actual terms and any applicable requirements. The assistant cannot settle that by selecting the file with the newest upload date.

Write the business rule explicitly: which event selects the applicable policy, which products it covers and who resolves exceptions. This article describes information handling, not legal advice about what cancellation terms a business may enforce. A policy owner must establish the real rule before the assistant can follow it.

A customer-facing answer needs the booking context as well as a policy document. When the booking reference is missing, the correct output may ask for it. Inventing an entitlement from a general policy is still an error when the paragraph includes a neat citation.

Give each policy a usable identity

Keep an immutable version ID, an owner, approval status, effective start and end, covered products and the rule used to select it. Record the document's language and whether a translation is authoritative or explanatory. An English translation updated yesterday can describe older Thai terms; upload time is not policy authority.

Separate effective time from system time. Effective time says when a rule applies. System time says when you received or approved it. A correction entered on Friday might clarify terms that applied from Monday. Keeping both dates allows an investigator to explain what the assistant knew on Wednesday.

Store dates in an unambiguous format and retain the customer's original wording. For Thai-facing operations, explicitly label Buddhist Era dates when they appear and convert them carefully for comparison. Do not silently interpret a short date string or assume that a customer in another country means Bangkok time.

The same discipline applies to delivery coverage, membership benefits and service hours. Price and stock facts may need a live system check instead of a document version. Give each fact an owner and a freshness rule suited to how often it changes.

A source can be real and still be the wrong source

Fictional booking facts
Booking ID: T-312
Booked: 2026-10-20, Asia/Bangkok
Departure: 2026-11-12
Product: Small-group day trip
Selection rule: Terms accepted at booking govern this scenario
Accepted terms ID: CANCEL-2026-09

Wrong draft:
Our latest policy allows cancellation until 48 hours before departure.

Better internal result:
Use CANCEL-2026-09 attached to booking T-312.
Do not substitute CANCEL-2026-11 solely because it is newer.
If accepted terms cannot be verified, route to the policy owner.

The better result does not invent the old policy's actual allowance. It identifies the evidence needed to answer. Once the relevant source supplies the entitlement, the customer draft can explain it in ordinary language, with a link or reference appropriate for that customer.

Keep the internal source record more precise than the customer reply. Staff may need the version ID and the selection rule; the customer usually needs their applicable deadline, the reason and the next action.

Filter before asking the model to interpret

Search should consider applicability before ranking similar paragraphs. Filter by product, policy status and relevant dates where your system supports it. OpenAI's retrieval documentation describes attribute filtering on stored files; this is one implementation option, not a complete policy engine. Retrieval documentation, checked 20 October 2026.

If your platform cannot apply the necessary filters, resolve the version in ordinary application logic and pass only the applicable source to the answer step. Asking a model to pick correctly from a pile of contradictory policies makes every answer repeat a decision the business could make once.

Check the retrieved passage itself. Does it actually state the relevant rule, including exceptions? A search hit for cancellation may omit the paragraph that excludes a particular package. Save the source excerpt or a stable reference with enough context to reproduce the answer.

When sources conflict, the workflow should return the conflict to a named owner. Do not instruct it to blend the policies into a compromise. A polite synthesis can create a rule that nobody approved.

Treat a policy update as a release

When the owner approves a new version, confirm that the searchable index contains it, the old version remains available only where still applicable and the selecting rule works. A file upload confirmation proves storage, not correct retrieval.

Test a booking before the change, one after it and a case with missing acceptance evidence. Include both languages if both are in use. Check cached answers and saved drafts: an assistant may keep repeating the old rule from conversation history even after the search index updates.

Record the selected policy version in each draft. Before sending, recheck when a relevant fact changed or the draft outlived its agreed validity window. The right window depends on the fact. A room's availability can change while someone reviews a message; a historical booking term should not change simply because time passed.

Avoid one universal fresh-for-24-hours rule. It is convenient to implement and often wrong for the business.

The policy owner can complete this without coding

Policy answer contract
Policy owner:
Version ID:
Approved on:
Effective interval:
Products and customer groups covered:
Which event selects this version?
Authoritative language:
Known exception route:
Source location:
Draft validity rule:

Before sending:
Booking identity verified?
Applicable version selected?
Relevant exception checked?
Current operational facts rechecked?
Conflicting source or missing evidence routed to owner?

Try this with the policy customers ask about most often. Ask a colleague to locate the correct version for two old cases without using the assistant. If that is difficult, improve the record before adding more AI instructions.

The result you want is explainable selection: this answer used this version because these facts made it applicable. That makes updates safer and customer disputes easier to investigate, even when a person writes the final reply.

Sources and dates

Sources checked on 20 October 2026. Worked examples are teaching examples unless explicitly attributed. Provider documentation describes product behaviour; the operating recommendations are DLVX editorial judgement.

  • OpenAI retrievalFile attribute filtering exists; policy selection and refresh procedures are editorial recommendations. Checked 2026-10-20.

Have a workflow in mind? Talk through the job with DLVX.