“Should we use RAG or fine-tune?” often combines two product problems. One concerns the information the assistant can access. The other concerns how consistently it follows a task. Separate those problems before committing to a training pipeline or a retrieval service.
Identify what changes between requests
A coach’s current programme, a customer’s subscription state or a recently edited help article can change independently of the model. Those facts need a retrieval or tool path that can access the permitted current record. Re-training a model whenever a customer changes a programme would not be a sensible source-of-truth strategy.
Use retrieval for source-backed answers
RAG adds relevant source material to the model’s context. It can help an assistant cite the material behind an answer, but retrieval quality still needs evaluation. Check that the system selects the right version, account and date range. A response can be grounded in a document and still be wrong for the user’s question.
Consider tuning for a measured behaviour gap
Fine-tuning adapts model parameters using training examples. It may be worth evaluating when a repeatable task or output convention remains unreliable after simpler changes. Build an evaluation set first, keep a held-out set and compare against the untuned baseline. An improvement on training examples alone does not establish better product behaviour.
Keep permissions in application code
Neither technique makes an unauthorised record safe to disclose. Filter retrieval and tool access using the signed-in account, and validate write operations on the server. Do not use a prompt as the only barrier between two customers’ data. A model’s confident answer is not evidence that the underlying record belongs to the requester.
Compare complete user tasks
- Can the answer identify a current permitted source?
- Does the assistant respond appropriately when the source is missing?
- Are stale or contradictory documents handled visibly?
- Does a tool action produce the intended application state?
- What are the latency and total cost per completed task?
In healthtech, keep the product’s scope explicit when evaluating answers. A scheduling or programme-navigation assistant is a different feature from clinical decision support. This article discusses architecture choices, not medical advice or a claim that a particular model is safe for a clinical use.
UltraFit360’s documented tool-based assistant is a relevant example of accessing context and actions through a product boundary. It is not evidence that UltraFit360 uses a specific fine-tuning or RAG deployment. The right choice for another product depends on its own data, tasks and evaluation results.
Primary reference: Google Cloud’s guide to choosing grounding and tuning
When should a product team choose RAG instead of fine-tuning?
Choose retrieval for source-backed information, evaluate tuning for repeatable behaviour, and keep account permissions outside the model in healthtech and SaaS products.