AI prototypes are usually built around the happy path: collect an input, send it to a model and render the response. Consumer mobile software lives everywhere else.
It runs on unstable networks and older devices. Users interrupt requests, submit unexpected inputs, change languages and return after an app version has changed. Providers time out. Structured output becomes almost structured. Safety policies reject a request that worked yesterday. A release that improves average quality can quietly increase latency or cost.
The engineering task is therefore not to make a model appear reliable. It is to build a product boundary that contains uncertainty. The following is the architecture I use to reason about that boundary.
1. Define a product contract before choosing the model
A feature needs a contract that the product can enforce. It should state the accepted inputs, the useful output, the maximum wait a user should tolerate, the data that may be retained, the failure behaviours and the actions the interface will allow next.
This contract must be expressed in product terms. “Return valid JSON” is an implementation requirement. “Give the user three editable suggestions without losing their original input” is a product contract.
The mobile experience owns the promise. The model is one replaceable component used to fulfil it.
Once the contract is explicit, model selection becomes an engineering decision. A smaller model may be sufficient for classification. A slower model may be justified for a high-value creative task. A deterministic local rule may be safer than generation. The system can route by task instead of forcing one provider to solve every problem.
2. Model every state the user can encounter
Loading and error are not enough. A robust AI flow normally needs distinct states for validation, queued work, generation, partial progress, completion, recoverable failure, policy refusal, connectivity loss and cancellation.
Those distinctions matter because the next useful action differs:
- An invalid input needs precise guidance before any request is sent.
- A timeout should preserve the input and offer a safe retry.
- A policy refusal needs an explanation and a route to an acceptable request.
- A parsing failure may be recoverable server-side without asking the user to start again.
- A cancelled request must not later overwrite a newer result.
On the client, I prefer one explicit state machine over several loosely related booleans. It prevents impossible combinations such as showing a completed result while a stale request still controls the loading state. Each transition can be tested, observed and connected to one interface behaviour.
3. Keep provider control behind a server boundary
A mobile client should not be responsible for provider credentials, prompt assembly, routing rules or unrestricted model parameters. Those concerns belong behind a controlled service boundary.
The boundary can validate authentication and input size, attach the correct prompt version, select a model, enforce timeouts, apply rate limits, validate output and return a stable application response. This keeps provider changes from becoming mandatory client releases and reduces the number of behaviours an old app version must understand.
Every request should carry an idempotency key when repeating it could duplicate a paid or visible action. Every response should use a versioned schema. Unknown fields should be tolerated where possible; missing required meaning should fail safely. The client should never assume that generated text is structurally valid merely because the provider returned success.
4. Build observability without collecting the conversation
Reliable operation requires enough information to reconstruct what happened. It does not require storing every private input and output.
Useful operational fields include a request identifier, app version, feature version, prompt version, model route, locale, duration, retry count, validation result, fallback path and a privacy-safe outcome category. Together, these allow a team to answer whether a regression is tied to one release, language, provider or configuration.
Raw content should be minimised, redacted or avoided according to the feature’s needs. Debug convenience is not a sufficient reason to create a permanent dataset of sensitive user material. If examples are retained for evaluation, consent, access, retention and deletion rules need to be designed as part of the system.
Client and server events should share the same request identifier. Otherwise a mobile timeout and a server completion look like unrelated facts, making it difficult to find duplicate work, stale results and provider latency hidden behind a generic error screen.
5. Evaluate the product system, not only the answer
Model evaluation often focuses on output quality. Production evaluation needs at least four layers:
| Layer | What to test | Failure it catches |
|---|---|---|
| Contract | Schema, boundaries, refusals and unsafe inputs | Invalid responses reaching the interface |
| Behaviour | Known examples across languages and use cases | Quality regressions hidden by averages |
| Journey | Timeout, retry, cancellation and restoration | Broken states around a valid answer |
| Operation | Latency, fallback, error rate and resource use | A quality gain that makes the feature impractical |
A compact evaluation set should include normal examples, edge cases and previously observed failures. It should run when prompts, models, tools, schemas or safety logic change. Exact string matching is rarely sufficient, but neither is a vague model-based score. The best evaluation combines deterministic assertions for the contract with structured review for semantic quality.
The mobile flow also needs behavioural tests. Given a request that times out, when the user retries, then the original input should remain and only the latest result should render. Given an old cached response, when the schema is no longer supported, then the app should discard it safely rather than crash. These are product guarantees, not model benchmarks.
6. Treat prompts and models as production releases
A prompt change can alter user-visible behaviour as much as an application release. It needs an owner, a version, an evaluation result, a rollout plan and a rollback path.
I prefer progressive exposure: internal testing, a small production cohort, monitored expansion and automatic or manual rollback thresholds. The exact percentages matter less than the ability to compare the new configuration against a stable baseline.
Rollout metrics should include more than provider success. Watch end-to-end completion, latency distribution, retries, user corrections, saves, reports and return behaviour. A response can be technically successful while creating more work for the user.
If a remote configuration can change the experience instantly, it deserves the same discipline as code that ships through a store.
A practical production gate
Before exposing an AI feature broadly, I want the team to answer:
- Is the user-facing contract explicit and testable?
- Does the interface represent timeout, refusal, retry and cancellation correctly?
- Can the provider or model change without breaking old clients?
- Are outputs validated before they reach product state?
- Can a request be traced across client and server without storing unnecessary content?
- Does the evaluation set include real edge cases and previous failures?
- Can the configuration be rolled back independently of an app release?
- Do we measure useful completion rather than generation alone?
Reliability is the feature users remember
Users do not separate the model from the rest of the product. They experience one system. When a request disappears, a retry charges twice or a confident answer breaks the next screen, the distinction between provider and application architecture is irrelevant to them.
The durable engineering advantage is a controlled boundary around probabilistic capability: explicit contracts, complete states, stable interfaces, privacy-aware observability, layered evaluation and reversible rollout. That system turns an impressive AI response into a mobile feature people can trust repeatedly.