Insights · AI & automation / Engineering
An AI workflow needs a baseline, not just a demo
Define the task, evaluate real examples, and keep consequential actions under explicit control before connecting AI to business systems.
A convincing demonstration shows that a system can succeed. For software people rely on, the useful questions are: how often does it succeed, under which conditions, and what happens when it fails?
That distinction matters when a model is connected to customer records, internal documents, or tools that can change business data.
Start with a specific task
“An assistant for the business” is too broad to evaluate. “Find the policy passage relevant to this support question and draft a response for review” is specific enough to test.
Describe the input, the permitted sources, the desired output, and who decides whether the result is good enough. If a simple rule or a conventional search interface solves the problem, use that instead.
Build a small evaluation set
Collect representative examples, including ambiguous requests, missing information, and cases the system should decline. Use data you have permission to process, with an appropriate privacy review before sending it to any external model provider.
For retrieval, check whether the right source was found. For generated answers, check whether the answer is supported by that source. A fluent answer and a correct retrieval are different things.
Record a baseline from the current workflow. Compare quality, time, latency, and cost on the same examples when you change the implementation.
Separate proposing from doing
Generating a draft and sending it are different capabilities. So are suggesting a record update and changing the actual record.
Give the workflow only the permissions it needs. Treat retrieved text and model output as untrusted input. Validate tool arguments independently and require human approval for consequential actions.
Make the approval specific: the reviewer should see the proposed action and the data it changes, not a vague “allow automation” switch.
Plan for the unhelpful answer
A useful system can say it does not know. It should show sources, expose uncertainty, and offer a clear path back to a person.
Keep enough operational evidence to diagnose failures without collecting unnecessary sensitive data. Decide in advance when an error should stop the workflow and when retrying is safe.
The goal is not the most autonomous system. It is a workflow that makes a real task easier without making its failures harder to see.