Choosing the Right AI System Starts with Knowing Your Priorities

Summary

This blog explores the six key priorities organizations should evaluate before implementing an AI system, including accuracy, explainability, control, data needs, time to value, and creativity. It explains how understanding these priorities helps businesses choose the right AI approach for each use case and achieve better outcomes.

Imagine your customer success team wants to build an AI system. They manage three time-consuming, repetitive tasks: confirming whether a refund request meets a warranty, understanding when a ticket needs to be escalated to a supervisor, and drafting personalized responses to common customer inquiries. 

Those may all sound like “customer support AI use cases,” but they all need to be handled in a unique way. Refund requests need to strictly follow policies, ticket escalation relies on historical support data, and customer engagement needs empathy and context. 

When designing and implementing an AI system, organizations need to consider not only the needs of the use case itself but also what the organization prioritizes overall. Understanding potential trade-offs before implementation helps organizations know where to use strong rules and where to allow system creativity. In this blog, we’ll explore six priorities that are key to any AI implementation: accuracy, explainability, control, data need, time to value, and creativity in outputs. 

Priority #1: Accuracy

Accuracy: how reliably the system produces correct outputs.

By now, everyone knows that generative AI systems aren’t always accurate. Whether they flat-out lie in a hallucination, conflate different concepts, or under- or overexaggerate nuances, there is always a chance that a generative AI system can say something wrong or misleading. When implementing your own AI system, it’s important you consider what your tolerance is for inaccuracies. Some use cases, such as an internal support chatbot, might have a window of 80-90% accuracy, while other use cases like legal or healthcare-related usage, might have near-zero. 

To understand how to prioritize accuracy in your use case, you might ask yourself questions like:

• What are the potential bad outcomes if the provided information is false?
• How much data do I have that can improve accuracy?
• Who relies on the outputs that the system produces?

Priority 2: Explainability

Explainability: how easily humans can understand why the system produced its result.

If your AI use case requires you to make decisions, flag issues, route work, or provide recommendations, you might need a high level of explainability. While the answer itself is useful to end users, if your organization answers to governance, legal, or regulatory committees, you likely need to know how the system got to the answer to begin with. If an AI system rejects an invoice, recommends a customer for outreach, flags a compliance concern, or escalates a support ticket, the people using that system need to know whether the output came from a defined rule, a historical pattern, or a generated interpretation.

To prioritize explainability in your use case, you might ask yourself questions like:

• Who needs to understand why the system produced its output?
• What decisions or actions will be taken based on the system’s recommendation?
• Do we need to trace the output back to a rule, source, data point, or policy?

Priority #3: Control

Control: the degree of oversight you have over the system’s behavior, access, and actions.

Where accuracy looks at whether the system produces the right answer and explainability asks how the system got to the answer to begin with, control asks how much freedom the system has in the first place. For example, if you’re building a customer support system, can it only send canned responses or can it craft something unique? Can it update a field in your ticketing platform or access customer records? Does a human need to review a response before the system sends it? When building your AI system, you need to decide not only the boundaries in which it takes action but also the freedom it has in its outputs. 

To decide how much control you need for your use case, you might ask yourself:

• What should the system be allowed to access, generate, change, or send?
• What actions require human review or approval before they are completed?
• Where does voice, tone, and approved brand messaging matter?

Priority #4: Data Need

Data Need: the quality and reliability of data needed for the system to perform the task you set for it.

Accuracy, explainability, and control each describe a different facet of what the experience is like for a human. Data need is our first factor that deviates to focus more on a system-centered need. Before you implement your AI system, you need to consider what it needs to know, where that information lives, how clean and complete it is, and whether the system can securely access it. A use case that only requires the system to follow a few fixed rules has a very different data need than one that requires the system to predict future behavior or generate answers grounded in your organization’s internal knowledge.

To understand how to prioritize data needs in your use case, start by asking: 

• What information does the system need in order to complete the task well?
• Is your historical data and current data clean and reliable enough to use?
• What contextual data might you need to create from scratch?

Priority #5: Time to Value

Time to Value: how quickly the system can be deployed and deliver value.

Every organization wants to see measurable return on investment in a quick amount of time, and AI projects are no exception. But the reality is that, depending on your use case, some AI projects can take significant upfront effort to best design, implement, and train users on. That’s because time to value must consider more than just AI system build, but also how quickly users can adopt it and adapt their processes. A use case that requires deeper integrations, stronger data preparation, additional governance, or widespread rollout will naturally take longer to implement. 

Although time to value is an important variable, it should rarely be the reason you wouldn’t move forward with an AI system that provides strong value. However, you can use some guiding questions to decide if you should start with a smaller project or pilot with a smaller team first:

• What is the smallest version of this use case that will still create meaningful value?
• What is the one workflow, team, and data source that we can start with? 
• What would we need to prove in a pilot before investing in a larger implementation?

Priority #6: Creativity in Outputs

Creativity in Outputs: the system’s ability to generate novel, adaptive, or personalized outputs.

The level of creativity you need in an AI system depends entirely upon what kind of output the system needs to deliver. If your AI system needs to follow a very set path and deliver information the same way every time, it needs a low level of creativity. But if the system needs to react to different context and new information, with each situation being a bit different, it needs a high level of creativity. But, organizations also need to consider how much variation and creativity is actually useful, and how much can be damaging to their brand. For example, a system that writes a first draft of a blog may need room to explore structure, tone, and phrasing, while a system that summarizes a support ticket should stay much closer to the source material.

To start thinking about how much creativity you should embed in your system, start by asking: 

• Where would variation, personalization, or pre-approved phrasing make the output more valuable?
• How much variation are we comfortable allowing across different outputs?
• Who will be seeing the creative outputs? What happens immediately after the system generates the output?

Choosing the Right AI Mix

The most valuable AI systems are designed around the realities of the work they support. Even within one department, different tasks may require different levels of accuracy, explainability, control, data maturity, speed, and creativity. For example, think back to our three customer success examples at the start of this blog. Confirming whether a refund request meets a warranty requires a high level of accuracy, control, and explainability. Understanding when a ticket should be escalated requires reliable data and clear decision criteria. Drafting personalized responses to common customer inquiries requires more flexibility, creativity, and attention to tone. 

That is why it is so important for organizations to start with their use case. When organizations understand what matters most for each workflow, they can make better decisions about where the system should be strict, where it should be flexible, where humans need to stay involved, and where a smaller pilot may be the right place to begin. If you’re interested in applying this kind of prioritization to your own AI opportunities, Zirous’s AI Accelerator can help you evaluate use cases through our priorities matrix and identify initiatives that align to your organization’s goals, readiness, and desired outcomes.