VRMRCK

Your winning AI strategy for 2024

No. 6 Δ # AI and strategy @ Peter Vermaercke

2023 was the year you had to experiment with AI. This year is where you need to see your first returns. But you know by now that’s easier said than done. Here’s your guide to a winning AI strategy in 2024.

It’s crunch time

2023 was the year you had to experiment with AI. This year is where you need to see your first returns. But you know by now that’s easier said than done. And you’re not the only one struggling. Gartner placed GenAI for the first time in the trough of disillusionment: the phase where interest wanes as experiments fail to deliver. Frederik Anseel rightfully questions where those big AI-powered productivity gains are. Teams tasked with AI projects in organisations small and large are struggling to deliver GenAI’s anticipated business value.

Everyone sees its potential but getting there turns out to be incredibly difficult. Almost 90% of AI projects fail nowadays, according to Dan Saffer, Assistant Professor at Carnegie Mellon’s Human-Computer Interaction Institute. Those are some pretty bad odds.

So, summer’s over and you have 4 months left in 2024. What are you waiting for? Let’s make your winning strategy.

Do or die, GenAI?

Before we dive in head-first and create our winning strategy, it’s important to understand why everyone’s struggling with making returns on their AI investment so we can learn from our mistakes.

  1. The scope of AI projects is too large and expectations of GenAI are simply too high. GenAI is not going to completely overhaul your business overnight.
  2. The data required is either nonexistent or of very poor quality. Most companies sit on large amounts of data. True. But that pile often more resembles manure rather than the promised gold.
  3. AI is seen solely as a technical problem, whereas currently it’s more a UX problem than it is a technology problem. Too little attention goes to how users interact with your product.
  4. AI is positioned as the product, instead of as a means to and end. Users don’t care with which technology you will solve their problems. If it doesn’t help users reach their goal, it’s useless.
  5. The focus is too much on reaching high model accuracy first (“we need to reach 80% accuracy before we can ship this”, rather than looking at the relative gains it brings compared to current benchmarks.

If you’ve recently build an AI-product yourself you can probably complete this list with several more items.

Your winning strategy

Now let’s look at how we can turn AI into ROI. Generative AI – and more specifically Large Language Models – is general purpose. It can tell you a whole lot about a whole lot of things. But to reap its benefits, you need to narrow its scope and point all its smarts on a specific challenge for which you can tailor and control its output.

Benedict Evans compares it with an electric motor:

Here we have a general-purpose technology, and yet the way to deploy is to unbundle it into single-purpose tools and experiences. However, seeing this as a paradox might just be misplacing the right level of abstraction. Electric motors are a general-purpose technology, but you don’t buy a box of electric motors from Home Depot - you buy a drill, a washing machine and a blender.

So you need to look for single-purpose solutions. There are 4 key elements to a winning AI strategy:

1. Pick the right challenge

Everything rises and falls on a challenge. Just like any organisation, you’ll have many problems to solve at any given time. But not all problems are equal. Start from a list of pain points you or your organisation are currently struggling with. Frustrating, time-consuming, difficult, costly and/or cumbersome processes that are a pain in the neck.

Keep in mind that automation is not the holy grail. Instead, explore how AI can augment the human capabilities instead of trying to fully automate the process. Use our Levels of Automation framework to ideate how to best bridge the two types of intelligence.

Your ideal problem to solve scores well on all of the following characteristics:

  1. Significant business value The use case should significantly help your organisation thrive. This excludes vanity AI projects just to jump on the bandwagon.
  2. Overall strategic alignment Solving this problem aligns with your company’s overall, broader strategic objectives. This ensures keeping interest – and budgets – over time.
  3. Part of a loosely-coupled process The problem you’re trying to solve can be clearly defined and is easy to separate from other processes in your organisation. This allows you to get results quickly.
  4. Direct access to users Go for a problem where you have direct access to (potential) users. To gain speed you need to validate early and often.
  5. LLM-friendly context Training models is hard and expensive. If you want to gain speed, aim for a problem that an LLM could help solve. Reap the benefits of the hard work of OpenAI, Google and Anthropic.

List all your high-potential problems to solve and use the Data strategy canvas to pick out the one that overall scores the best.

2. Gather a dedicated, human-centred team

Creating a successful AI product is not a challenge you can just drop with your IT team or expect from your business units to do on their own. For AI products, a cross-functional team that closely collaborates is not just preferred, but required. Next to subject-matter experts from the business, you need at least (UX) designers/researchers and (ML) engineers involved from start to finish. Great AI products are created by business people with basic understanding of AI’s capabilities, together with AI experts with basic understanding of the business. This crossover is where the proverbial magic happens.

If your challenge involves any ethical or legal sensitivities (cf. AI Act), you’ll need to involve specialists in this field too. Ideally your core team can look at the problem from all relevant perspectives and slowly but surely build towards a solution that mitigates the biggest risks.

AI products are just as much a UX problem as they are a technical problem. Your team needs to be human-centred by focusing on the people you’re solving the problem for. This isn’t anything new but for an AI product this is even more make or break. Your team needs direct access to these people from the get go. This goes for (external) customers but just as much when building for internal teams. You can have the most amazing and powerful model, if nobody understands it or trusts it, it’s worthless.

You need frequent and direct contact with target users to ideate and validate along the way. Let them try the solution so you can shape it together. This will also help you understand how to best position the product. Nobody cares if it uses AI. Which of their problems are you solving? The how doesn’t matter. AI is here to add value, not to be the show pony.

At the same time you need to know what you can promise. A human-centred team can focus on bringing the smarts of your AI-powered product across to the users in the best possible way. Set the right expectations from the start. Make it clear what users can expect from using your product and what they shouldn’t expect so they’re not disappointed after trying it out.

Create a cross-functional, human-centred team that keeps a laser focus on the challenge at hand and the people for whom you’re solving it.

3. Validate feasibility first, usability second

Once your team understands the problem and the challenge is defined, you’ll want to work on a technical proof of concept. You want to validate the technical side first before anything else. In typical (non-AI) projects we tend to look at the usability side first. You want to imagine the best possible experience and adapt the technology accordingly. Not so with an AI product. It’s perfectly possible your problem is still a hard nut that can’t be cracked by the current state of AI. Or it could simply be too costly. Shiny new models come at a hefty price. You can figure out both things during the POC. Can it be done? And will it ruin us financially if this takes off?

An important part of this technical POC is gathering all required data and validating whether it’s actually good. Only this way you can find out whether the pile of data you’re sitting in is actual gold or rather menure.

Your team needs to do continuous user research to actively gather feedback from users. Make sure you have a process in place to use this feedback to improve the product. Let them know how you’re going to use this and how it benefits them. You need to sweat the details of the user experience to get it right. Small details can have a huge impact for an AI-powered product. You can only find out which these are by frequent prototyping and testing.

Start with validating the technical challenge first and – if successful – follow up with continuous user research throughout the project to constantly prototype and test with actual users.

4. Prepare everyone for a marathon, not a sprint

AI-enabled products are rarely a hit from the start. The potential is there, the tech works, some first signs of success appear but you’re not there yet. Stopping at this stage will set up your product for its final journey to the Elysian Fields of Tech. The excitement will wane of quickly and so will your budget. Building an AI-enabled product is a marathon. It takes time to get the technology right, tailor the user experience, the copy… and that’s okay. If you picked the right problem, it’s going to be worth it.

For your time and budget allocation you need to take this into account from the start. Too many promising solutions have been abandoned because the interest faded away when the first results were not as groundbreaking as hoped, or didn’t deliver from the start. Prepare all your stakeholders for a long run instead of a sprint, to make sure all your stakeholders have the right expectations.

Another expectation to manage is the accuracy of your model’s predictions or output. Specific percentages are thrown around and big claims are made about it (”we can’t go live before we have an accuracy of 80%”). The problem is this accuracy is always anchored by perfection: 100%. But that’s the wrong way to look at it and… humans make errors too. So it’s better to benchmark it against the current process and see how much better you score in terms of output quality, speed and cost. Don’t let yourself be pinned down on a certain accuracy number, rather focus on the improvements compared to the current way of working.

Make sure to communicate regularly with all involved about the progress you’ve made and the challenges you’re still trying to tackle. Sharing user research findings is also a great way to keep people involved and interested in the process.

So what are you waiting for? Let’s make 2024 the year of AI’s first ROI at your organisation. Doing this all by yourself can be daunting. Let your team be surrounded by experts that can get you up to speed faster.