Build/Buy, Pause/Play: Scaling the Future of AI in Retail and CPG Is Complicated
Roughly one year after ChatGPT’s oxygen-sucking public debut, the retail and consumer goods industry is grappling with a host of evolving challenges in the use, adoption, and scaling of artificial intelligence. While AI isn’t new, the thrust of generative AI into the hands of just about anyone changed the game.
Tech leaders and practitioners across myriad industries gathered in New York last week for the annual AI Summit to share experiences, best practices, cautionary tales, and more. Among the takeaways for business leaders:
Responsible AI Should Be — Full Stop — the No. 1 Priority
Don’t waste your time with mere lip service when it comes to the ethical considerations for responsible AI. From a business standpoint, sustainable success hinges upon it: Skip obtaining buy-in at all levels and kiss your scalable dreams goodbye.
Today’s companies require constructs that will lead to trustworthy AI, stressed Inderpal Bhandari, Walgreens Boots Alliance board of directors member and the former global chief data officer of IBM. This includes not just ethics and governance boards but also a slew of mechanisms to monitor, measure, and audit aspects like fairness and transparency.
Engage with your peers and learn more about Building the Future of Unified Intelligence at Analytics Unite, held May 1-3 in Chicago. Register here.
There are also considerations surrounding the environmental impact for some generative AI tools, said Taha Mokfi, associate director of data science at HelloFresh. “You don’t want to spin up the GPU to ask what is the capital of the U.S. That’s really costly as opposed to searching on Google, which is much less.”
These are both top-down and bottom-up mandates — lacking trust from senior leadership means poor resource allocation, while lacking trust from the rank-and-file promises poor adoption rates.
- Related: Future of AI in the Supply Chain
“If they’re concerned about losing their job — if they’re concerned about AI essentially picking up the knowledge they have, encapsulating that in software, and then it being shipped out to some vendor — it’s not going to happen,” Bhandari noted.
While this cultural aspect is the hardest one to solve, it’s the most critical for AI’s use within business. “If you’re unable to solve that problem, it’s not going to scale. You might have a success or two, but you won’t get it to scale across the enterprise,” he said.
Obtaining buy-in at all levels can mean speaking different languages depending on the stakeholder, advised Eugenio Zuccarelli, manager of data science at CVS Health.
It can also mean starting small, and Wayfair has done just that, initially using generative AI’s summarizing function with about 8-10 customer service agents.
The early tool was a bit janky, conceded Mokfi, but things improved. “As we invested more and more in training our specialized models and improving our LLM models, the speed increased, and with that we were able to roll it out for much larger populations,” he said. “Right now, thousands of our agents are using that application, and that’s reducing our average handle time dramatically.”
The experimentation aspect coupled with A/B testing is helping build trust within the organization, he said, and they’ve applied the same application in different use cases.
Don’t Build On Rented Land, But Vacationing Can Be Smart
Determining whether to build or buy is an old challenge for tech teams, but generative AI’s unprecedented rate of change added a new wrinkle. While companies should be leery of outsourcing their competitive advantages, the benefits of being a first-mover can make sense to partner for some core capabilities.
In retail, experimentation with generative AI abounds, but actualization remains difficult and expensive, according to Deborah Weinswig, Coresight Research CEO. The biggest misconception for today’s retailers is that you have to go big, whether that means initiatives or partnerships; however, there’s a plethora of providers, some of whom can drive better, faster, and cheaper results.
- See also: 2024 CPG Tech Budget Priorities
“We’re finding if they are doing the PoC or the experiment with folks only externally, that they’re able to move much faster and make decisions,” Weinswig said.
Technical Debt Ahead
Garbage in, garbage out: Talk to most executives about the biggest challenges surrounding generative AI, and this tops the list. Feeding models with clean data is critical, but privacy concerns are real, with many companies understandably wary of open-source tools. Things get even trickier when synthetic data is introduced in fine-tuning models, noted Kriti Kohli, Shopify senior manager, applied machine learning.
“As you generate some of the synthetic data that you're using to fine-tune LLMs, how does that synthetic data actually map to the real probability distribution of what you're going to see once this product is actually out in the wild?” asked Kohli.
All of this takes a toll on infrastructure, and the speed of innovation in this area is so fast, you’re almost always building up tech debt, she said.
Technical debt is indeed significant surrounding generative AI, and it’s led by a lack in best practices, agreed Ranadeep Singh, a data engineer at StubHub. Today’s tech landscape has many people learning as they go and tools being developed on the fly.
“[There is a] lack of knowledge of best practices of how do you evaluate your prompts? How do you evaluate the tool you're using? A bunch of those concerns don't have a definitive answer today, and I think that's a concern because that’s building up technical debt,” he said.
“As more becomes available out of the box, it’s on us to evaluate some of these models,” said Saira Kazmi, CVS Health executive director, enterprise data and machine learning engineering. “Because some of the companies are not open sourcing the training of data or the process of training these models, it’s going to be probably harder to use them in regulated industries. It may be easier to use them in cases where you're not impacting decisions around health or finance.”
AI Literacy Is Transforming Roles
Generative AI’s potential to drive efficiencies is among its most powerful siren calls. Srini Kandala, head of AI, technology and online business at Sears, showed how they’re using AI-fueled chatbots and avatars to recruit home repair technicians. Doing so has helped relieve hiring managers from the time-consuming process of writing interview debriefs, he said.
And as employees increase their AI literacy, the rise in accessible technology means both tech and non-tech roles will evolve. StubHub’s Singh compared it to the introduction of Excel, noting that he now has product managers running SQL queries, and he expects this to only grow as technology is more accessible for everyone.
“Because of these AI technologies, everyone is just more capable. [Looking ahead], you won't have front-end or back-end software engineers. You won’t have prompt engineers. Everyone will be a full-stack engineer. With just an understanding of how to build systems, you can build every part of it,” he said.
Not Everything Needs AI, But AI Needs a Sandbox
Even at an event stuffed with AI evangelists, it was widely accepted that not every challenge needs AI. It’s an urge that should be tamped down but not completely extinguished, and CVS Health’s Zuccarelli acknowledged it can be difficult for those with technical backgrounds.
“One of the challenges is trying to forget about the technical components. … It’s really an art to be able to understand the highest value has to come from the business value,” he said. “Sometimes the solutions are going to be Excel or nothing too fancy, but being able to focus on that usually leads to the most value.”
At the same time, today’s practitioners require sandboxes and time to play in them, and providing early access to new tools can be important to obtain buy-in from the data scientists on your teams. After all, enabling this relentless curiosity is what often drives technology forward.
“Once you get a hammer for the first time, you don't know exactly how you might be able to use it. Just give it to them. Give them a chance in the sandbox, start playing with it to figure out what's what,” said Jeff Chan, senior manager, enterprise innovation at TD Bank Group. “Once they get to understand what the hammer does, then they can come back to us, and we can say, ‘How do we connect the dots between what you just learned on the hammer and to solve an actual problem?’ And then that's where magic happens.”
