The key to predictive AI project success

The exciting “rocket science” portion of any predictive AI project might be in training the model. But without proper focus on deployment, that rocket will be headed nowhere fast.

At NRF Nexus, Gooder AI founder and CEO Eric Siegel said machine learning projects that aim to improve business operations through the predictions they generate  — more often than not — don’t meet their intended targets.

Instead, they end up in a “no man’s land” between business professionals and data professionals, with each saying it’s not their role. 

The data scientist, he said, believes the model is certain to be deployed because its value is a no-brainer, and anything else might be considered “management issues.” The business stakeholder, meanwhile, is saying, “No. That semi-technical, that level of detail? I delegate all that stuff. That’s why I have data scientists.”

In that way, then, the “hose and the faucet routinely fail to connect,” Siegel said. “It’s really ironic: By focusing so much on this core rocket science, it’s like we’re more excited about the rocket science than the launch of the rocket.”

Deployment, however, is “the thing that matters,” and “the whole point” from a business perspective.

“That’s paydirt: the operationalization, productization, putting it in the field, changing operations,” he said.

“You don’t get any value just from the number crunching. The machine learning AI is not intrinsically valuable. It’s only if you act on it, and that’s what deployment is: where you’re actually changing operations to improve them with those predictions.”

Siegel suggested an alternate route: BizML is a set of business practices designed to increase collaboration — and, therefore, success in applied machine learning projects.

Of the six steps, the last three are “universal”: preparing the data, training the model and deploying the model. Before that, however, are three more “preproduction” steps: establishing the deployment goal (the value proposition), establishing the prediction goal (more detail), and establishing the metrics (both business and technical).

Gooder AI is currently doing an early-stage product trial that uses an interactive interface to turn technical metrics to KPIs, allowing users to visualize and explore what’s possible.

He advises business stakeholders to upskill on the semi-technical side so they can actively participate in the process from end to end. Too often, he says, machine learning projects focus on technical metrics such as precision, recall and accuracy, rather than on business metrics that anyone can understand — things like profit, savings and ROI.

“The business stakeholder would also say, ‘Hey, to drive a car, I don’t have to look under the hood,’” he said. “Which is true. I personally don’t know very much about the engine. I don’t know where the spark plugs are. But I’m totally an expert to drive.

“I know the rules of the road, momentum, friction, how the car operates, the mutual expectations of drivers. And now we totally need the same level of expertise to drive a machine learning project through to successful deployment.”

For those that might shy away from gaining semi-technical understanding, Siegel offers it in more detail in his recent book, “The AI Playbook,” which he described as “totally accessible, much more … generally applicable, interesting, exciting and easy to understand than high school algebra. It’s just not a part of the common curriculum yet.”

Siegel also shared case studies from UPS and FICO, as well as compared generative AI and predictive AI. Generative AI may be easier to use than predictive AI, he says. “But it’s harder to use well.”