Who needs a data scientist?

Salesforce.com’s annual Dreamforce conference recently descended upon downtown San Francisco featuring 171,000 registered attendees. Because my campus is just a few blocks from the site of the conference, this event regularly hits my calendar. And for the past few years, machine learning and artificial intelligence have been a huge topic at Dreamforce.

Two years ago, after a slew of acquisitions of ML/AI-based vendors (which also brought in a great deal of data science talent), Salesforce announced a new initiative called “Einstein.” Although it launched much like a product, Einstein is really a set of AI technologies and APIs that are intended to make it easy (for Salesforce, their customers and third-party developers) to add predictive power and intelligence to nearly any activity on the now-massive Salesforce platform. From what I can tell, a big part of Einstein’s value proposition is its ability to efficiently and effectively simplify a great deal of data preparation, as well as much of the core model building required to address specific business problems – all while maintaining data confidentiality and customers’ trust.

Since that initial announcement, much has happened with Einstein. In particular, in 2017, Salesforce launched two major new products called Einstein Analytics and Einstein Discovery [1]. Einstein Analytics embeds predictive tools for Salesforce’s sales, service and marketing clouds through a collection of context-specific applications. Meanwhile, Einstein Discovery is marketed as a vehicle to leverage huge amounts of data (from both Salesforce and non-Salesforce sources) to spot trends, identify issues and offer actionable model-driven prescriptions (in the form of PowerPoint presentations!). A key tagline from the Einstein Discovery marketing: “Understand your data without a data scientist!”

As it happens, just a few days after Dreamforce, I had a visit with my colleague Nathanael Rosidi. Dr. Rosidi has spent the better part of the last decade working as a data scientist within the healthcare space, most recently as a senior marketing scientist at Genentech. When I mentioned the Einstein initiative to him, it was clear that I had hit a nerve. After a brief discussion over cocktails, Dr. Rosidi followed up in writing a few days later:

While these kinds of products can enhance the way business traditionally operates, they are usually off-the-shelf, black box solutions that are made to fit all departments across all industries.

An emerging downside I’ve noticed since these types of software packages have become popular is a perception that data science is a commodity that can be bought as off-the-shelf software (or easily outsourced). In a lot of places, the result is a reduction in an investment for in-house analytical expertise. While this isn’t true for companies where data science is a core component of the company’s competitive advantage (for example, Stitch Fix, Google, Airbnb), this seems to be increasingly true for companies where data science isn’t perceived as mission critical, especially in departments like operations, sales and marketing. Executives in these departments seem particularly ripe for the kinds of marketing messages that Salesforce is putting out there – and they often don’t really know what they are and are not buying.

Here are some of my thoughts as to why one cannot automate data science completely and still expect a comparable ROI.

First of all, let’s remember that data science is 80 percent data cleansing and 20 percent model building. The first steps of data cleansing are fairly simple and can be automated. It involves tasks like dealing with missing and extreme values in your data. However, the next steps are more complicated and require deep domain knowledge of your business and industry. They often involve dealing with feature selection (i.e., choosing the data of interest) and creation of new and proxy metrics that describe your business. Because each problem is unique to your business, these tasks are subjective and requires an understanding of the trade-offs of every parameter in the model. Important note: Data models certainly can and do sometimes fail, but being able to drive discussion and consensus about the trade-offs with the leaders of your team will help to ensure that everyone is on the same page regardless of how the model performs. That just doesn’t happen if the model is a black box.

The last 20 percent of data science is building models using a set of mathematical equations to best describe your data and offer predictions and recommendations. Which model you choose is up to you and clearly depends heavily on your business problem. But once you’ve implemented your model, how do you know the output is any good compared to other approaches? How will your model and your insights change over time as your data and your business changes? The most effective approach to data science is to constantly iterate on how you cleanse the data and implement the models. The interactive human component ensures that both the data and model continue to best represent your evolving business.

I actually agree with Dr. Rosidi on much of this stuff. But I had long ago [2] accepted the inevitability of packaged analytic applications becoming a bigger and bigger part of the marketplace. This is partly because of the shortage of data scientists and the high cost of custom application development, but also because analytic software packages like Einstein Analytics and Einstein Discovery promise to reduce the time it takes to gain business value, something that is increasingly alluring in today’s fast paced business environment.

For their part, the folks at Salesforce already appear to be trying to address some of these concerns. The choice to open up the Einstein APIs to third parties has created opportunities for many new ML-based application vendors [3] with specific domain expertise, all purporting to bring the power of predictive analytics to bear on business problems associated with sales, marketing and customer service. Einstein Analytics also includes its own self-service model building capability, while customers who have their own data scientists can also directly access the underlying models generated by Einstein Discovery.

But any manager or executive relying on a packaged ML/AI solution should take Dr. Rosidi’s comments and concerns to heart. Whether you choose to build or to buy, make no mistake: None of this is either simple or easy.

Notes & References

1. The two products are now available from Salesforce as a discounted bundle
2. http://analytics-magazine.org/analyze-this-standardized-analytical-solutions/
3. See for example https://www.conversica.com/blog/top-10-ai-companies-to-visit-at-dreamforce/