Cloud analytics: a disruptive technology

At the fall 2010 INFORMS Annual Meeting, I organized an INFORMS Roundtable session focused on cloud computing. Speakers from Microsoft, IBM, RightScale, Gurobi Optimization and Frontline described their work with cloud-based analytics. Almost five years later in May 2015, Gartner issued a “Market Guide for Optimization Solutions,” which asserted that, “despite very few cloud deployments today, most platform vendors are moving to cloud.” Was this accurate? In early 2016, just how prevalent – and how practical – are cloud-based analytics applications?

As president of an analytics software vendor who’s in close contact with leading-edge customers as well as other analytics vendors, I’m aware of both usage statistics and specific cases, from us and others, that I cannot cite – customers often consider their cloud analytics solutions a competitive advantage, and won’t permit us to name them. But I can say with certainty that there were far more than “a very few cloud deployments” of analytics tools in early 2015, and growth over the last 12 months has accelerated. One statistic I can cite: At the time of the Gartner report, Frontline had 45,000 users of its cloud spreadsheet analytics tools for Excel Online and Google Sheets, and that number has nearly tripled in less than a year, as this is written.

A Disruptive Technology

Analytics in the cloud is following the pattern of other disruptive technologies (including personal computers), described so well in Clayton Christensen’s classic book “The Innovator’s Dilemma.” The early users of cloud-based analytics are not always, or even usually, the traditional large BI or analytics users – they are both new users of analytics per se, and existing users pursuing new applications that weren’t practical before. Some cloud-based analytics tools are less mature and lack all the features of desktop tools, but this isn’t what’s important for the new users and their applications (meanwhile, the gaps are rapidly closing).

Growth in cloud-based analytics is happening both because it enables new uses and benefits, and because old objections are fading.

Security and Privacy: Fading Concerns

In the discussion that followed the speakers at the 2010 INFORMS Roundtable meeting, representatives of nearly every company cited concerns about information security and privacy. The analytics group leaders on the Roundtable, and even more so their upper management and IT staffs, could not imagine that public clouds such as Amazon Web Services and Microsoft Azure could be a secure place to store their sensitive data and analytics results. Several reps stated that their company would “never” make use of cloud services for their proprietary applications. (I wouldn’t hold them to these statements in 2016.)

But one needs only a newspaper to realize that the alternative to cloud computing – a firm’s own data centers – is hardly a protection against loss of valuable proprietary data. In 2013, Target lost 40 million credit and debit card accounts, plus data on 70 million customers. In 2014, Home Depot had a breach of 56 million credit card accounts, JP Morgan lost sensitive financial information on 76 million households and 7 million small businesses, and eBay had a breach of 145 million customer accounts. And in 2015, Anthem lost 80 million patient and employee records, Ashley Madison had 33 million names of affair-seekers exposed, the IRS disclosed full tax returns of 500,000 people to hackers, and the Office of Personnel Management lost sensitive records – including security clearance details – on 22 million federal employees. Not one of these breaches occurred in a cloud computing system.

In 2013, the Central Intelligence Agency chose the “newcomer” Amazon Web Services to build a secure cloud service for the intelligence community. According to Fortune magazine in mid-2015, the “intelligence community loves its new Amazon cloud.” If it’s secure enough for the CIA, is it secure enough for you?

Advantages: Scalability, Accessibility, Collaboration, Agility

An advantage often cited for cloud computing is scalability – the ability to “spin up or down” virtual machines in minutes, when physical machines can take weeks to months to install, and usually must be written off when not needed. Though we’re unable to cite customer names, GAMS Optimization, Gurobi Optimization and Frontline Systems have seen this happening in practice – we’re aware of multiple customers scaling up to hundreds of virtual machines in the cloud 24×7, just for their own applications. Thanks to scalability, Frontline offers use of its Apache Spark Big Data cluster, running on Amazon Web Services, to academic customers at minimal to no cost. These are new applications in the spirit of “The Innovator’s Dilemma” – they weren’t economically practical before.

Accessibility of analytics tools using only a browser has opened up opportunities for new users, again in the spirit of “The Innovator’s Dilemma” – especially in foreign countries where powerful computers and desktop or server software sold on permanent licenses are not nearly as common as in the United States and Europe. Many of Frontline’s new users of its cloud-based Solver (optimization), Risk Solver (simulation) and XLMiner (statistics/data mining) tools are from those countries, and are using the free versions of Excel Online or Google Sheets. Their applications may be modest compared to big company IT-sponsored analytics initiatives, but their numbers are large and rapidly growing.

Easy collaboration is a major advantage of tools such as Microsoft’s Power BI, Tableau Online, SAS Cloud Analytics, FICO Analytic Cloud and Frontline’s XLMiner.com and Rason.com services. Companies of all sizes are now operating with geographically distributed teams, and members of those teams collaborate every day via online meetings and live chat. They need, and now they can get, immediate access to results from analytics applications – not just slide presentations from projects where key insights were discovered months ago. The significant impact on agility in decision-making is increasingly important in a competitive world with few geographic barriers.

Analytics in the cloud isn’t just a future trend – it’s happening now. Based on our data, I’m confident that while you’ve been reading this article, someone new, somewhere in the world, whom you never thought of as a “data scientist” or “management scientist,” has started using advanced analytics tools in the cloud. They’re learning how to make a difference with analytics. Are you ready?