Nurturing Artificial Intelligence

It’s baaack! Artificial intelligence (AI) is once again a hot topic in U.S. defense research and development and related procurement. The recent attention mostly focuses on “machine learning” (ML) and “deep learning” (a subset of machine learning) this time around. Both public and private sector organizations in the business of developing machine-based systems to augment human decision-making are enjoying a new round of excitement and funding. Success in such areas as chess, the board game Go and the “Jeopardy!” TV game seem to indicate real breakthroughs.

Watson demoed by IBM employees. Source. Raysonho @ Open Grid Scheduler / Grid Engine [CC BY 3.0 (https://creativecommons.org/licenses/by/3.0)], from Wikimedia Commons.

As some experienced analysts remember, however, we’ve seen this before, with a surge of optimism about AI in the 1960s and 1970s, then a rebirth, mostly called “expert systems,” in the mid- to late-1980s. Both these bursts of activity and enthusiasm were followed by what was called “AI Winter,” as disappointed sponsors stopped supporting research and development in AI. Will it be different this time?

A new study by the Center for Strategic and International Studies (CSIS) [1] highlights an important factor neglected in the national security digital transformation, a factor that is critical now: the development of what the study’s authors call an “AI Ecosystem.” The AI Ecosystem involves:

• training people to implement AI,
• engaging people to interact as collaborators with AI systems and with their developers as the systems are designed, and
• making sufficient training data available for the AI to “grow” to encompass the full range of issues involved in its missions. 

CSIS is one of the Washington, D.C., area’s leading think tanks dealing with defense and national security issues. Andrew Hunter, director of the Defense Industrial Initiatives Group and a Senior Fellow in the International Security Program at CSIS, as well as director of the study [2] and chair of the panel discussion to introduce it, says, “It was a challenge to capture all the topics we discussed, to try to create an overarching concept and narrative. We think we were able to tie things together. Investment, international involvement and adoption form a common thread.”

The report noted a number of issues that have impeded the adoption and utility of AI and proceeded to offer a number of recommendations for U.S. government action to advance the development and implementation of AI (see sidebar story). In summary, the report’s authors concluded early AI, or rules-based AI, was limited in its capacity compared to today due to the need to hard code every rule or behavior. More promising developments, often leveraging machine learning, are paving the way to AI systems that will introduce the capability to collaborate with humans, augmenting rather than supplanting human decision-making. This implies a much larger investment in the human side of the field and a different view of the skills those people will need – and of the data required to support inference.

Hunter was joined on the panel by Ryan Lewis, vice president of In-Q-Tel (IQT) CosmiQ Works; Erin Hawley, vice president of Public Sector, DataRobot; and David Sparrow, a researcher at the Institute for Defense Analyses (IDA). IQT CosmiQ Works (“CosmiQ”) is an applied research lab that focuses on artificial intelligence for remote sensing data. DataRobot is an automated machine learning software platform. IDA is a long-established consulting organization that supports high-level decision-makers in the Department of Defense. It was a lively and informative discussion.

Many people consider IBM’s Watson a leading example of current AI. However, the documented accomplishments of Watson in real applications seem to have fallen short of the developers’ hopes, and it may end up instead as an example of AI being harder than anticipated. A number of researchers who have worked with Watson report that it requires a great amount of human preparation of data before it can meaningfully contribute. “I struggle a little with what Watson means,” Hunter said in an interview shortly after the panel discussion. “[Is it] one algorithm? A suite? A suite with humans? What’s the conceptual approach?

“We shouldn’t limit our imaginations to the way machine learning is done today,” he added. “The only way it will get done in the future and be most applicable to military problems is to spread out our bets.”

Redefining the Subject

Hunter’s recommendation means diversifying the very idea of how machine learning is used today, not just varying inferential methods. Further, it means rethinking the processes and applications to which machine learning is applied. This implies that what some operations researchers and management scientists would try to do as AI developers or evaluators may not work well.

The problem also implies a need for a different approach to analysis – and a warning from Sparrow to the analytics profession to focus on the real problems people have rather than solving mathematically tractable, simplified problems. “When I left theoretical physics, I knew I was leaving theoretical physics,” Sparrow adds. “In analysis, we’re supposed to aid in making better decisions right now. Theoretical work is important, but it’s not our emphasis.”

In Sparrow’s view, ML is moving in the direction of emulating intuition – or “system 1” thinking from Kahneman’s book “Thinking Fast and Slow” [3]. “When we go to deep neural nets and machine learning systems, one can argue that all we’re getting are trained gut reactions that have been socialized in ways we don’t understand,” Sparrow says. “What these systems produce is more like hunches than solutions to logical problems. That makes testing and evaluation really challenging. The problems may lie more in the training data than in the algorithms. We don’t know how to test.”

This applies to data-driven ML systems, and is exacerbated by the enormous, complex state space for decisions and the importance of human machine interactions if machines that collaborate with humans are to be key.

Sparrow has worked extensively in test and evaluation and noted the difficulty of assessing AI systems. “If you want artillery rounds that fail fewer than once in 10 million shots, you can’t test a hundred million rounds,” Sparrow notes. “You have to have an underlying theory to define the search space, so you know which events would most stress the system you’re testing. We don’t have a theory like that for AI, much less AI that works peer-to-peer rather than tool-to-user.” The key to getting AI into use in the field, Sparrow says, is “a much more aggressive experimentation program to support both development and evaluation.”

Adds Hunter: “How does AI get integrated into operations? AI tends to be limited in what it can do and does not always deliver meaningful outcomes. But people who can integrate it into broader systems can do more. This looks like a market opportunity. For example, I’ve heard a lot of stories about much being done in finance but have little exposure to information about how well it does.” 

DataRobot approaches artificial intelligence from a very defined perspective. Motivated to combine the talent of top-ranked data scientists, DataRobot builds software that can train, test, compare and deploy machine-learning models to augment business users with state-of-the-art data science and machine learning. Asked to explain DataRobot’s success in applying AI in the financial services, insurance and healthcare industries, Hawley says, “We focus on a piece of AI, namely automated machine learning (AML), where we look to support cultures that embrace the data-driven enterprise. AML is a powerful application of machine learning that drastically speeds up the process of creating accurate predictive models, while also lowering the barrier to entry for organizations that want to leverage highly complex and sophisticated algorithms to specific business questions, all without the requirement of a full data science team to do so.”

According to Hawley, this type of approach could be applied to answer a wide variety of very specific business questions. For example, DataRobot can help address the huge backlog of U.S. government security clearance applications waiting to be processed and adjudicated.

“Take the scenario where the government needs to prioritize and streamline a list of who is most likely to be approved (or denied) for a clearance,” Hawley says. “By taking a sample of the historical data where the government has already made similar decisions about other applicants, a user could feed that information into the software with the click of a button. DataRobot would then build and compare models based on hundreds of the latest machine learning algorithms so that the users can easily understand and gain visibility into the people with the greatest likelihood to be granted a clearance, as well as those who need further investigation and a greater level of scrutiny. This entire process can be done within hours and automate a process to prioritize tens of thousands of applications that would historically have taken months to complete.”

Train the Models Differently

As Sparrow points out, developing AI that is more intuitive and human-like poses serious issues about testing and evaluation, and the problems most likely can be traced back to inadequacies in the training data. Therefore, data science and data handling are critical to the process.

CosmiQ has also focused on one relatively narrow problem set: computer vision techniques for remote sensing applications. Specifically, CosmiQ focuses on machine learning implementations that automatically extract visual foundational mapping features such as buildings footprints, roads and other objects from overhead imagery, primarily from satellites and aircraft. While the development of algorithmic implementations and associated software tools is important, it is equally pressing to make sure end users understand how to interpret the results from machine learning models.

“The solution is not just building a model but selecting an appropriate evaluation metric to accurately represent results for the given application,” Lewis says. “Both are necessary to kick off a machine learning project or pilot effort. As we’ve started open-sourcing more tools at CosmiQ, as well as high-quality, labeled data sets through our participation in SpaceNet, we’ve made updates to open source models and workflow based on lessons learned from our follow-on applied research projects and SpaceNet challenges. Looking ahead, it will be important to continue this work as more data sets and software tools become available.”

In the mapping and geospatial analytics domain, understanding what tools and data are necessary for particular mapping feature is of foundational importance. “Understanding what data are interesting or potentially useful is a hard and time-consuming problem,” Lewis says. “Developing and implementing the appropriate data labeling taxonomy can highly impact what kind of questions researchers and companies can try to answer. We have to make sure there are robust, curated data sets available for experimentation to make sure we can begin to address those types of questions quantitatively.”

This need becomes more difficult to address when many of the data sets are classified. “Classified data sets tend to be highly protected, managed in siloed organizations,” Hunter notes. “We need much more openness and sharing to be able to train AI systems. The U.S. may be on the wrong side of the openness issue. The Chinese government owns all the data and can combine and compare data freely – within their own overall control, of course. Here in the U.S., we’re more protective of privacy and confidentiality, but then we have commercial enterprises that can gather and share an astonishing amount of personal data.”

The point: The United States may need to do some serious rethinking about classification rules, and ethics in general.

Another data-related issue that calls for more thought is the way big data is managed. Much modern data handling is designed to make storage most efficient, often to the detriment of efficient retrieval. A common method called “sharding” efficiently finds available storage just the right size for chunks of data – and throws away any information about which data it came in with [4]. As earlier studies in AI indicate, randomly scattering data and expecting the vastly powerful search engine to find and reconnect it all is not a good idea. Structuring data storage to facilitate efficient retrieval, rather than to make storage most efficient, looks like another promising area for research.

Summary and Conclusions

The CSIS study strongly called for a shift in emphasis in AI: training people to use AI well, interacting with it creatively, and providing meaningful feedback to AI designers is a critical need, perhaps the most critical. More needs to be done to train humans and improve data to leverage AI. Potential rival nations are already intensively doing some of this work, so these needs, in the study authors’ view, deserve a high priority in U.S. government support. For defense and intelligence applications in particular, the needs include changes in selection, skills mixes, training, organization and management of people, and serious revisions of how data are managed. Analysts who wish to participate in this effort will need to develop creative thinking and social interaction skills, engage across disciplines, and perhaps de-emphasize formal logic and well-established solution methods.

References

1. Hunter, Andrew, Lindsey Sheppard, et. al., 2018, “Artificial Intelligence and National Security: The Importance of the AI Ecosystem,” Center for Strategic and International Studies, November. Downloadable from the site for the discussion event [2].
2. https://www.csis.org/events/artificial-intelligence-and-national-security-importance-ai-ecosystem
3. Kahneman, D., 2011, “Thinking, Fast and Slow,” New York, N.Y.: Farrar, Straus and Giroux.
4. Samuelson, D.A., 2014, “The Sharding Parable,” OR/MS Today, April issue.