Deploying High-Quality and Trustworthy AI

Artificial intelligence (AI) has huge transformational potential in many industries, but most organizations have found it difficult to scale up adoption. The issue of AI’s trustworthiness – or lack thereof – has often been seen as a key barrier. Last year, the European Commission released its Artificial Intelligence Act, a comprehensive regulatory proposal that has highlighted the importance of transparency, reliability, robustness, fairness and other characteristics of a high-quality AI system. Industry-level regulators such as the ECB and national regulators such as Banque De France have made similar recommendations. 

Much ground has been covered around the conceptual frameworks to deploy trustworthy AI across numerous publications and conferences, but little is known about industry actors’ responses to this challenge. In practice, how do they embed AI quality and trustworthiness requirements into their model development, testing and deployment processes? Are there any significant gaps across industries and geographies? How are leading AI adopters approaching this? 

Practitioners’ forums to share perspectives on these important questions are rare but essential to the industry to make progress on the trustworthy AI challenge. In September 2022, I helped organize a Europe, Middle East and Africa (EMEA) AI executive roundtable, an invitation-only virtual event for data science, technology and business executives who are driving AI adoption within their companies. The event also included representatives from international organizations and standard-setting bodies. 

The primary goal was to facilitate peer-to-peer learning discussions between AI/ML industry executives across EMEA, and selected counterparts from the U.S., on how to make AI initiatives successful in the real world, through a focus on AI quality and trustworthiness. The event began with a panel discussion on the challenges to achieve real-world success from AI, featuring senior AI executives from retail, finance and gaming leaders and moderated by the former chief data officer of Standard Chartered Bank. 

Three broad themes were covered: 1) the state of AI adoption across industries, 2) attitudes toward emerging regulations and 3) the key elements of a robust AI quality framework. 

The key takeaways include the following.

• Investment into AI and ML continues at pace, across industries and geographies. Participants confirmed that investment into the people, processes and technology needed to build and deploy AI/ML systems remains a priority. Even in the context of highly regulated industries, we heard of extensive use of AI/ML, ranging from back-office automation initiatives in financial services to real-world evidence generation in healthcare and pharma.

• Technology-first vs. technology-enabled business models show clear differences in the scale and speed of AI adoption. On one side are the technology-first “new” businesses – across sectors such as e-commerce, ride-hailing, search and gaming – for which large-scale adoption of AI/ML is not an optional curiosity. It is the only way they can survive, given their scale and business models. There is no way that hundreds of millions of products could be dynamically priced or recommended withoutusing AI/ML.
  
  On the other side are traditional businesses, for whom AI/ML has largely been a means to incrementally improve existing processes. For example, banks have had statistical and rule-based models in place for credit decisioning and fighting financial crime for decades. They have well-defined frameworks to govern these high-stakes use cases. This can result in a high bar for the adoption of new AI/ML approaches (i.e., “Why break things when they work?”) and limit AI adoption to small pockets of experimentation. 

• There is little evidence of European companies facing a major regulatory handicap in AI adoption. The existence of a robust privacy regime (GDPR) and the tagging of several AI use cases as “high risk” in the upcoming EU AI Act do not appear to have had a huge impact on European companies’ adoption of AI yet. There is a fault line between industries, but our conversations have not unearthed much evidence of a systemic disadvantage arising from being in Europe vs. North America (or Asia). If anything, the certainty that Europe’s regulatory regime and associated standards and certification efforts might bring (versus the more fragmented and uncertain landscape in the U.S.) could place European companies in a position of advantage.
  
  Some of the companies in geographies without national or regional regulation, such as the U.S., welcomed the prospect of regulatory guardrails that would positively channel development efforts and remove bad actors from the field of competition.

• Trustworthy AI cannot (just) be about regulatory compliance. To date, a lot of the conversation around making AI trustworthy has focused on regulatory expectations around ethics, fairness, transparency and explainability. However, expecting data scientists and their business or technology partners to meet ethical goals as a standalone objective is futile. The only sustainable way to make AI trustworthy is to ensure such objectives are tied to business key performance indicators (KPIs). For example, explainability is arguably even more important for internal buy-in and customer trust than it is for regulatory compliance. Customer and media backlash to perceived or demonstrated malfeasance may be faster, harsher and more likely than possible regulatory penalties.

• A broad consensus is forming around core technology requirements for trustworthy AI. Different industries and companies are at varying maturity levels regarding the technical components of a trustworthy AI system. However, a baseline of common requirements is forming, based on guidance from regulators and industry bodies, corresponding standards initiatives, and existing risk frameworks such as model governance in financial services or equipment safety in manufacturing. 

• A key theme is the emergence of a more holistic “AI Quality” framework, expanding from the previous, individual considerations around explainability, fairness and ethics. This new AI Quality framework includes considerations such as:
  • Complexity-performance trade-offs.
  • Model robustness and resilience in the face of real-world changes.
  • The security implications of using AI models. 

The AI/ML technology ecosystem is fragmented and does not yet address all these requirements neatly. However, panelists reported seeing rapid evolution in this space, and leading adopters are beginning to orchestrate end-to-end AI/ML pipelines that consciously incorporate AI Quality elements.

These discussions provided valuable insights into industry actors’ responses to the trustworthy AI challenge. Their focus has clearly moved from high-level frameworks to practical considerations around implementing AI quality and trustworthiness requirements into their model development, testing and deployment processes. Most are still unsure about how to do this at scale, but leading adopters are showing the way by demonstrating strong leadership from top management, facilitating cross-functional collaboration, establishing clear lines of accountability and providing appropriate tools to data scientists.

Further, there was a sober optimism about companies’ ability to complete this journey, partly based on the sense that the gap between regulatory and technology requirements is rapidly narrowing. The AI/ML technology ecosystem is still fragmented, but some tools are well identified. Regulatory uncertainty persists in some areas, but customer demand for trusted AI systems is driving action.