Taking a Stand for Data Ethics

During the 2019 INFORMS Annual Meeting in Seattle, five data science researchers and practitioners gathered with a group of meeting attendees to discuss “Debiasing Decision Making - Ethical Data Mining and Eliminating Algorithmic Bias.” The participating panelists included Nathan Colaner, director, Bridge MBA, Seattle University & Data Ethics Researcher; Bill Franks, chief analytics officer, International Institute for Analytics; Jitendra Mudhol, founder, CollaMeta & Executive Fellow, Miller Center for Social Entrepreneurship at Santa Clara University; and Brian Wright, professor at the University of Virginia’s fledgling School of Data Science, the first school  added in 50 years.

AI/ML Emergent

According to Mudhol, there are more than 300 human biases that trip up our decision-making abilities; Colaner reminded us that we need more consistent sensitivity to data interpretation upfront.

“AI has gone from zero to frequently implemented in a very short time. Eighteen months ago, the only thing discussed was the privacy of personal data. Now, another layer is finally getting its due,” said Franks. According to him, we are operationalizing data into processes that impact us in myriad ways. Algorithms are making more and more significant decisions for us, and about us, that have a significant impact on our lives.

The idea that companies have to tackle the question of algorithmic bias, and the risk it poses to their bottom line, is now emerging. Companies need to evaluate and take a stand on this issue, and then re-evaluate iteratively. The problem is that when these decisions are made in the line of fire, they are not always made optimally. As the practice of inserting AI and ML into more and more businesses processes increases, the risk to companies and the risk of society getting it wrong also increases exponentially.

Understanding the Front- and Backend of the Data Ethics Pipeline

Wright took a bold stance when he said that data science has nothing to do with algorithms. His point was that the onus for handling questions of bias and ethics rests solely with humans – in the questions we are asking and in what we do with the data. Where is the data coming from? What questions are being asked? What do you operationalize? How will it be used? What is the analytical technology framing? All of this is where humans are needed to create, consume and assess, which can be accomplished by taking the necessary thought experiments seriously and appropriately contextualizing the questions.

An analytical process flows from the questions to collection to analytical model(s) to the use of data and the use of the model itself. Every single point in the process matters from a data ethics perspective. It may be necessary to establish industry-wide codes of conduct for data ethics.

Building a Choice Architecture

At CollaMeta, Mudhol and his colleagues refer to this conscious tackling of bias as “choice architecture.” They look at the work of behavioral science researchers, such as Daniel Kahneman, Richard Thaler and B.J. Fogg, to see that it is possible to put the right triggers in place to ensure better, more ethical outcomes. The challenge that they see is often too narrow a framing of a particular problem, often taking a short-term approach without fully vetting long-term outcomes.

What Keeps These Experts Awake at Night? 

According to Colaner, it’s worrisome that many of these technologies have the potential to increase and solidify the divisions between various social strata in 2019 America. There is a threat of bias of both over- and under-collection of data; for example, over-policing in some areas and under-policing in others. If an entire generation of people are systematically denied home loans or even education, where will that put us as a society?

Another concern is that historical models, which are the bases for many projections and establishing forward-looking traits in algorithm development, have the propensity for “decay.” If a silly tweet from someone’s adolescence has the potential to destroy their career as a young adult, is that the society we want to build?

Another topic discussed by the panel was optimization. Does it come with limitations? What are we optimizing? Profit? Utility? Fairness? According to some attendees, we are never going to “win” by optimizing. It is more important that we ask the right question, and add in other variables in a non-math sense.

Is it possible to easily tweak the existing ethical and legal frameworks to account more fully for data ethics? No, these are thorny, complex problems. For instance, if a social media company changes its business model from freemium to charging a premium for privacy and transparency, then other socioeconomic implications arise. Who can afford to pay? Who is left out? The ethical overlay to this type of business problem is challenging.

Is it possible to completely remove bias? According to Wright, it is not. Instead, a more realistic goal would be to “minimize and understand.” Minimize the amount of bias through thoughtful planning and programming, and then thoroughly understand (and bravely face) the bias that remains and determine with fortitude exactly what should be done about it. It is time for companies and policymakers, as well as academics and end-users themselves, to take a stand on this important issue.

Adopting a Framework for Data Ethics 

The panel generally agreed that a common, single framework for data ethics may not work for all, but adopting a framework, any framework, even a ready-made one, coupled with an ethical corporate mindset and conscientious execution, is more important. It may not be possible to get everything right in one shot; iteration is the key. In addition to those of the panelists, the Alan Turing Institute, Allen Institute for AI, AI Now Institute and the Partnership on AI were some referenced organizations for exploring such frameworks.

For additional information, check out an interview with Heidi Eisips on the INFORMS podcast Resoundingly Human.