Let the Robots Debate

I previously wrote an article on stump speeches for political party candidates using a large language model (LLM) such as OpenAI’s ChatGPT. The first round of the 2024 U.S. presidential debates made me wonder if artificial intelligence (AI) might give us a more respectable transcript, so I asked. It basically gave a very concise debate dialogue highlighting candidate differences on the economy, healthcare, climate change and immigration. Truthfully, the output did not at all capture the personalities of the two presidents.

Another approach was to ask the LLM for stump speeches, which produced a slightly more personality-driven response, including themes from President Biden’s past speeches, focusing on unity, progress and addressing key issues like healthcare, education and climate change. And for former President Donald Trump, AI emphasized themes of American greatness, economic prosperity, strong borders, military strength and critiques of political opponents. Again, and arguably, AI produces an overall more respectable tone.

I do not suspect the candidates’ preparation included AI-generated responses to questions. However, I downloaded the lengthy debate transcript and decided to spend just a few minutes analyzing the content. Similar to when I looked at the AI stump speeches, I generated word clouds and examined word counts that were underneath the results.

Biden’s word cloud calls out “people” and “idea” as repeated words, 38 and 35 times, respectively. He also uses the contraction “we’re” the most, at 48 times. “World” is mentioned 25 times. Trump also uses “people” often, followed by “country,” 71 and 49 times, respectively. He uses the contraction “he’s” 49 times as well, which upon inspection, is often in a critique of his opponent. “We’re” is used 31 times and “world” 19 times.

It is not surprising to see many of the same words used by both candidates in a debate format because they are both responding to the same or similar questions. Thus, I wanted to see if I could get some separation based on sentiment analysis. GenAI suggested using the “syuzhet” package in R, and within that, using the NRC Emotion Lexicon. Table 1 shows the results, which overall look very similar. Biden shows a slight lean toward positive over negative language, whereas Trump’s positive and negative scores are almost equal. Trump also has a higher score for fear and Biden for trust. But overall, these appear to be fairly small differences.

Disclaimer: I have no intention here of weighing in with my own personal political opinion; I only write this to provide a thought exercise on the intersection of AI and bias with the upcoming election as a backdrop. 

What did you think of the debate? Let’s start another. Do AI and the analysis tools used here give the results you would expect? Presumably, GenAI models have a much wider range and depth of content to draw upon than any single person. Are the current tools more or less biased than individual citizens?

Update: Since writing this column, Biden has dropped out and Kamala Harris has jumped in as the Democratic candidate. Trump and Harris have agreed to a debate. I would encourage an analytical study of the language used and the sentiment they evoke. Perhaps there will be a greater degree of separation this time? Maybe?