Debate analyzer

One more time with feeling: A new way to look at the presidential debates

With apologies to gone-viral Ken Bone, most Americans have decided who they’ll be voting for on Nov. 8. Some have already cast their vote. Therefore, this year’s three presidential debates may be more for show than substance.

It’s probably also fair to say that most folks who watched the first two debates viewed the responses and repartee through their own filter bubble, which is really what happens every day as people turn to their Facebook friends and Twitter followers for news.

The filter-bubble phenomenon — only getting news and information from people who agree with you and think like you — is becoming a bigger and bigger challenge for journalism and democracy.

We decided to try a little experiment connected to the presidential debates. Since no commentator or news organization is capable of piercing “the other’s” filter bubble and no amount of fact checking or timelines or nonpartisan studies may change that, we wondered whether bringing a cold, hard computer into the fray might make a difference.

Dale Musser, chief technology adviser for the Donald W. Reynolds Journalism Institute, built a debate analyzer application that was quickly able to deliver the text of the recent debates — and some historic ones too — to IBM Watson’s tone analysis tool and provide the results for us to review.

IBM Watson won’t be voting in the upcoming election and really doesn’t care about Supreme Court justices or deleted emails or hot mic comments. It simply looks at the words in a file and assigns a rating on an emotional scale.

Many of the results are predictable, which is probably a good sign for Watson.

What we’ve discovered so far:

  • There is very little joy in this year’s debates and a whole lot of sadness.
  • Scores tended to be high throughout for “agreeableness” and “analytical.” That sounds about right since the debate format encourages that kind of phrasing.
  • IBM Watson doesn’t get sarcasm.
  • Comparing the first TV debate — Kennedy-Nixon in 1960 — with the most recent debates, you can definitely see a dramatic decline in the statesmanship of our politicians. (Interruptions data)
  • Watson, at least with this tool, doesn’t evaluate voice and facial expressions. Layering voice and facial expressions analysis on top of the Tone Analyzer could add significant insights.

Reviewing the Watson analysis is somewhat similar to wine tasting. For example:

In the second Trump-Clinton debate, when the question of the Access Hollywood hot mic tape came up, Trump was confident and agreeable, with hints of sadness and anger. Clinton was analytical and agreeable with hints of confidence and disgust.

When the question of the deleted emails was raised, Clinton acknowledged that using a private internet server was a mistake and delivered her response in a very calm and confident manner.  Watson scored her response as mostly analytical and agreeable but with hints of disgust and confidence. Trump’s counter was also analytical and agreeable but with hints of anger and disgust.

The only real moments of joy during the second debate came when Clinton had the opportunity to talk about her accomplishments over her 30-plus years of public service and when Trump grudgingly accepted Clinton’s compliments about his children — “I don’t know if it was meant to be a compliment, but it is a great.” — and praised Clinton for being a fighter.

Another emotion that didn’t show up often was tentativeness, but it was interesting when it did. Clinton had higher tentative scores when answering a question about leaked comments about having two positions on issues — public and private — which makes sense, but she also received a high tentative score for comments about choosing the next Supreme Court justice.

Trump had the highest tentative score when he was asked to disclose how many years he had gone without paying personal federal income tax, which also makes sense, but he had high tentative scores when talking about making changes to the Affordable Care Act.

If you want to review, the ratings yourself for the Trump-Clinton debates and some others thrown in for fun, go here.  We’ll employ the Watson “emotion detector” for tomorrow night’s third and final presidential debate, as well.

Special thanks to CQRollCall for providing the debate transcripts we’re using in this experiment. Stay tuned as we refine the tool and look for other ways to improve journalism with machine learning.


Comments are closed.