An automation and AI experiment in real time

Newsrooms across the country have been facing a daunting task when it comes to election coverage. 

In an effort to inform readers about who and what is on the ballot — from the presidential race all the way down to state legislative contests and county or city council positions — journalists are wading through hundreds of names, office titles, descriptions of measures and propositions. And that’s just to provide previews of the ballots, county by county. Wait until election day when they have to add vote tallies to each race and measure in real time as polling places close and votes get counted and recorded. It’s a huge task — and one that is critical to democracy.

Right now, most of this work is laboriously done by hand as information is either typed in or cut and pasted from PDFs and websites into a more user-friendly format. But what if AI chatbots and other digital tools allowed us to automate this work for the previews and election-night results? How might we put our limited reporting and editing resources to better use by cutting out the manual labor and instead have more time to make sense of the results to inform, empower and engage our audiences? That would be a game changer for every local newsroom struggling to find enough time, people, money and technical expertise — especially as news staffs continue to shrink amid dwindling revenue and budget cuts.

Bay City News, which is keen to experiment with new technologies, set out to ask and answer these questions with support from the Reynolds Journalism Institute, using the upcoming election as a real-time experiment. So far, what we are discovering is truly revolutionary. We will be able to give consumers accurate information in a much more efficient, comprehensive way, directly through our free, public service site LocalNewsMatters.org and via news partners who are part of our affiliated Bay City News, a B2B news service with dozens of media partners in the region. 

Here’s a summary of our project, which can be seen in two places: our “Civic Engagement Hub” and our “Experiments in AI” sandbox.

Problem

A gigantic ballot for the Nov. 5, 2024 election presents a monumental challenge for Bay City News (which covers 13 counties in the greater San Francisco Bay Area and Northern California) and other news outlets trying to cover key local races. Specifically, we need to:

• preview and explain hundreds of city- and county-level contests and ballot measures for ourselves, our partners and readers; 
• monitor and report election night returns in real time;
• accurately summarize results and write follow-up stories based on interviews and analysis. 

For election previews, the work up until this year was done by hand. The same goes for manually tabulating results but even more stressful because of the time pressure. It requires a huge lift by already strapped staff who could be more usefully assigned to report and write meaningful stories and analysis. It requires significant extra spending to pay for stringers and overtime for news organizations like ours with tight budgets. It can lead to human data-input errors and delays in getting information out. And it fails to serve readers with timely and accurate results in a searchable format. 

We typically go to county elections’ websites and scour the PDFs that list all the races and measures. We move that information bit by bit into a simple, straightforward list for our Bay City News wire service media partners at TV, radio, print and digital outlets, who are doing their own election coverage. We also present the local races and measures in a public-facing dashboard on our nonprofit site as a public service.

We follow a similar workflow on election night, updating results as batches of results come in sporadically, and tallying and updating final/certified results after close of polling. We generally attach a journalist to each county to update the results and write the stories — but the sheer volume of content is barely manageable for the editors who are fact-checking, copy editing and doing quality control before publication.

Solution

We split our project into three parts:

1. Use data scraping or enhanced AI to pull from each county’s ballot PDFs and move that data into a spreadsheet. We wanted to select only the key fields such as name, occupation and office for the races and only the measure title and description for the ballot propositions. It took some trial and error to produce perfect results that did not add unnecessary information or did not drop certain fields or mangle names with accents. 
2. Apply code created by ChatGPT or Claude.ai to the spreadsheet data in order to pull it into a user-friendly dashboard on our Civic Engagement Hub so that readers can easily find — through scrolling or search — the races and measures that matter to them. A few different people reviewed all the data at each stage to be sure it was right.
3. Write a program to gather real time results as reported by each county on election night to auto-populate the templates and avoid manual data entry. One challenge is that each county presents their ballot information in a different way so the data fields are not uniform and they don’t make their test sites live until very close to the election.

Looking ahead

Depending on the formatting and process used by each county, this project could scale to be used in all 58 counties in California for the next election. Based on learnings about the use of data scraping and AI here, it could scale for national use. We’re already well on our way to improving our process thanks to the AI tools we have used so far. The next test will be Nov. 5 — a high-stakes day for all of us.

---

Cite this article

Rowlands, Katherine Ann (2024, Oct. 29). Can election season be less crazy? Reynolds Journalism Institute. Retrieved from: https://rjionline.org/news/can-election-season-be-less-crazy/