Big Local News and the Reynolds Journalism Institute will make Agenda Watch a full-featured platform for journalists, one that provides the ability to research past agendas and minutes as well as subscribe to alerts based on topics and keywords relevant to a reporter’s beat

Keeping tabs on community decision makers is an essential part of a local reporter’s job. Many of us have published stories about school board meetings that stretched deep into the night, or angry residents denouncing the latest housing project before a planning and zoning commission. We’ve unearthed important contracts and controversial new ordinances tucked away in the consent calendars of city council agendas.

Unfortunately, as newsrooms shrink and news deserts expand, a growing number of decisions that affect the local community are made in the dark.

In our new partnership project, Big Local News and the Reynolds Journalism Institute will make Agenda Watch a full-featured platform for journalists, one that provides the ability to research past agendas and minutes as well as subscribe to alerts based on topics and keywords relevant to a reporter’s beat. 

We hope the platform will serve as an early-warning system when a local agency is preparing to debate affordable housing, environmental cleanup, noise ordinances, and myriad other issues. We will also make our growing corpus of documents available to journalists hoping to conduct more in-depth research and investigative projects. Our goal is to help reporters more easily keep tabs on the issues that matter most to their communities.

We’re also looking to scale nationally by expanding an open source web scraping library that targets software commonly used by agencies to publish agendas and minutes.

How we started

It all began in a computational journalism class at Stanford University several years ago. Students in the course created a basic prototype that enabled reporters to receive email alerts when a topic of interest cropped up in the agendas of more than a dozen Bay Area cities.

The mission of Agenda Watch, a project of Big Local News, has always been to help journalists more easily surface stories from public meeting agendas. These documents must be published before a local board or commission convenes, and they can be a treasure trove of news stories for a reporter who can access them and translate their bureaucratic jargon.

The big picture

In the coming years, we expect to gather millions of documents from thousands of agencies across the country. And we’re hoping that journalists, researchers, and members of the public will help us find novel uses for those documents.

Our partners in news and academia have already suggested a variety of interesting computational projects, including:

• Extracting structured data about government contracts
• Enriching the platform’s search by annotating documents with topics and identifying mentions in the corpus of people, places and organizations. 
• Using natural language queries (as opposed to traditional keyword-based searches) to locate relevant documents. For example, we’re hoping users will not only be able to craft searches such as “affordable housing AND Alameda County,” but that they’ll be able to search using more natural questions or statements such as “Show me all the documents from Alameda County that mention affordable housing.”
• Generating the first draft of breaking news stories using natural language generation techniques
• Detecting news in video recordings or live streams of public meetings

We’re very excited about these types of experiments, both for their potential to improve our platform and to support traditional watchdog reporting. 

What’s next

For now, the immediate future for Agenda Watch is building out our platform. We’re looking to begin work on the design and implementation asap. Step one involves contracting with a web developer (or team) who can help execute our vision. 

Interested? See the job description and how to apply here.

While we do that, we’re also writing more web scrapers. We’re planning to expand in Georgia in collaboration with The Associated Press and Atlanta Journal-Constitution, and we’re looking to cover Philadelphia and a number of other communities in Pennsylvania in partnership with The Data Desk at Temple University,  The Philadelphia Inquirer and other local news organizations.

As we embark on this process, we welcome ideas on how we can create the most useful platform possible. Hit us up on @BigLocalNews or email with your feedback or thoughts. We’d love to hear them!