Journalism’s purpose is to facilitate understanding. Journalists gather and interpret facts, develop those facts into coherent stories, then present those stories to people who want to understand their world. This activity requires significant resources, including a workforce numbering nearly 50,000 people in the United States and an ecosystem of organizations, processes and professional ethics that shape journalistic work. The journalism community is society’s sense-making apparatus.

Government also has a sense-making apparatus. Like society, government has developed a community whose purpose is to gather facts, develop stories and deliver those stories to people who want to understand the world. This community also deploys significant resources, including a workforce that is considerably larger than the journalism workforce, and it operates in a complex ecosystem of agencies, processes and laws that shape its work. In a government context, we refer to this sense-making activity not as journalism but as intelligence.

The relationship between the journalism and intelligence communities has traditionally been adversarial, focusing on leaks, surveillance and the identity of sources. It has perhaps also been tainted by a failure to distinguish between intelligence and the use of intelligence. This conflict, however, has obscured a remarkable range of similarities between journalism and intelligence as professional practices.

Says John Miller, deputy commissioner of intelligence for the New York City Police Department and a former award-winning reporter and anchor for ABC News, “The work of intelligence officers and reporters is extraordinarily similar.” In addition to sharing a common purpose, both communities also see themselves as skilled artisans engaged in sense-making as a craft. They are both primarily motivated by societal concerns and both hold objective truth to be a sacred value. Differences between journalism and intelligence in areas such as transparency and accountability should not be minimized, but the similarities are such that journalism might find new approaches to shared challenges by engaging with the intelligence community.

The most significant opportunity for productive engagement may be technology. As journalism has been convulsed by the internet, its response has been handicapped by the absence of a research foundation for innovation. Technological innovation in journalism often consists of small projects seeking quick application of popular technical fashions to journalism, or the building of tools for various newsroom tasks. Short-term results are paramount; challenging, risky, deeply researched efforts are not common.

As intelligence becomes more and more like journalism, research programs funded from intelligence budgets become increasingly applicable to journalism.

The intelligence community, in contrast, is exceptionally experienced at sponsoring and managing foundational technical research. It is aided by a budget that exceeds the wildest dreams of the journalism community, and by a long engineering tradition rooted in cold war imagery and signals intelligence. The intelligence community maintains multiple pathways that move innovation from concept to deployment, including agencies like the Defense Advanced Research Projects Agency and the Intelligence Advanced Research Projects Activity, academic institutions like the National Security Agency’s Laboratory for Analytic Sciences at North Carolina State University, and venture capital funds like In-Q-Tel. Their funding has supported many of the tools used every day by developers in newsrooms, including the MongoDB database and key natural language processing technologies.

In recent years the focus of the intelligence community has turned toward areas that are of direct relevance to journalism. Intelligence practices in the 21st century have been revolutionized by two radically new approaches: Open Source Intelligence and Activity Based Intelligence.

OSINT is the collection and valuing of information obtained from publically available sources, and it has quickly become a critical methodology for intelligence organizations. ABI refers to the integration of multiple sources of information to build coherent stories about the activities of specific individuals or organizations, and has proven to be the only viable approach to understanding many of the threats facing modern societies.

The OSINT and ABI revolutions have occurred at a time of unprecedented change in the environments that intelligence services are trying to understand. These are often “gray zone” conflicts in which media and social media are used to deliberately manage popular opinion and influence elections, practices that are sometimes referred to as the weaponization of narrative. This turmoil has occurred during a time in which intelligence analysts, like the rest of us, are swamped by the volume and velocity of information now available to them.

As intelligence becomes more and more like journalism, research programs funded from intelligence budgets become increasingly applicable to journalism. These programs don’t try to produce the quick demos and easy-to-use tools that journalism’s innovation projects often do, but instead explore fundamentally new ways of navigating information and knowledge. In doing so they are building a technical foundation that is directly relevant to journalism but which journalism has been unable to build for itself.

Examples of research programs with potential application to journalism are common. DARPA’s Information Innovation Office funds projects like Causal Exploration, which focuses on models and simulations that provide in-depth understanding of conflicts dominated by complex political, economic, ethnic and religious tensions. The NSA-funded Laboratory for Analytic Sciences is developing OpenKE, an open source knowledge enrichment framework to continually scan a wide range of structured and unstructured data sources for information relevant to a topic. IARPA’s Knowledge Discovery and Dissemination program developed techniques to quickly extract intelligence from new data sources. As with any speculative research endeavor, these programs may or may not deliver new useful technologies. Nonetheless, they are exploring alternative approaches to sense-making that might eventually benefit all organizations tasked with helping people to understand the world — whether in intelligence or in journalism.

Twenty-first century journalism occurs in a technologically mediated and increasingly chaotic media environment. To fulfill its sense-making purpose, the journalism community must find new ways of making the world coherent to its consumers. And it must do this despite being poorly equipped in funding, experience and culture to conduct foundational technical research. Attempts by the journalism community to experiment with new ideas in dedicated innovation labs have largely failed, as evidenced by the closure of The New York Times R&D Lab, BuzzFeed’s Open Lab and many others. Similarly, the real-world impact of major sources of funding for journalism innovation, such as the Knight Foundation’s Prototype Fund, has not been impressive. It may be unreasonable to expect the journalism community to deploy its scarce resources on risky, long-term and highly technical foundational research that it is not well equipped to do. Perhaps a different model of innovation can be developed for journalism, relying more on partnerships with other communities facing similar challenges. If so, then research programs undertaken by the intelligence community might offer a risk-free way for journalism to explore technologies that offer alternative solutions to familiar problems.