When I started my residential fellowship at the Donald W. Reynolds Journalism Institute 10 months ago, I wanted to investigate how 360/VR content creators could better put the audience at the center of our production process so that immersive journalism, and immersive technology in general, may reach its full potential.

One of the biggest obstacles we face as creators is the lack of available qualitative data about audience consumption of immersive content. There are two main reasons for this:

1. Distribution channels only provide limited (quantitative) metrics that are not consistent between platforms.
2. Gathering audience comments, whether online or in person, is resource-intensive and the information acquired is limited, which makes the ROI uncertain.

Combined, these reasons make it difficult for creators to learn from their audience, and then iterate. This in turn hinders the production of better, more engaging immersive content since a) what you don’t measure, you can’t improve and b) qualitative research informs the design process. In addition, as non-fiction storytellers, knowing more about what our audiences perceive, like, and want would help us to stop relying on gut feelings to inform our editorial decisions. I am also convinced that more and better data would greatly contribute to the financial sustainability of emerging media through more efficient monetization.

When I arrived at the University of Missouri, I set up to design a tool that

• facilitates audience research for 360°/VR content

and

• provides evidence-based best practices for immersive storytelling.

Today, I am equal parts thrilled and proud to present the outcome of my fellowship after months of work, research and tapping into the knowledge and experiences of over 70 immersive content creators through interviews, a survey and tests of a low-fidelity mockup of the tool.

These two videos will explain in depth, and in my French accent, my project and its findings.

In the first video, I go into more detail regarding the reasons and ideas guiding my fellowship project, including the results of a survey circulated among a diverse sample of experienced VR/360° content creators.

Here’s how and why we can improve 360/VR journalism

In the second video, you can learn more about how the tool’s design can help immersive journalists and 360 content creators deliver better experiences for their audiences.

Create better VR/360 experiences by listening to your audience

In addition to the videos, here is a summary of what I found:

• There is a hunger for guidance and best practices for immersive storytelling, no matter the field (non-fiction or other).
• Gathering useful, actionable qualitative feedback is difficult for several reasons (technical or otherwise).
• 90% of those surveyed say they want a tool to help them test content and conduct research on their audience to gather qualitative feedback. (The proportion is actually higher among those who already have a process in place).
• 75% are ready to use the tool very frequently (“often” or “almost always”) in their production workflow.
• If it means creating better content, nearly every respondent said they’re ready to dedicate more resources (up to several days) to testing content, conducting audience research and gathering feedback.
• While the tests showed the low-fidelity mockup is [of course] perfectible, 100% of the testers — all professionals in the immersive field — say it is a “good” tool to get user feedback.
• Almost none are aware of any similar tools,
• Testers overwhelmingly appreciated having the results of the user research gathered and presented visually on one clear report page.

Throughout my project, I was encouraged to see my work parallel ongoing audience-centric shifts in the journalism industry, through an increased focus on the needs and behaviors of ‘end users’ (i.e. the audience), engaged journalism, public-powered models, events around conversations with readers/viewers and audience-first strategies.

No innovation happens in a vacuum and I hope these insights will benefit all forms of reporting, including immersive journalism.

What’s next?

The objective of my fellowship was not to deliver a working product by the end of the year, but rather to explore out-of-the-box ideas and concepts to improve immersive storytelling and its monetization. Building a working tool while at RJI would have required coding expertise, and more time and resources than the already generous amount I had at my disposal.

However, I now have collected enough data points and have seen enough interest for such a tool in the immersive journalism community, that I plan to take the project to the next level. Together with Vragments, a Berlin-based VR studio and my technical partners throughout the project, we are drawing up roadmaps and exploring options to gather market research and garner investors’ interest to pave the way for the development of the tool.

Knowing more about our audience is key. It’s key for our reporting, and it’s key for the future of immersive journalism, and of immersive technologies in general. If you are interested in 360/VR, and you would like to know more or want to help us/contribute and make this tool a reality, please reach out to me via Twitter or email at myfirstname dot mylastname at gmail dot com. I am looking forward to working with all of you who want to make immersive storytelling better.

Thanks!

Wait, wait, there’s more: a deeper dive

If you have made it this far, thank you so much! Be aware though that the sections that follow are going to get a bit more technical. In addition to the videos, I would like to share with the industry some more detailed findings and data points that I have uncovered during my project, which either validated assumptions or guided me towards a better final concept. I want to share these not only for the sake of transparency but also because I sincerely believe they can point the immersive community in the direction we should go to provide better, more engaging experiences.

Let’s go through them step by step:

The interviews

First, to get a better sense of the problem I hoped to tackle and to gather meaningful information, I drew inspiration from design thinking and its first step ‘Empathize’. I started my project by conducting 16 in-depth interviews of immersive content creators. Each semi-structured discussion lasted about one hour, some took place online and some face-to-face, in Missouri or during ONA 18 in Austin. In total, I spoke to nine men and seven women, hailing from five countries on three continents, involved in a broad range of roles in the field: immersive journalists, innovation leaders, independent content creators, CEOs, and educators.

I was particularly interested to hear about their challenges, how they informed their editorial decisions and what guided their production processes. Despite the diversity of profiles, many lamented the difficulties they often face to get good qualitative feedback: from editors, from clients or from the audience. They identified various causes: lack of resources (time, especially), technical difficulties, working remotely, or varying level of literacy when it comes to the medium. In addition, for some, the editorial and business propositions of 360/VR were an issue: what’s the effect of 360/VR on the overall quality of the news produced, and what is its impact on the bottom line of the company?

When prompted about the kind of questions they would like to ask their audience, the interviewees shared dozens of interrogations and topics they would like to explore further. Drawing inspiration from existing research, I sorted these into five categories: questions about a) the flow of the piece, b) transmitting information or a message, c) presence and immersion, d) the enjoyment of the audience, and e) creating an impact.

Last but not least, interviewees provided numerous feature requests and suggestions for their vision of an ideal audience-research or usability testing tool. These requests could also be arranged in categories: front end/User Experience, back end, what the output should look like internally, and what it may look like publicly. These insights proved very useful for the rest of my project.

The survey

With the help of the University of Missouri’s Information Experience Lab, experts in usability testing, I drafted a survey and circulated it on industry Facebook groups, social media as well as newsletters/mailing lists. Over a couple months, it was answered by a decent sample (n=46) of immersive content producers. Before we go deeper into the results, I am happy to share that, in the interest of openness and radical transparency, I am making the whole survey dataset public. The public version has been carefully anonymized (by deleting open-ended questions which may have helped identify participants) and some categories have been collapsed for clarity’s sake.

As mentioned earlier, the most significant result of the survey is that 90% of the professionals surveyed want a tool to help them test content and conduct research on their audience to gather qualitative feedback. This proportion actually is higher among those who already have a process in place. This tool has the potential to become a central aspect of their respective immersive production workflow, across domains: 75% of respondents are ready to use the tool very frequently (“often” or “almost always”), and the share goes up to 98% if those planning to use it “sometimes” are included. Also, if it meant creating better content, nearly every respondent say they are ready to invest more resources (in the case, working days) to test content, conduct audience research and gather feedback. As a matter of fact, the respondents who said they can’t afford to spend more time on these tasks are only in the single digits.

These key results seem to indicate that:

• There is a real need for a tool to test content, conduct audience research and gather qualitative feedback.
• Such a tool has the potential to become a central part of the workflow of immersive content producers.
• The current low rate of content testing is due to a lack of clear ROI rather than a paralyzing lack of resources.

Of course these 46 professionals do not represent the whole industry. Yet their profiles makes them ideal prospective users of the tool I was designing, i.e. an experienced group with diverse profiles and working conditions. Indeed, about two thirds of these surveyed immersive professionals work in non-fiction 360°/VR (journalism, documentaries and education); in total 4 out of 5 have been in the industry for over a year. In addition, two out of three work in small teams (2 to 6 people, including the respondent) and the time spent (in working days) to produce an immersive experience was evenly split: one third worked less than a week, one third between a week and a month, and one third over a month.

In addition, it was illuminating to find that, despite their experiences, most of the respondents admit they have a lot to learn:

• 41% say they either agree or strongly agree that they know what their audience likes; a mere 30% know what their audience wants.
• They want to improve their knowledge though: when asked whether they want to know more about these likes and wants, almost all (97%) answered positively (either agree or strongly agree).
• This shows the existing hunger for guidance and best practices, especially since over a third (36%) of the respondents explain they already have a process in place to test content, conduct audience research and gather qualitative feedback. Clearly, these existing processes have yet to answer all of their questions.

It was also interesting to find out what aspects of immersive storytelling respondents consider important. To gauge this, I used the same categories as the ones created (see above) to classify the questions the interviewees were presented with.

• The flow of the piece, transmitting information or a message, presence and immersion, and creating an impact scored close to 90% or over of respondents either agreeing or strongly agreeing they are important.
• Enjoyment of the audience was not prioritized as much. Close to 20% of respondents said they strongly disagreed, disagreed or were undecided on its importance. This is perhaps due to the large number of non-fiction immersive storytellers among the respondents, for whom entertainment may not be a central value in their work.

Using the same categories, we can also see what aspect of immersive storytelling the respondents are most interested to learn more about  — by asking their audience. Over 94% of respondents answered positively (a combination of possibly, very probably and definitely) about asking audience questions on transmitting information or a message, presence and immersion, the enjoyment of the audience, and creating an impact. This clearly indicates that the hunger for best practices and guidance isn’t limited to one single side of the trade, but is rather across the board. The ‘flow of the piece’ category scored slightly lower: 15% said they would definitely not or probably not ask questions about it. This result may not be significant and might be due to the relative expertise storytellers think they have in this domain.

Building and testing a mockup of the tool

I worked with Berlin-based XR studio Vragments to build a low-fidelity prototype/mockup of the tool, which integrates a lot of the data points, feature requests, and design ideas that emerged from my research. The goal of this WebVR-based mockup is to replicate a user’s journey as they act as the tester of an immersive experience. The story is regularly interrupted by prompts with various feedback modules (yes/no questions, Likert scales, etc.) users can answer (even if some are non-functional, like the voice-to-text, in the mockup).

I wanted to put this mockup in the hands of immersive content producers and see what they thought when testing an audience-research and content testing tool. So I commissioned the University of Missouri’s Information Experience Lab to conduct an onsite and remote user experience study for the mockup. It was conducted by the IE Lab’s usability experts, lead by PhD candidate Gayathri Sadanal and under the guidance of Dr Isa Jahnke. A total of 12 participants – all professionals in the immersive field – were recruited for the study: 6 men and 6 women, from 4 different countries on 3 different continents, with diverse roles and level of experience (from 6 to 100+ months) in the industry. The average length of a testing session was 54 minutes.

The report resulting from this study produced numerous valuable insights. Many participants’ remarks focus on the in-experience navigation, the design of the feedback modules, their location, and other points of the UX and UI. These comments point out several clear areas for improvement, which was expected since the mockup is a low-fidelity prototype. This information is nonetheless extremely helpful, and will be put to good use when Vragments and I develop a working beta version of the tool. On the concept itself, 100% of the testers really like the idea of a tool like ours to get users feedback. As one commenter put it: “I really liked the tool, it was great to see something like this, I look forward to its release.” Participants also like the design of the mockup report page — where responses from the audience are presented visually and can be explored in more detail.

“It is very clean and good overview, if I click on it tells me more that’s good, it seems very intuitive to me as a user,” explained one of the participants.

Overall, these results validate many assumptions and design choices made at different stages throughout my fellowship. It’s also encouraging that almost none of the participants know of such a tool for immersive content. Only one of them said that they had worked before with a tool where we can annotate videos (like frame.io). Combined with the survey results, this study clearly strengthens the case that there is a need, a desire AND a space for an audience-research tool dedicated to immersive content, and that the mockup I produced during my fellowship is on the right track.

The collaboration with the IE Lab will bear results beyond my fellowship as well. This was the first time the IE Lab team tackled a study involving 360/VR content in an immersive environment and they had to adapt some tools and methods for this groundbreaking research. So this news experience serves their scientific community, the IE Lab team is looking to publish the results and learnings in a peer-reviewed journal and I have pledged to help them to the best of my ability.

On a final note

I would like to express my deep gratitude to all the people who made this year such an intellectually-stimulating, horizon-expending venture. Many thanks to the RJI team, especially my supervisor Mike McKean as well as Randy Picht for their trust and giving me the resources to bring my project to fruition. Thank you as well to the members of the Missouri School of Journalism and the Futures Lab faculty who I had the opportunity to meet and pick their brains: Ebony Reed, Damon Kiesow, Jim Flink and many more. I must also thank Johannah Grenaway, Nnamdi Egwuonwu, Yanran Huang and Ran Huan, Missouri Journalism School students who helped me with my project as part of their senior Capstone. I am grateful as well for the other RJI Fellows, in particular Nico Gendron, for being sources of inspiration, and for the Journalism360 family (hi Laura!), and the immersive journalism community in general, for their continued support and shining spotlights on my work.

Finally, I am forever thankful for the unconditional love and support of my family as I uprooted my life, and theirs, to Missouri — some 7,500 kilometers away from home — for this fellowship. I could not have done it without you.