Optimizing content limits on a user-by-user basis

Should publishers communicate content limits to their audiences? The question becomes more pressing as the products that manage paywalls become sophisticated enough to make predictions about who likely subscribers are. We decided to test this idea with the Reynolds Journalism Institute this year in a partnership project where we are experimenting with an optimized wall.

At the Bangor Daily News, a 132-year-old news media publisher covering most of the state of Maine, we implemented our content meter in early 2018, granting visitors eight free pageviews per month at first, and later reducing that number to five and granting increased access on registration. For the past six months or so, we have granted a strict five free pages per user.

At first, our content meter communicated frequently when users were approaching their content limits. With time, we realized that an optimized wall would vary according to what visitors were reading, what devices they were using, how they arrived on the site, and other statistical shorthands for propensity to subscribe.

In other words, an optimized wall should not guarantee a certain number of page views per a period, but weight the number of page views based on the type of content and other factors unique to that individual reader.

Why does a subscription propensity model preclude firm metering practices?

Let’s imagine a news website with three content categories, A, B, and C. (By “content categories,” we mean the traditional newspaper taxonomic entities – these could mean sports, local news, national news, and so forth.)

Sitewide, the publisher makes 50% of its revenue from advertisements and 50% from subscriptions – but with significant variances among content categories:

• Category A: 50% subscriptions, 50% ad revenue
• Category B: 75% subscriptions, 25% ad revenue
• Category C: 25% subscriptions, 75% ad revenue

So, a better content meter for that publisher would weight pages accordingly:

• 1 page in content category A is worth 1 pageview
• 1 page in content category B is worth 1.5 pageviews
• 1 page in content category C is worth .5 pageviews

If the publisher’s meter was set at four (notional) pageviews, an optimized meter might allow for any one of these experiences to reach the paywall:

1. Page A1 (1) + page A2 (1) + page A3 (1) + page A4 (1) = WALL
2. Page B1 (1.5) + page A1 (1) + page B2 (1.5) = WALL
3. Page C1 (.5) + page C2 (.5) + page C3 (.5) + page C4 (.5) + page C5 (.5) + page C6 (.5) + page C7 (.5) + pageC8 (.5) = WALL
4. Page A1 (1) + page B1 (1.5) + page C1 (.5) + page B2 (1.5) = WALL

In experience 2, the publisher is likely to maximize revenue because the reader hits the paywall faster, prompting them to subscribe earlier. In experience 3, the reader is less likely to subscribe, and so the publisher maximizes revenue by giving the reader as much room to accrue ad impressions as possible while still preventing permanent free-riding.

Content type will not always be the most significant factor in propensity to subscribe, so to maximize revenue from ads and subscriptions at the same time, publishers can use other propensity data (implicitly in the form of revenue mix ratios), basing their model on device, acquisition channel, and other dimensions to create as fine-grained a system as their technical capabilities allow. Using a propensity model with an optimized content meter, a pageview isn’t a pageview, but a variably weighted quantity of what the reader can access on your news site.

Even for a rudimentary propensity model like the one outlined above, publishers might ask: How should we communicate our meter to visitors if we don’t have all the relevant data about how that user will behave over time yet, and can’t say how many pages the user will be allotted?

On the first pageview in any session, publishers don’t know what link the visitor will click on next, or what their second article will be (if there is one), so an optimized meter precludes a set pageview budget in at least some cases.

Using our content meter provider Piano’s tools, we set up an experiment to test three models of content meter communication against each other to address this problem:

1. Forward Count
   This method is the traditional content meter communication method (“You have read 1 out of 5 stories for the month,” and similar). This method may induce budgeting behavior (a reader thinking, “I’ve read one story, so now as long as I only read 4 more over the next 30 days, I still don’t need to subscribe”). This was our control – it’s how we’ve always managed our messaging.
    

1. Backward Count
   This method communicates the number of articles already read, placing an implicit emphasis on the reader’s relationship with the publisher (“You have read 4 stories so far this month,” and similar.) This method stresses the reader’s affinity with the publisher without communicating a fixed content limit.
    

1. No Count
   In the experiment, we used this method to emphasize the value of local journalism without discussing reader consumption or communicating when the reader will hit a paywall. We were curious about how avoiding discussion of either prior or remaining pageviews would impact conversions and revenue.

In our meter setup, at the end of each subscription experience, users can either click through to a subscribe page that includes annual and monthly options, or they can purchase an annual subscription directly from wall messaging.

Over a 23-day testing period, we logged several million impressions in total across the three sets of messaging.

Revenue outcomes

In terms of cumulative revenue earned by messaging type, No Count came out on top, out-earning Backward Count messaging by 57% and Forward Count messaging by 41%. Piano calculates that there is a 79.31% Bayesian probability that No Count messaging will earn more subscription revenue than the other two, and it’s very unlikely to lose money compared to the other two messaging strategies we tested.
 

Conversion outcomes

Each messaging style was displayed to users in roughly similar numbers, but in terms of raw conversions, the three messaging styles performed roughly in line with each other – there was less difference between the three options in conversions than in revenue, but No Count was still the winner, exceeding Backward Count’s conversion total by 10.8% and Forward Count by 18.6%.

The Bayesian probability of No Count converting more users than either of the other two messaging strategies over a longer period of time is only 53.05%, which is not bad, but hardly a lock. 

Where did site users convert?

Each potential subscriber saw a set sequence of four messages with a Subscribe button (leading directly to annual subscription), but each page that prompted a meter message also had a Subscribe button outside of the modal window that allowed for purchase of a monthly subscription (or print, purchases of which we did not gather data on through Piano).

For the No Count messaging strategy, 13.7% of all subscribers subscribed before hitting the paywall beyond which they couldn’t read anymore, while for Backward Count, only 4.3% did, and for Forward Count, 7% did. No Count may help move conversions to earlier in the user’s journey through the site, but this result is well within the margin of error, so more data is needed.

What does it mean?

One of the reasons we undertook this multivariate test was to answer the question, “What happens if we couldn’t communicate content limits anymore?” Will no longer clearly messaging pageview limits hurt us?

For one thing, we found that on average, across all three experiences, hitting the wall was itself the single most important factor in inducing users to subscribe (though, as mentioned, it was a little less important when we didn’t signal when that would happen).

We think there’s some evidence that even if it generates a similar number of conversions, values statements that implicitly communicate a paywall (while precluding content budgeting) spurred a higher average spend per subscriber than allowing users to budget, or emphasizing how many pages they’ve already read.

Visitors who saw No Count messaging appeared more likely to purchase annual subscriptions than monthly subscriptions. Retention is a separate question, but this cohort’s average spend was 39% higher than visitors who saw Backward Count and 15% higher than visitors who saw Forward Count.

Every market is different, but we don’t think local news publishers who are considering a move to a more flexible content meter should be afraid of putting their revenue at risk if they stop telling site visitors what their content allotment is. In our case, not communicating past pageviews or allowing users to set a content consumption budget seems to drive up subscriber Average Revenue Per User (ARPU), even if it doesn’t profoundly affect likelihood to convert by itself.

So if not messaging a content limit doesn’t lose revenue, then we are free to optimize content limits on a user-by-user basis, without concern that the messaging strategies required in doing so will cost more money than they make us.

With this conclusion in mind, over the coming months we plan to implement and optimize a metering model that weights pageviews using revenue metrics. We’re excited to undertake this project with support from RJI, and we look forward to sharing our findings once we’ve implemented the required technology and gathered data about outcomes.

Conrad Lumm is the product marketing director at the Bangor Daily News.