The opposite of relative rankings, which I wrote about earlier this week, is evaluating performance against a baseline.

Of course, there is a bit more work in baselines because baselines have to be thoughtfully designed. And thoughtfully designed really covers two different exercises: 

1. The factors we consider in building an appropriate baseline

• The type of article (news, non-news) or even the topic/section of publication (technology vs. dining).

• Factorizing what kind of external promotion the article received and measuring performance relative to the common distribution of promoted articles.

• Factorizing what kind of on-platform promotion the article received: roughly speaking, how much homepage time and in what position relative to your usual scroll-depth? Homepage promotion “downpage” isn’t at all the same as being ranked in a prized spotlight position that’s reliably in view of all homepage pageviews.

2. Paying attention to the standard distribution of your articles against this baseline

You usually would think of this as relative percentiles. This is particularly important when you consider that depending on the type of content you measure, you may find, for example, that most articles tend to perform within a small spread of performance level — whereas for other baselines, content may be more evenly spread across the scale. 

Say you’re a general news publisher with a strong Facebook presence, and you promote five articles via your page per day. Your mix of Facebook stuff is generally trending toward lifestyle content (which does better on Facebook), but there’s occasionally a more news-related item in there.

The lifestyle content tends to perform across a smaller performance range: Basically, everything tends to perform well, and outliers (blockbusters and utter crashes) are rare. Your social teams know what they are doing.

With news, interest can have a lot of variation of interest. Think about the early days of COVID or the death of Queen Elizabeth II — high peaks of interests. So reporting on the performance of an article on Facebook should have its own baselines with a factor looking at the profile of the article to further subset that baseline. 

Then, the referring platform itself — Facebook, here — can have its own baselines or become the baseline for another calculation. That’s what Axel Springer does with their alerts that inform the newsroom about certain traffic patterns. 

“We push alerts when we see articles coming strong from Google Discover but with low social,” Janis Kitzhofer, the head of editorial analysis at Axel Springer in Germany, told me. 

The calculation here is that if an article is able to get a certain amount of traffic from Google, it should be able to get a proportionate amount of traffic from social — and that proportion may be different depending on the type of content or the organisation. In this case, the baseline is actually one trend line of performance (Google). 

You can think of it as “articles that get at least N PVs for Google” — and calculate other baselines like “number of PVs from social.” So an article may find itself in the group of Google Discover high performances, and then its performance across other dimensions is evaluated against baselines for this group of Google high performers.

Think of baselines as being why you have sports league. Is it that interesting to only force rank item? Usain Bolt is always going to be running at the front of any race — but that doesn’t mean there is not a lot of potential in a kid who is running laps around their local park.

Baselines are about comparing things on a realistic basis of homogeneous items. And, if used properly, they should help us identify the undiscovered supermodels in our midst.

If you’d like to subscribe to my bi-weekly newsletter, INMA members can do so here.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .