What’s Really the Deal with Facebook Metrics Glitches?

facebook metrics2

During the course of the last quarter of 2016, there were about three cases of glitches in the Facebook metrics. This elicited varied reactions (even a lawsuit) from the advertisers and publishers of the social media giant. Facebook, gave reassurances that these glitches only involve a handful of advertisers and that the problem metrics weren’t billable.

 

But is there really nothing to worry about?

 

Facebook Metrics Glitch: Three Cases of Miscalculation

Case 1: Video Ads View Time Inflation

Facebook revealed the problem on this metric in September 2016. Facebook disclosed this information through a post in their “Advertiser Help Center”, saying that the glitch is present for 2 years already. The glitch caused calculations for average time on video to “artificially inflate”. Facebook only counts video views that exceed 3 seconds, thereby inflating the average view time. According to Publicis Media, Facebook admitted that this metric measurement method caused them to overestimate video views by 60-80%. The formula for “Average Duration of Video Viewed” metric is supposed to be the total time on video divided by the total number of playbacks.

 

The hiccup in video views metric did not affect billing – Facebook assured publishers and advertisers of that. But still, it affected the overall ad spend. A video ad campaign may appear high-performing, or a branded video appear more engaging than it really is. Publicis Media, for example, purchased an estimate of $77 billion worth of ads for their clients in 2015. So we aren’t actually talking about individual publishers and advertisers but of a collective count of ad agencies that invested client money on ads that may or may not have furthered their marketing campaigns.

 

Case 2: Miscalculation Bundle

Facebook could’ve been busy checking out their metrics after September. November saw them addressing several more issues with different metrics.

  1. Page Insights Bug

Facebook first reported on a bug that affected all pages on October 4-14, 2016. This bug prevented Facebook from reporting any organic engagement on Page Insights. But there was another bug a month after. This time, it’s causing a miscalculation of the organic page reach in a 7-day or 28-day period reach summary. The calculation produced a simple sum of the page’s daily organic reach instead of excluding repeat visitors, inflating the number. Other reports and summaries on the Page Insights dashboard remained unaffected including the ones for paid reach. The bug was apparently alive and kicking since May, causing consequent 7-day reach summary to decrease by about 33% and the 28-day reach summary by 55%

Face Metrics Page Insights Organic Reach Summary

The Page Insights bugs at first looked like a simple glitch. But when you take into account how Facebook actually defines reach, you’ll see that there’s certainly room for improvement. For one, organic reach and paid reach have different definitions. Organic reach is when an organic post appears every time a person refreshes their News Feed. Paid reach, on the other hand, follows strict counts based only on “viewable impressions”. Viewable impressions happen when a post enters a person’s screen.

  1. Video Completion Miscalculation

Facebook automatically detects the length of the video upon upload. The problem is that due to the difference in video players and device specifications, some videos can become longer or shorter for a fraction of a second. This is often due to syncing problems between the audio and video track. Moat, a third-party analytics firm brought this discrepancy to Facebook’s attention. The discrepancy caused an undercount in the “Video Views to 100%” metric.

 

The rate of completion is important for advertisers. This audience action determines the success of the delivery of the video’s message. But if the discrepancy is only at the last split second, it may not matter much.

 

  1. Time on Instant Articles Miscalculation

Since August 2015, time spent per article on Instant Articles was over-reported by about 7-8%. The error is caused by the method of calculation used. Facebook was calculating the average time spent across a histogram. A more effective formula was to divide the total time spent on an article by the total views.

 

  1. Referrals on Analytics for Apps

This is another miscalculation. Referral is a metric that tracks and evaluates posts made via an app or website. The metric is supposed to only count the clicks directly towards the app or website but other click events on the post were erroneously included. Over-reporting on referrals reached to an average of 6%.

There are also some minor problems that Facebook noticed. One is the Interest Lists. Facebook is retiring the feature for the reason of “low consumer usage” and the unnecessary doubling of profile follower counts. People can follow someone through their profile page and through an interest list at the same time. The retirement of the interest list can decrease most profiles’ follower counts by 5%. There is no information though if this can also affect interest targeting in Facebook’s ad targeting options.

RELATED :  ZeroPark Review

 

Case 3: Reach and Engagement Miscalculations

December saw another announcement of miscalculations but also fixes to some Facebook metrics. The fixes were first published on December 9 but another update was necessary on December 16. The update was about another bug in Instant Articles (see details below).

  1. Shaky Estimated Reach

During ad creation, estimated ad reach is something that warranties tweaking on the ad’s targeting options. It’s somehow a downer when you realize that it is really that accurate in of itself. Facebook did not explain further though their improved audience sampling and extrapolation methodology. But they guarantee that it will be more accurate and it will take account of not only Facebook and Instagram audience but also on the Audience Network. There’s an expected 10% increase or decrease in the audience size, though.

Facebook Metrics Estimated Daily Reach

  1. Live Video Reactions

Facebook metrics for reactions are not that well defined yet. In live videos, for example, someone can select different reactions for different points of the broadcast. The issue is in the reporting of these reactions. The “Reactions on Post” column in the Page Insights, reports only one reaction per person/unique user. The other reactions were then allocated to the “Reactions from Shares of Post” column. So, it is not a miscalculation at all. The other reactions just got lost along the way.

 

Facebook Metrics Facebook live reactions on post

 

  1. Like and Share Discrepancy

Facebook also identified a discrepancy between the likes and share counts delivered by the Graph API and the counts on a URL search result in the Facebook app. This could mean that the like and share metrics count is different from the mobile search query count for some reason.

Facebook like share mobile search discrepancy
The shares on the search result are quite far from the actual share count on the post itself.

 

  1. Instant Articles Again

The issue with Instant Articles this time is an underreporting of mobile traffic from iPhone and Android on comScore products. This glitch is said to happen on September 20 to November 30, 2016. It is also an isolated case, only affecting several Facebook partners.

 

The Aftermath: Facebook’s Move

Facebook states they already solved all previous issues with their metrics. Most of the time, these solutions involve formula change, adjustment, and improvement. Facebook also posted “An Update on Metrics and Reporting” on November 16 to inform partners about any action they have undertaken to resolve any glitches in their metrics.

Here is a summary of what the update entails:

  1. After an industry clamor for Facebook metrics to have an independent measurement tool, Facebook increased third-party verification. They are now working with companies like comScore, Moat, Nielsen, Integral Ad Science (IAS) and other reviewers that would measure and report traffic and other Facebook metrics. Moat and comScore were the ones that found out some of the discrepancies in measurement after all.
  2. There’s a plan for the formation of a Measurement Council. Facebook’s global Client Council is instrumental on different innovations not only in measurements but on Facebook’s product direction. And the Measurement Council would be another of their contribution together with other business and measurement executives.
  3. Facebook metrics are continually evolving. Facebook now aims to clearly and regularly communicate their metrics. There would be an internal review process, in-product definitions of metrics, direct communication of updates to partners, and the formation of the “Metrics FYI Blog” to openly explain meaningful updates on the Facebook metrics.

At the end of the day, any glitch on the Facebook metrics is just that: a glitch. Is this problem isolated to Facebook? Sadly not. Publishers and advertisers admitted that the problem is bigger than Facebook. The problem with internet metrics accuracy bypasses any platform. But for Facebook to rise as a reliable advertising platform, it should invest not only on proper tools but also in fostering trust.

facebook metrics

Mustapha Ajermou
Follow me