As media companies and advertisers search for better ways of measuring the effectiveness of content, a lot of attention has been focused on what some like to call "engaged time." Designed by companies like Chartbeat, the publishing-analytics company, Upworthy — which recently open-sourced its "attention minutes" measurement — and the Financial Times, these metrics try to measure the time that a reader actually spends with a story instead of just tracking clicks. But is engaged time a red herring? ProPublica general manager and former Wall Street Journal assistant publisher Dick Tofel thinks it might be.

Tofel lays out his argument in an essay he wrote for the Media Impact Project, a venture launched by the USC Annenberg School for Communication and Journalism that's trying to improve the theory and practice of media measurement.

In a nutshell, Tofel argues that while "engaged time" might be an improvement over blunt or fatally flawed instruments like raw pageviews or even unique visitors, it still suffers from some fairly major problems. For one thing, he says the simple fact that a person spends more time with a specific story doesn't necessarily mean that they are actually more engaged. It could mean that the story is just really long, or it's a video clip — and the longest stories aren't always the ones with the most impact.

"The metric implicitly posits that longer stories — assuming they are read for longer periods — are more valuable, more engaging, more salient than shorter stories. But our experience teaches us that this is not the case. The most memorable stories are not always (or perhaps even frequently) the longest."

Most content gets no attention at all

Tofel also argues that encouraging media companies to focus on engaged time or "attention minutes" could cause publishers to lengthen the content they are producing, in the same way that focusing on pageviews drove some media companies to spend all their time pumping out tiny chunks of viral clickbait. In other words, the metric itself could have unintended consequences: "Do we really want reporters and editors to undertake a constant effort to take up more of a reader's time, to always lengthen stories where feasible… to use more video simply because people may mindlessly watch?"

And what do some of the companies pushing the idea of engaged time or attention minutes have to say? The Media Impact Project has gotten essays in response from Chartbeat CEO Tony Haile as well as Daniel Mintz from Upworthy, Anthony Hitchings from the Financial Times and Jonathan Stray of the Overview Project, a document-mining system the Associated Press is building. Haile says he agrees with Tofel that measuring the actual impact of a specific story is difficult, but argues that the idea behind engaged time is somewhat different:

"The fight is not between the short form content that receives 30 seconds of attention and the long form that receives 300… the fight is with content that does not receive any attention at all. Simply distinguishing the value of content able to capture a meaningful expression of attention from that which does not would have more impact on the economics of digital publishing than anything else."

By implication, there are two main strategies publishers can follow, Haile says — they can either create great short-form content that gets people to return regularly, or they can create long-form stories that get readers to engage for a long time. The only people who suffer under this scenario, he says "are those who create content that no one reads and to which no one returns."

Perhaps there is no holy grail

Jonathan Stray says like Tofel, he is unconvinced that engaged time is the holy grail of reader measurement: "Journalism is very much a multi-stakeholder endeavor, so why should we imagine that a single number can capture all aspects of the activity?" Daniel Mintz of Upworthy, meanwhile, says he agrees that engaged time isn't the ultimate barometer for media measurement, but argues that it is another tool that can be used in addition to all of the existing metrics in order to get a better handle on what readers are doing and why:

"We're not advocating… that publishers and advertisers should ignore pageviews and uniques and shares, in favor of attention as the only metric that matters. Instead, we're arguing that right now, we measure the input (pageviews and uniques) and the output (shares) of a system with no real understanding of what is happening in-between. Measuring attention provides invaluable insight into that gap between input and output."

It's more than a little ironic that while publishing online provides an almost infinite ability to measure every twitch and click and eye movement of the reader, media companies and advertisers aren't really that much closer to figuring out how to tell whether what they are publishing is actually having any effect or not. About the only thing we can say for sure is that as long as there are companies publishing content online and trying to appeal to advertisers, this debate will likely continue.

Post and thumbnail images courtesy of Thinkstock / Darren Klimek and Shutterstock / Zastolskiy Victor