Analog Surveys in a Digital World

Texterity and BPA have teamed up to once again tell us what digital magazine readers are thinking. As a digital magazine marketer, I’d be remiss if I didn’t tell you that 89 percent of readers said they were satisfied or very satisfied with their digital magazine. But as a technology provider, I’m once again embarrassed at the questions being asked that could be answered so much better from tracking software than from reader opinion.

From the get-go, such surveys are flawed because the vast majority of people agreeing to take them are already self-selecting themselves as being fans of digital magazines. After all, how likely would you be to answer 36 questions about something you don’t like?

What truly drives me crazy, though, are the questions that are asked which any reputable vendor already knows the answer to, including:

How much time do you spend reading your digital magazine?  While it sounds good to say 47 percent of readers spend at least 30 minutes inside the digital magazine, what this really says is that self-selected fans of digital magazines believe they spend at least 30 minutes inside the digital magazine. Truth of the matter is, the average digital magazine readers spends much more time inside the publication than they spend on a Web site, but more than 30 minutes?

 More to the point, reputable vendors KNOW this information — for each title and for all of their titles – and it’s far more vital than what self-selected fans "think." The average reader will spend more than five minutes inside a Nxtbook and visit 20 pages. Because it’s an average, half will spend more time. Half will spend less. Period.

Similarly, readers are asked if they "prefer" to read online, or download issues for online use. Again – why ask what they prefer? We know the exact answers, both for each Nxtbook and for all Nxtbooks. (Eight to 10 percent download the offline version with us – not that I particularly think it matters.)

Forty-eight percent of readers "say" they clicked on an advertisement in a digital magazine. Forty-eight percent? Really? Once again, good digital magazine vendors have the data to show you higher click-through rates than your Web site, though not anything as loopy as 48 percent. Why insult the intelligence of anyone to suggest such high numbers, though, particularly when the real numbers are found inside decent tracking data? I dare say that if any medium anywhere can get 48 percent of users to click on an ad, we should all just go do that (or at least advertise our wares in a medium that does that).

Another point in the survey says that digital magazine readers are engaged because a large percentage of them open the digital magazine within a week of being notified about it. In what world is how quickly someone opens an email an indicator of engagement? Engagement can be measured many way — time spent, click-through rates – but I’ve never heard speed at which an e-mail is opened be one of them. Moreover, we’re finding that e-mail is only responsible for 50 percent of Nxtbook readership — so this point is moot for half of our audience.

Quite frankly, I don’t see a fit for such "analog surveys" of a digital product. I said it a few months ago when Zinio told us how their readers "felt" and I’m saying it again. Research like the recent Gilbane Study are critical because they offer publishers hard data about people are doing – and not fuzzy figures about what self-selected fans "think" they’re doing.

We have offered and will continue to offer such surveys to our customers only because for the publisher who hasn’t done a digital magazine before, it can shed some perspective on how their launch will go. But in aggregate, the "stake in the ground" information seems to lose a grip on reality.

Incidentally, if somebody more versed in marketing research than I cares to comment on the benefit of opinion-driven satisfaction surveys in lieu of data-driven research, I’m all ears. I’m sure there are some "perception is reality" thoughts that have merits that I’m not articulating well.