The most recent Vanity Fair has a long piece on Facebook and their new algorithims and processes for serving better ads. In it, there’s this paragraph:
Moreover, the company has created a computer framework that allows it to constantly revise the Facebook site. At any time of day, hundreds of different versions of Facebook are running on the Internet—with a changed color here, a moved button there—and the user response to each variation is measured. And the same is done with advertising. Through what executives call A/B tests, a company can run different ads at the same time, have Facebook engineers measure the response, then ditch whichever ones had less of an impact.
Vanity Fair is a general interest publication, so it’s fine that they explain A/B tests to the uninitiated. But the piece loses credibility with a single phrase: “…what executives call A/B tests..” Wait, did Facebook execs invent A/B tests? No way. The sentence should have read: “Through what’s called A/B tests…” so as not to imply that A/B testing is in any way unique to Facebook.
It’s a tiny word thing. But it caused me pause, and made me not trust the rest of the article in terms of insight. To be sure, had the thrust of the piece been a profile of an exec or an examination of the company’s public policy efforts or any number of other things — it’d be a forgivable slip up. But it in fact centers on the company’s ability to serve targeted ads and monetize user behavior, and the A/B testing point is held up as evidence of innovation. And the author subtly overstates the newness and interestingness of that point, which conveniently supports his broader thesis. I ask myself, “What else could he be getting wrong?” and switch from reading to skimming.