The most recent Vanity Fair has a long piece on Facebook and their new algorithims and processes for serving better ads. In it, there’s this paragraph:
Moreover, the company has created a computer framework that allows it to constantly revise the Facebook site. At any time of day, hundreds of different versions of Facebook are running on the Internet—with a changed color here, a moved button there—and the user response to each variation is measured. And the same is done with advertising. Through what executives call A/B tests, a company can run different ads at the same time, have Facebook engineers measure the response, then ditch whichever ones had less of an impact.
Vanity Fair is a general interest publication, so it’s fine that they explain A/B tests to the uninitiated. But the piece loses credibility with a single phrase: “…what executives call A/B tests..” Wait, did Facebook execs invent A/B tests? No way. The sentence should have read: “Through what’s called A/B tests…” so as not to imply that A/B testing is in any way unique to Facebook.
It’s a tiny word thing. But it caused me pause, and made me not trust the rest of the article in terms of insight. To be sure, had the thrust of the piece been a profile of an exec or an examination of the company’s public policy efforts or any number of other things — it’d be a forgivable slip up. But it in fact centers on the company’s ability to serve targeted ads and monetize user behavior, and the A/B testing point is held up as evidence of innovation. And the author subtly overstates the newness and interestingness of that point, which conveniently supports his broader thesis. I ask myself, “What else could he be getting wrong?” and switch from reading to skimming.
Yep. I stopped reading at that phrase.
In my view this example is less about reader “trust” and more about the lack of domain knowledge of the journalist. It simply demonstrates that the writer does not know that this is a term of art in the business. Thus he is relying on what he learns from the interviews to report, which means (first) as you point out he is unlikely to provide any new insights directly and (second) the reporting is subject to manipulation by Facebook since that’s probably the main source of information. However, there is the possibility that some techniques that Facebook is using could be described in the article. In any case, what it means is that it’s an article written for people who do not already know what A/B testing is, just like the journalist.
By the way, you spelled “algorithms” incorrectly. I’ll let that one slide as a typo. 😉
I agree this example is less about reader “trust” and more about the lack of domain knowledge of the journalist, as DaveJ says.
I had a visceral reaction to your own remark about subtly overstating a point to conveniently support a broader thesis, however. I have a similar response to the writing of many of your friends and mentors, especially that of such luminaries as Arnold Kling and Tyler Cowen. I don’t trust the motives of anything they say, even if I should happen to agree with a particular point.
This is only roughly analogous, but I read a recent post at Kling’s new blog, linked to by his partner-in-crime Cowen (one does need to keep abreast of what the enemy is up to), in which he uses rather weaselly language to slip in an assertion that “Probably more borrowers were “victimized” by Freddie Mac, Fannie Mae, and FHA than by Wall Street” in a screed ostensibly about housing and wealth destruction.
It may sound innocuous, but this Big Lie, promulgated in the right-wing disinformation campaign waged by Wall Street and its allies to deflect blame for the financial crisis to Fannie Mae, Freddie Mac and the Community Reinvestment Act, has been thoroughly discredited — yet Kling and his like keep resurrecting the zombie meme that federal affordable housing policies were responsible for the proliferation of high-risk mortgages over the last decade.
This sort of skullduggery committed by a credentialed economist (one who was also a senior economist at Freddie Mac from 1986 to 1994) is a far more egregious violation of the implicit trust more naive readers might place in his writing than any slip-ups in “tiny word things” a journalist contributing to Vanity Fair might commit, short of outright libel or slander.
PS:
My God, your comments are being overwhelmed by spam. It’s getting more and more difficult to find the legit responses in the comments RSS feed.
Marked a whole bunch of comments as spam — hopefully that will get WordPress to block them going forward. Sorry and thanks!
“the lack of domain knowledge of the journalist”
I’m not sure what’s worse: the lack of domain knowledge or the journalist deliberately taking the position of lacking the knowledge to emulate the presumed lack of knowledge of the reader. How many times have you seen a competent, knowledgable journalist suddenly go “stupid” and play the part of the (presumed to be) ignorant reader/viewer?
Ignorance vs dumbing-down should be more of a false choice than it is.
Good point. More common than we think, I bet.
Dave, Vince: Surely if the journalist displays ignorance of his topic, this should decrease the rational reader’s trust in the quality of the (rest of the) article?
An on point quote from Michael Crichton’s 2002 speech, “Why Speculate?”:
Excellent quote. Thanks.
I would guess that the phrase was the result of the editorial process – that a top editor, line editor, fact checker along the way said they needed to attribute A/B testing to someone, that the writer fought back, and the phrase is a compromise. Or it could be the writer. But I doubt it.