3.3 Checking It Out

(This section is updated. Two paragraphs quoted from Howard Rheingold’s “Crap Detection 101” essay were inadvertently not indented in the original; this has been fixed here and will be fixed as soon as possible in the ebook and print editions as well. Apologies.)

In late 2009 a journalist in Tennessee wrote a shallow and ill-informed column (no longer available online) about citizen journalism, or rather what he imagined it to be. He discussed me and my work for several paragraphs and got almost everything wrong, including a) misspelling my name, b) misidentifying my current academic affiliation, c) claiming I’d left the news business when I stopped writing a column, and a number of other things. He capped this cavalcade of mistakes by advising everyone looking at citizen journalism to do what “real” journalist do: to check things out before believing them. I nominated him for the (nonexistent) Irony Hall of Fame (Media Wing). I got no reply to my email requesting corrections.[1]

The experience reminded me, not for the first time, that the news field would greatly improve if every journalist was the subject of this sort of poor journalism—there’s nothing like being covered to understand how flawed the craft can be. It also highlighted two issues you need to consider when you want to gauge the quality of the information you’re getting. One is simple accuracy; in this example, the misspelled name and wrong employer were egregious. The other is the choice of topic and the slant of the reporting; the Tennessee columnist wanted to promote his own craft while slamming something he considered inferior.

Factual errors are part of the journalistic process. They happen, and in a deadline-driven craft we can understand why. But when errors are blatant and careless, they call into question everything else the journalist does. Worse, when they’re not corrected promptly and forthrightly, is the message of arrogance they send to the audience.

Of course, you can’t check everything out yourself. (Although you can and should, as I’ll discuss later, be careful about what you create in your own media.) But when you’re looking into something where being wrong will have consequences, and if you are unsure of the source of the information, you have every reason—even an obligation—to check further.

Howard Rheingold, an author and friend, has been at the forefront of understanding the digital revolution. In a terrific 2009 essay called ”Crap Detection 101”(riffing off a long-ago line from Ernest Hemingway), he wrote about some of the ways to check things out. Here’s a key quote:

The first thing we all need to know about information online is how to detect crap, a technical term I use for information tainted by ignorance, inept communication, or deliberate deception. Learning to be a critical consumer of Web info is not rocket science. It’s not even algebra. Becoming acquainted with the fundamentals of web credibility testing is easier than learning the multiplication tables. The hard part, as always, is the exercise of flabby think-for-yourself muscles.

Fortunately, crap detection tools are far more powerful today than they were a decade ago; the bad news is that too few people know about them. That has consequences: Many more people have started to rely on the Web for such vitally important forms of information as news, medical information, scholarly research and investment advice that the lack of general education in critical consumption of information found online is turning into a public danger.

So, no, Bill Gates won’t send you $5 for forwarding this chain email. The medical advice you get in a chat room isn’t necessarily better than what your doctor tells you, and the widow of the deceased African dictator is definitely not going to transfer millions of dollars to your bank account. That scurrilous rumor about the political candidate that never makes the mainstream media but circulates in emails and blog posts probably isn’t true. The data you are pasting into your memo or term paper may well be totally fabricated.

There are innumerable tools and techniques that you can use to winnow out the falsehoods, and people who work hard to help you understand what’s real and what isn’t. Here are a few of my favorites. (As always, we’ll be compiling a much longer list, broken out by topic area, at mediactive.com.)

Checking Out a Web Page

Howard Rheingold’s Crap Detection instructions are a great place to start. Similarly, Scott Rosenberg (another author and friend) has come up with a long list of ways to check out a given website. A sampling:

• Does the site tell you who runs it—in an about page, or a footer, or anywhere else? Is someone taking responsibility for what’s being published? If so, obviously you can begin this whole investigation again with that person or company’s name, if you need to dig deeper.
• Check out the ads. Do they seem to be the main purpose of the site? Do they relate to the content or not?
• Look up the site in the Internet Archive. Did it used to be something else? How has it changed over the years? Did it once reveal information that it now hides?
• Look at the source code. Is there anything unusual or suspicious that you can see when you “view source”? (If you’re not up to this, technically, ask a friend who is.)

Detecting Accuracy

Remember the bogus email in Chapter 1, where the writer was claiming things about Al Gore and Oliver North that weren’t true? Snopes.com helped me learn the reality. This site is all about confirming or debunking the stories that race around the Internet every day. Look around Snopes, and be amazed.

UrbanLegends.about.com, a site run by the New York Times, is also helpful for sorting out paranoid nuttiness from truth.

FactCheck.org, a political fact-checking site, and its FactChecked.org companion site for students and teachers, will help you sort through a few of the political claims tossed around our republic. Your best bet, I’d suggest, is to assume that everything you see in any political advertisement is at best misleading, especially if it’s an attack on a candidate or campaign.

QuackWatch.org is invaluable for debunkery of, you guessed it, bad information about health.

In the experimental category I’m a fan of MediaBugs.org (another project on which I serve as an advisor). Scott Rosenberg, with the help of people like you, is compiling a database of journalistic errors, including notes on whether or when the mistakes are corrected. If he can get enough buy-in from journalists at all levels in his early experiment in the San Francisco Bay Area, this could become a national resource of note.

Detecting Slant

The Center for Media and Democracy, which leans left politically, has created an invaluable information trove about the organizations that seek to persuade us to buy or believe. It’s called SourceWatch, and I frequently check it out.

The mass media consistently misrepresent science and medicine. Ben Goldacre, a British doctor, writer and broadcaster, runs BadScience.net, where he routinely demolishes bad reporting in ways that help readers understand how they can be more discerning themselves. If you follow his work you’ll more easily spot bad science reporting yourself.[2]

Fabrice Florin‘s project Newstrust.net, aimed at persuading communities of readers to grade reports based on a variety of criteria, is a promising approach. I encourage you to join and add your own knowledge to the database. (Note: I’m an advisor to the project.)

Again, this is the briefest of lists. The key point is that the more something matters to you—the more you have at stake—the more you need to investigate further. I’ll be adding a fresher and more extensive list to mediactive.com.

Risks, Statistics, Lies

Asking yourself whether something makes sense is especially relevant in understanding risk. Journalists have been, as a trade, beyond negligent in explaining relative risks. Local television news, for example, has been almost criminal in its incessant hyping of crime even during times when crime rates were plummeting, helping persuade people that danger was growing when it was in fact shrinking. While the individual crimes and victims are all too real, the overall incidence of crime has been grossly overstated. And legislators, all too happy to “do something” in response to media-fed public fears, often pass laws, such as Draconian sentencing for non-violent crimes, that do much more aggregate harm than good.

Medical news reports, moreover, tend to vary from ill-informed to downright crazy; the unwillingness of a significant portion of the American population to get vaccinated for the H1N1 flu, based on paranoid rumors and media reports, is downright scary. Panic is often the greatest danger, because it leads to bad responses, and when the media fuel panic they are doing the greatest of disservices.

Statistics are a related problem. Too few people understand statistical methods or meaning. If you hear that such and such product or substance is linked to a 50 percent rise in some low-incidence disease, you need to also understand that the likelihood that you’ll get that ailment remains vanishingly small.

These are issues of slant, not accuracy. But they have everything to do with our understanding of the world around us.

I can hear you asking this important question: Who has the time to look into all of this, anyway? No one does. But when you’re looking at something you care about, or when you’re just suspicious, isn’t it worth taking a little extra time to check just a bit further?

The bottom line is to start with common sense. Heeding that first bit of skepticism can save you a lot of pain later.

---

^[1]I took the newspaper editor, Tom Bohs, to task at greater length on the Mediactive blog in a post entitled, “That Hallowed Standard of Accuracy: Oops.”

^[2]“I don’t want to be a guy who says ‘This is good and this is bad,’” Goldacre told me. Rather, he wants to help people understand how they can go about being more careful in their media consumption.

“The reality,” he said, “is you’ll never be able to have a set of rules for whether someone is reliable or not. What you can derive clearly are heuristics: time-saving devices and shortcuts. They are reasonably accurate, but they misfire sometimes. People try to game our heuristics. This is what quacks do when they buy fake doctorates.

“What’s interesting about reading online with linked text is that heuristics become quicker. One of most powerful heuristics I use is whether someone writing about scientific research links to the original paper or at least to the press release. If not, I won’t waste any time reading it.”