Anecdotes Have Their Place — But Not as Proof of Your Ideas

ben-white-178537Human beings are story-telling animals. We need stories, including great fiction, to make sense of our world. We crave a narrative arc for our lives — events that fall into a clear pattern of beginning, middle, and end with clean resolutions and clear lessons learned.

Yet we also have the capacity to tell stories that delude and even harm us. If you’re an idea entrepreneur or someone who curates content in any field, this is a point you cannot afford to forget. To preserve your credibility as a writer and speaker, avoid common mistakes when using anecdotes.

Recognizing anecdotes

Anecdotes are mini-stories. We tell them to make a point. Some that I’ve heard are:

  • “You can’t trust the Surgeon General’s report on the dangers of smoking. My grandfather smoked every day and still lived to be 90.”
  • “Dietary supplements work. I know someone who took that raspberry supplement featured on Dr. Oz’s show. She lost a lot of weight.”
  • “Yoga is dangerous. One my friends injured herself last week during class.”

When I hear statements such as these in casual conversation, I sometimes opt to let them pass by. But when a client’s blog post or book manuscript leans heavily on anecdotal evidence, I’m obligated to share my concern.

Remembering the dangers

In a classic post —Hubris, Not Bad Writing Or Design, Sinks Most Self-Published Nonfiction — April Hamilton illustrates how anecdotes can delude idea entrepreneurs:

A tax attorney who’s struggled with her weight for years finds she’s somehow managed to lose fifteen pounds in one month. On reflection she realizes she’s been eating a lot of hazelnuts lately. Her internet research shows nuts are often encouraged as part of a high-protein, low-carbohydrate diet, and she finds some studies that report hazelnuts have antioxidant properties. BOOM! The Hazelnut Crash Diet book is born. 

A computer programmer’s YouTube parody of a celebrity is brought to the attention of the celebrity, who mentions it on a late-night talk show. The clip goes viral in a matter of hours. In the morning, the man learns what happened and finds he has several interview requests from the media…BOOM! How YouTube Can Make You Famous is born.

A caregiver in a nursing home notices the elderly in her care seem more responsive and alert when she plays music over the facility’s public address system. BOOM! Using Music To Beat Alzheimer’s Disease is born.

Yes, these are hypothetical examples. But they’re not far removed from what I often find on the Internet and bookstore shelves. Much “how to” and self-help material is anecdote-based. And there are entire industries based on this kind of skimpy support.

Eight specific problems with anecdotal evidence

From the fields of logic, statistics, and research design come a powerful list of problems with anecdotal evidence. Following are a few.

1 Small sample size. Anecdotes are commonly based on a random sample of one. And yet one person’s isolated experience proves nothing.

The fact that my friend’s nicotine-addicted grandfather lived to be 90 doesn’t mean that smoking is safe for you and me. The Centers for Disease Control states that smoking kills more than 480,000 Americans each year. Now that’s a meaningful sample size.

2 Confirmation bias. This is the tendency to notice information that supports our existing beliefs — and to ignore information that contradicts those beliefs.

Today our nation’s political discourse is polarized, and one reason for this is confirmation bias. Many Republicans rely on Fox News; I depend on the New York Times. We filter out sources that challenge our views.

To overcome confirmation bias, seek out information that challenges (disconfirms) your current thinking. Do what scientists do: They don’t try to prove a hypothesis. Instead, they merely say that they failed to disconfirm the hypothesis. And they’ll remind us that disconfirming data can occur at any point in the future.

3 Reporting bias. People who seem to benefit from a medical treatment or product are more likely to report their experience. Those who tried the same treatment or product and had no benefits have little incentive to share their anecdotes.

Besides, people who die from diseases or unhealthy behaviors such as smoking are no longer around to share their anecdotes with us!

4 Confusing correlation with causation. When event A occurs shortly before event B, this does not prove that event A caused event B. As statistician Tyler Vigen points out, forgetting this can lead us straight into absurdity.

For example, the divorce rate in Maine correlates strongly with per capita consumption of margarine. This is not a valid argument for banning margarine!

5 Confounding factors. People who live to a ripe old age despite smoking might have other factors — such as an unusual genetic load — working in their favor.

Yoga injuries can result from a failure to follow the teacher’s instructions rather than any inherent flaw in this type of activity.

The person who lost weight while taking a supplement might have also made behavior changes — such as eliminating fast food — that account for the weight loss.

Scientists try to rule out such confounding factors with randomized controlled trials (RCTs). In these experiments, participants are randomly assigned to two groups. One group gets a treatment; the other group does not. Participants are also carefully screened to be a similar as possible in other relevant characteristics, such as age, gender, and habits.

These conditions give researchers more confidence that any change experienced by participants is actually due to the treatment — not to a confounding factor.

6 Faulty memories. Physicians will tell you that one of the biggest challenges in taking a medical history is human memory. We can omit key facts, combine details from various events, or simply forget that an event ever occurred.

7 Embellishment. We also tend to tell and retell our favorite anecdotes. And during social events, we sometimes discover that we get a bigger laugh or more dramatic reaction by subtly altering key incidents or glossing over minor details. As a result, anecdotes can change radically over time, becoming useless as a form of evidence.

8 Lack of replication. Scientists publish papers that describe in detail how they carry out their experiments. (Look for the Methods section.) The reason is to encourage their peers to follow the same procedures and see if they get similar results. When a scientific finding is widely repeated (replicated), we’re more confidence that it’s accurate.

The biggest flaw I see in self-help and other “how to” books is lack of replication. There’s little or no evidence that anyone besides the author has implemented the suggested strategies and seen the same results.

Using anecdotes responsibly

Seeing the flaws in anecdotes does not mean rejecting them entirely. To include them in a credible way:

  • Be thorough. Use fewer anecdotes and make them longer. Tell an interesting story while including as many facts and events as possible. Running examples — anecdotes that cross from section to section or chapter to chapter — are one way to do this. In addition, document anecdotes in detailed notes and archive them for future reference.
  • Supplement anecdotes with credible research. David Allen did this in the second edition of his popular book Getting Things Done: The Art of Stress-Free Productivity. He added a chapter titled “GTD and Cognitive Science” with summaries of studies that support aspects of his method. Allen’s work is still anecdotally-based, but this nod to science gives the book more weight. In addition, he offers many detailed anecdotes that relate to specific GTD strategies.
  • Use anecdotes for color and for clarity. Anecdotes can be entertaining and instructional. They can be memorable. They can drive home an abstract idea with concrete examples. And they can demonstrate how to carry out a set of instructions. In short, use anecdotes for illustration rather than proof.

Finally, remind your audience about individual differences: What works for me might not work for you. And even what works today might not work in the future.

Simply put, YMMV — your mileage may vary.

Also see:

Does Your Personal Experience Prove Anything?

Anecdotes Are Sexy But Not Conclusive

Won’t Get Fooled Again—Three Levels of Credibility in Self-Help Books

John Butman on Stories, Methods, and Metrics—Three Staples of Nonfiction That Can Backfire on Authors

Photo by Ben White on Unsplash