Anecdotes Are Sexy But Not Conclusive

Chris1stDollarAICIf you’re writing a “how-to” book, then how do you know that your ideas actually work?

As an author, your goal is to give instructions that consistently help people produce a specific outcome. This means offering credible evidence for your claims. And evidence means something more than personal anecdotes.

When I talk about this with prospective clients, the conversation typically goes through certain stages. I’ll illustrate with an extreme (and fictional) example:

Me: Your book lists 123 techniques that readers can use to change their self-defeating habits. Where did those techniques come from?

Author: Well, I read a lot of self-help and spirituality books.

Me: Great. Me, too. Readers will want to know what’s new or different about your book.

Author: Well, I put my own spin on everything. Plus, the techniques worked for me.

Me: Really? You personally tested all 123 techniques?

Author: Well, not all of them. Most of them.

Me: What about the techniques you didn’t test?

Author: Well, my friends like them.

Me: Your friends? How many?

Author: Well . . . actually, my wife read the first draft of the manuscript. She really likes everything.

Me: Did anyone else read it? Maybe someone with a more objective point of view?

Author: Ahhh . . . ummm . . . well, not yet.

Me: OK. No problem. We can work on that. Now, did you do a literature review?

Author: Literature review? What’s that?

Me: It means you look through any research that is relevant to the subject of your book. There are hundreds of published studies about habit change.

Author: Really? Where do you find that stuff?

Me: In professional journals that publish peer-reviewed papers.

Author: I don’t bother with that stuff. It’s too dry.

Me: So as of right now, we don’t have solid evidence that your techniques work.

Author: Of course they work. They worked for me, and my friends, and my clients. I can give you lots of good anecdotes.

Me: Well, anecdotes aren’t quite enough. They are useful as stories that you can use to lend a human touch to your writing. But a few isolated anecdotes don’t count as evidence for your ideas.

Author: No? Why not?

Me: For lots of reasons. Because anecdotes are only based on the experiences of a few people. Because they’re often embellished when retold over time. And because anecdotes don’t involve careful observation or isolate any variables. When you produce a new result in your life, how do you know that one of your 123 techniques created it? Human behavior is complex. Events have more than one cause.

Author: Hmmm. . . . (Thinks to himself: This guy is going to be a pain in the ass. I need another editor.) . . . . Well, this is all real interesting. I’ll get back to you.

Also see Does Your Personal Experience Prove Anything?