Does Your Personal Experience Prove Anything?

file4851264123548Many of the book manuscripts I’ve seen over the years present a system for behavior change that’s based solely on a sample of one—the author’s personal experience. The premise of these books is I did X, and here’s how you can do it, too. (For X, substitute any desirable outcome: lose 25 pounds, make a million dollars, attract a loving partner, find parking spots on a regular basis, etc.)

What’s easy to miss is the underlying assumption: If it worked for me, it will work for you.

This is exactly where many authors—and bloggers—try to build a towering edifice of content on a crumbling foundation.

WHY N = 1 MATTERS

Scientists who design experiments use the capital letter N to denote the number of subjects involved. If a psychologist includes N = 2000 in a paper about her study, it means she observed 2000 people.

Why pay attention to this number? Because the more subjects, the better. A study based on observations of 2000 people (or 20,000 or 200,000) inspires more confidence than a study based on 20 people. With more subjects, we have more confidence that the conclusions will generalize to the population at large.

If N = 1 (meaning that the author is the only subject), then the work inspires little—if any—confidence that the observed results will apply for the rest of us.

I use the same logic to evaluate books on an informal basis. If one person tells me that the system recommended in a how-to book worked for her, I’ll make a mental note. If three people tell me that the book works, I’ll get interested. And if ten people recommend that book, I’ll definitely check it out.

WHY N = 1 FAILS

There are many reasons why what works for one person might fail for the rest of us. I’ll focus on three:

It’s possible for people to draw dumb lessons from their personal experiences. Check out this example about a tech journalist who concludes that Apple is in big trouble because he sees more people with Android phones than iPhones at a single convention.

Correlation does not equal causation. The outcomes that an person experiences might be due to coincidence and have nothing to do with the program he recommends.

For example, medical researchers have a term to describe symptoms that resolve themselves simply with the passage of time: These conditions are called self-limiting. If the problem you’re dealing with is self-limiting, then simply waiting it out might be more effective than any strategies you try to implement. Those strategies might correlate with a solution but not cause it.

For more examples of the correlation = causation fallacy, see this wonderful post by April Hamilton.

Life is complex. Human behavior is multi-faceted. Any event can have multiple causes. When we recommend one strategy or technique as the way to produce a given result, it’s almost always possible that we’ve missed another contributing factor.

THREE WAYS TO AVOID N = 1

Limit your assertions to “this is how it worked for me.” I appreciate authors who tell their story in a moving way without the arrogant assumption that my story will unfold in the same way.

This is one of the reasons I enjoyed Walter Isaacson’s biography of Steve Jobs. There are many lessons that I can draw from Jobs about how to live well (and how to not live well). But Walter makes no attempt here to offer a program for “succeeding the Steve Jobs way.”

Perhaps this seems obvious. Yet many of the proposed self-help books I see are biographies or memoirs in disguise. In brief, they’re ultimately about me, not you.

Test, test, test. If you can conduct a well-designed, formal scientific study to test your program—great. Chances are, however, that you don’t have the resources to do this. Next best is to recommend your system to as many people as possible and track their results over time as closely as you can.

This is one of the many reasons that I appreciate David Allen’s Getting Things Done system. David used these ideas with hundreds of consulting clients over decades before publishing a book.

Present a process that readers can test. The most powerful how-to book is one that helps readers to discover what works for them.

I’m enjoying James Altucher’s Choose Yourself for this reason. The book is loaded with personal anecdotes (including times when James failed miserably). Yet his core recommendation is a set of daily practices. You can test them for yourself and see what sticks.

The work of Ramit Sethi has the same spirit. He recommends specific behaviors for us to adopt and encourages us to systematically track results.

There are useful lessons here for all of us who seek to influence the way that people think and behave:

  • Offer possibilities, not prescriptions.
  • Recommend specific behaviors to implement rather than vague generalities.
  • Be willing to revise your system based on feedback.
  • And above all, be humble. As James Altucher once put it: “Note, this is only what works for me. For you, I have no idea.”