How Do You Know That Your Stuff Works?

_DSC6301I’m a fan of the Getting Things Done method (GTD) for managing projects as explained in David Allen’s best-selling book. Still, I felt some familiar concerns when I read Paul Keegan’s article about David Allen. Though the tone is upbeat, the following passage snagged my attention:

…Allen’s book is notable for being nearly devoid of research citations, footnotes, and other source material. Most of its assertions begin with the phrase “In my experience…” There is no research, for example, to back up one of the book’s central claims — that commitments made and abandoned are robbing our lives of energy and attention and that only when we close these “open loops” can we achieve a state of relaxed focus.

No research to back up the central claims… How often does this apply to my clients’ work, and to my own?

And does this bother anyone besides me?

This issue goes deeper. It’s one thing to lack rigorous evidence, but it’s quite another to dismiss the very need for it. Keegan notes this about Allen’s attitude toward GTD:

No studies exist proving that it increases productivity, decreases stress, or boosts the bottom line, Allen admits, but he says such questions miss the point entirely. “Anybody who experiences this and still needs proof didn’t get it,” he says.

Based on his videos and podcasts, I see David Allen as smart and supremely nice. Yet this attitude—if my stuff doesn’t work for you, it’s your fault—strikes a false note.

Assertions that are backed only by “in my experience” are examples of reasoning based on anecdotal evidence. And the problems with anecdotal evidence are legendary. Our cognitive biases—such as cherry-picking examples and making inaccurate observations—kick in immediately. The dilemma is that we love to tell stories (anecdotes) and frequently delude ourselves with them.

To his credit, David Allen has loads of anecdotal evidence to support GTD. He’s coached people on his methods for decades. His consulting business is doing well. And smart people such as James Fallows and Dan Pink swear by his stuff.

I endorse GTD, too. But I’m willing to admit that it might not work for everyone—and that it’s not scientific.

My goal is ask two questions about any nonfiction I write: Do I have evidence? And how good is it? The answers might disturb me. But least I’ll proceed with intellectual honesty.

Also see:
– Won’t Get Fooled Again—Three Levels of Credibility in Self-Help Books
– Six Signs of Well-Baked Content
– Who Is an Expert, Anyway?

One comment

Comments are closed.