How Reporters Get Psychology Research Wrong

Back in 1998, the Internet was still new to most of us. Newspapers could get mileage from headlines such as Isolation Increases With Internet Use.

Yet the story behind such headlines is a reminder to keep your crap-detector up and running when reading about research on human behavior.

What made those early headlines about Internet use so misleading was a patina of credibility. After all, a published study was often cited — Internet paradox. A social technology that reduces social involvement and psychological well-being?

The abstract for this article even noted that:

… greater use of the Internet was associated with declines in participants’ communication with family members in the household, declines in the size of their social circle, and increases in their depression and loneliness.

What the reporters forgot is that all studies are not created equal. Reading beyond the abstract for this one, you discover that:

  • The study involved only 256 people from a single city in the United States — Pittsburgh.
  • Before the study was completed, 87 people dropped out.
  • Only people new to the Internet were included. Anyone with an existing online connection was excluded.
  • Depression and loneliness were measured subjectively. Rather than being assessed by an expert, people simply reported how they felt.
  • Even the heaviest Internet users reported only slight increases in depression and loneliness.
  • The above limitations were openly acknowledged by the study’s authors.  (Notice the question mark at the end of the article’s title.)

In fact, later research indicates that greater Internet use links to having more offline relationships.

The lesson here is to think critically about any research report. Ask questions such as:

  • How many people were included in the study? (In research parlance, what was the sample size?)
  • Can we generalize findings about those people to the population at large?
  • How did the researchers measure the critical variables in the study (in this case, Internet use, loneliness, and depression)?
  • Does the study actually support a cause-and-effect relationship between variables (such as Internet use causes social isolation)?
  • Was any cause-effect relationship strong enough to matter?
  • Could something else explain the relationship between variables (perhaps Internet users were depressed and lonely before the study began)?
  • Do other relevant studies point to a similar — or different —conclusion?

For more suggestions, see the entertaining Joe Hanson on Crap-Detecting Science News.

Turning Complaints Into Commitments

In their wonderful book How The Way We Talk Can Change the Way We Work: Seven Languages for Transformation, Robert Kegan and Lisa Laskow Lahey note that many workplaces are dominated by NBC — nagging, bitching, and complaining.

The good news, however, is that our complaints are actually clues to what we value. For example:

  • The person who complains about gossiping and back-stabbing among coworkers can also say that she values direct communication.
  • The person who complains about boredom at work can also say that he values being challenged.
  • The person who complains about her overwhelming work load can also say that she values completing projects on time.

The key is to ask yourself: What is the commitment behind this complaint? Posing this question can create an immediate shift in perspective and behavior.

Complaining drains energy, alienates people, and changes nothing. Stating your commitment starts a new conversation, clears a path to behavior change, and invites people to join you.

For ideas about what to do after stating your commitment, see Immunity to Change — How We Sabotage Our Best Intentions and Overcoming Immunity to Change — Run a Smart Test.

Overcoming Immunity to Change — Run a SMART Test

1736_500Why do so many of our plans for behavior change tend to fizzle out and fade away?

In Immunity to Change: How to Overcome It and Unlock the Potential in Yourself and Your Organization, Robert Kegan and Lisa Laskow Lahey offer one answer: We hold a Big Assumption — often laden with emotion and hidden from conscious awareness — that undermines our intention.

In a previous post, I gave an example. A woman says that she intends to stop gossiping at work. At the same time, she holds a Big Assumption: If I refuse to take part in gossip, my colleagues will see me as “Miss Holier-Than-Thou.” Given this assumption, it’s unlikely that her behavior will actually change.

Acting yourself into a new way of thinking

Discovering a Big Assumption is liberating. Once it’s openly stated, you can detach from it, think critically about it, and experiment with a new behavior.

Kegan and Lahey recommend that you run a SMART test of the Big Assumption by acting against it. The acronym stands for Safe, Modest, Actionable, Research-based, and Test:

  • Safe and Modest — Ask yourself: What is one small thing that can I do differently to challenge my Big Assumption? Remember that you can still hold on to your Big Assumption most of the time. The goal is to suspend it temporarily, and only in certain circumstances.
  • Actionable — Choose a behavior that you can do immediately.
  • Research-based — Remind yourself that you’re simply running a Test. This means acting against your Big Assumption in a small way and then observing what happens as a result. The goal is simply to collect data — not to “improve” yourself.

An example

For instance, the woman who values honest communication could set a new intention: When I’m tempted to repeat a rumor about my boss, I will remain silent.

This plan involves a modest change in her behavior. It’s also something that she can do immediately. In addition, it’s relatively safe: Her statement involves only one behavior in a specific setting.

According to Kegan and Lahey, the SMART test is about “gradually building up a psychological space between ourselves and our Big Assumptions in order to move them from subject to object, where we can look at them, turn them around in our hands, and consider altering them.”

Even small changes in our Big Assumptions can enlarge our sense of possibility — and start breaking down our immunity to change.

Immunity to Change — How We Sabotage Our Best Intentions

078796378XWilliam Perry, an educational psychologist who taught at Harvard, once described his response to people who asked him for help: “I listen very hard and ask myself, What does this person really want — and what will they do to keep from getting it?

In How the Way We Talk Can Change the Way We Work: Seven Languages for Transformation, Robert Kegan and Lisa Laskow Lahey refer to our “hidden immune system.” This system creates psychological antigens to change, effectively locking our current behaviors — even those that create suffering — firmly into place.

Kegan and Lahey aim to expose this hidden immune system. By understanding how it works, we can stop defeating ourselves and start changing for good. This involves a four-stage process.

1. Describe your commitments

Complete this sentence: I am committed to the value or the importance of … .

For example: I am committed to the value or the importance of more open and direct communication at work.

2. Describe what you’re doing — or not doing — that undermines your commitments

Kegan and Lahey quote this example from a woman who said that she wanted her colleagues to communicate openly and honestly and avoid back-biting: I don’t speak up when people are violating the norm I value. Silently, I collude in it being OK to talk behind one another’s back.

3. Describe your competing commitments

Now imagine yourself actually changing the behaviors you listed in Step 2. Does this lead to any fear or discomfort? Those feelings probably stem from a hidden commitment that contradicts and competes with your first commitment.

For instance, the woman who said that silently colluded with back-biting discovered that she was also committed to “not being seen as the Righteous Crusader, Castrating Bitch, or Miss Holier-Than-Thou… .” She wanted coworkers to feel comfortable with her—not to see her as a self-appointed enforcer of rules about how to communicate.

4. Describe the Big Assumption behind the competing commitment

This is an assumption about what will happen if we violate the competing commitment. For example:

I assume that if people did see me as a Righteous Crusader, Castrating Bitch, or Miss Holier-Than-Thou, then I would eventually be completely shunned, have no real connections in my office other than the most formal and functional, and actually I’d find work a nightmare from which I couldn’t wake up.

The Big Assumption reveals the raw core of our immunity to change. Once this assumption is openly stated, however, it starts to lose its power. We’re free to suspend the assumption, question it, and actively test it.

Real change becomes possible once we look at our assumption rather than look at the world through it.

Monitor What Matters — Measurement as a Path to Behavior Change

I’m enjoying Psychological Self-Help by the late Clayton E. Tucker-Ladd. This site grew out of his work as director of the Counseling & Testing Center at Southern Illinois University, where he taught courses in applying psychology to daily life.

Psychological Self-Help — which you can read online and download for free — is the result.

One of Clayton’s aims was to help students carry out “self-help projects” — experiments in personal change. “The task is to find out what self-help methods work for you” Clayton wrote; “that is research!” Chapter 2 of Psychological Self-Help gives detailed instructions for doing this.

Why monitoring matters

Self-help projects start with monitoring your current thoughts, moods, and behaviors.

Monitoring matters because our subjective impressions about behaviors that matter — such as how often we exercise and how much we sleep — are often inaccurate. Monitoring cuts through illusion and reveals truth.

The trick is to monitor in a way that’s informative and simple. There are two major options.


Maybe you read this classic Lifehacker post about Jerry Seinfeld’s advice for young stand-up comics.

Seinfeld told Brad Isaac that “the way to be a better comic was to create better jokes and the way to create better jokes was to write every day.”

Seinfeld also revealed a way to develop that daily writing habit:

He told me to get a big wall calendar that has a whole year on one page and hang it on a prominent wall. The next step was to get a big red magic marker.

He said for each day that I do my task of writing, I get to put a big red X over that day. “After a few days you’ll have a chain. Just keep at it and the chain will grow longer every day. You’ll like seeing that chain, especially when you get a few weeks under your belt. Your only job next is to not break the chain.”

The essence of this technique is counting a specific behavior. Calendars are one tool for doing this. You can also make tick marks on an index card that you carry in a pocket or purse. Or, use a behavior-tracking app.


Some things that matter — including emotions — are hard to count or measure objectively. No problem. In any moment, you can simply rate your emotional state on a scale from 1 (not sad, afraid, mad, or glad) to 10 (very sad, afraid, mad, or glad).

The power of awareness

“Every problem — and every desired behavior or feeling — can be measured by counting or rating,” Clayton wrote. “By measuring the problem every few hours or maybe every day or two, you can tell how serious the problem is and if you are changing.”

If you’re writing a book that aims at behavior change, then suggest counting and rating.

Your readers might discover — as Jerry Seinfeld did — that monitoring by itself can change behavior. This is the point where awareness and action merge to create a surprise benefit.

Three Problems With Life Coaching

Do you you remember Stuart Smalley, the character played by Al Franken on Saturday Night Live? He looked in a mirror and recited an affirmation:  I’m good enough, I’m smart enough, and — doggone it — people like me.

I hear a little of Stuart’s voice in life coaches who advocate affirmations, tout the law of attraction, or lead you in a guided visualization (ignoring recent research about this practice). Even goal-setting — a major focus in life coaching — has its own issues.

Going deeper, I find three reasons to question life coaching as a path to behavior change.

Lack of certification

If you want to call yourself a life coach, you can do that today. There’s no formal certification process and no widely accepted definition of life coaching.

Lack of specialization

There’s a common assumption that you can work with a life coach to meet any goal or acquire any skill — even if the coach has never met that particular goal or developed that particular skill. I’d prefer to get coaching from someone with demonstrated competence in achieving a specific outcome.

Lack of training in mental health diagnosis

Many life coaches make a clear distinction between coaching and psychotherapy. In practice, however, this distinction is hard to maintain.

Conditions such as depression and anxiety can be subtle and difficult to diagnose. How can a life coach without training in mental health diagnosis know when to refer you to a psychotherapist?

What to ask a life coach

I worked with a life coach briefly. At first, it was exhilarating. The conversation centered unconditionally on me. I launched into a passionate and unfocused soliloquy that went on for weeks.

In the end, nothing much about my behavior actually changed.

OK, so I take responsibility for that. But in retrospect, I wish I’d started life coaching by asking a few questions:

  • How did you become a life coach?
  • What kinds of outcomes can I expect from life coaching — and what outcomes can I not expect?
  • What kind of process will we use, and how do you know that process works?

If I ever work with a life coach again, these items will lead the agenda.

To learn more: What Can Coaches Do For You? from the Harvard Business Review.

Joe Hanson on Crap-Detecting Science News

Joe Hanson, host of It’s OK To Be Smart, posted this useful video. Please view it before claiming that your next article, book, blog post, or presentation is “based on science.”

Joe cautions us to watch out for:

  • Headlines that consist of a question.
  • Headlines that include quotation marks.
  • Words that reveal uncertainty, such as link, association, correlation, suggest, and baffled.
  • The word breakthrough: True breakthroughs are rare.
  • The word controversial: Controversy does not always mean accuracy.
  • Articles that reinforce stereotypes.
  • Articles that appear on sites with lots of advertising: You could be reading “churnalism” or “click bait.”

In addition:

  • Distinguish between press releases (marketing material) and good journalism (fact-checked and balanced).
  • Read to find out if the article was peer-reviewed and published — or simply presented at a conference.
  • Read closely to determine whether the reporter talked to the main researcher.
  • Remember that “shit flows uphill”: misinformation spreads like wildfire, especially online.
  • Don’t claim that science “proves” anything: Findings are based on current data and can be revised when new data emerges.

Bottom line: Blend curiosity with skepticism. Keep an open mind — but so open that your brains fall out.

To Engage Readers, Make Your Core Messages Easy to Find

What readers ultimately take away from a non-fiction, “how-to” book is one big idea and five to ten supporting ideas that lead to positive behavior change. Let’s structure our books so that readers find those ideas without friction.

Here we can take a cue from good newspapers, which place key messages in predictable places — headlines and lead paragraphs. We can do something like this in a book manuscript.

The key is to remember where readers will go first and craft those sections with care.

1. Write a table of contents that sells your book

When considering whether to buy and read your book, many people will flip to the table of contents. Reel them in by giving them something of substance here. In addition to chapter titles, list the subheadings within each chapter as well.

Also write titles and subheadings that truly inform. Peter Bregman does this in 4 seconds: All the Time You Need to Stop Counter-Productive Habits and Get the Results You Want. So do Jason Fried and David Heinemeier Hansson in Rework and Remote.

For more inspiration, study the headlines in good news sources such as the New York Times and BBC. Copyblogger offers some excellent guidance on headlines as well.

2. Make the first few pages shine

After scanning the table of contents, people are likely to land on the first few pages of text — the introduction or first chapter. This is arguably the most important part of your book.

If you wrote a compelling proposal for your book, then draw on the work you’ve already done. Look in particular at the overview section of the proposal. This makes the case that you have an original and effective solution to an urgent problem that readers face.

3. End with a bang

Close each chapter or part of your book with a list of key take-aways. The folks from 99u do this well in their books, including Make Your Mark: The Creative’s Guide to Building a Business.

End your book as whole with another summary. Round it out by suggesting concrete ways for readers to act on your ideas:

  • Guide readers to define projects and next actions.
  • List Tiny Habits for readers to adopt.
  • Provide scripts — examples of what readers could say in a conversation or write in an email.

When we guide readers from ideas to action, we create a legacy that will outlast our words.

Organizing 11,000 Ideas — Here’s How Robert Pirsig Did It

One of my favorite books is Zen and the Art of Motorcycle Maintenance by Robert Pirsig. Less well known is Pirsig’s second book, Lila: An Inquiry Into Morals, which was published 17 years after Zen.

During those years, Pirsig took notes for Lila on small slips of paper. He used them like index cards, writing one idea on each slip and filing all the slips in a big box. (Many people — including Robert Greene and Ryan Holiday — still prefer this method.)

Pirsig ended up with over 11,000 slips. Most of these fell logically into various categories. But many did not.

To save his sanity, Pirsig created five special categories for rogue ideas. Consider using these categories whenever you organize any large body of notes:

  • Unassimilated is a holding zone for recent notes that still need to be reviewed and filed.
  • Program is for instructions about what to do with the rest of your notes. If Pirsig had visions for a whole new set of categories, for example, he filed those ideas here.
  • Crit slips describe all the notes you want to destroy and the reasons for destroying them. Often these ideas came to Pirsig in moments of despair. Rather than immediately trashing his notes, however, he simply noted his first thoughts and filed them here to review later.
  • Tough is for important notes that don’t fit in any existing category.
  • Junk is for notes that initially seemed important but now look useless. “Most of the slips died there but some reincarnated,” Pirsig wrote, “and some of these reincarnated slips were the most important ones he had.”

The beauty of these categories is that they allow you to keep notes organized while your ideas are still incubating. In particular, you’ll make room for the “junk” ideas that later emerge as shining gems.

Readers Will Tweak Our Instructions — And That’s OK

Idea entrepreneurs sometimes labor for years to create step-based processes. These authors load their books with methods, tips, strategies, applications, and action plans. Yet the inevitable fact is that people will tweak our precious processes to fit their personal style.

Instead of berating our audiences for “corrupting” our content, let’s let go of our attachments and allow for individual differences.

As a case in point, consider David Allen’s book Getting Things Done: The Art of Stress-Free Productivity.

Avoid an “us versus them” mindset

Like many best-sellers, GTD (the acronym for David’s book and the method it presents) inspires zealots and heretics.

Some people swear by GTD and describe it as the Bible for productivity.

Others berate the GTD community as a cult and dismiss the method as hopelessly complex.

(To see what they’re talking about, get a GTD overview.)

Such polarizing reactions are unnecessary. Underlying them is an all-or-nothing, love-it-or-leave-it mentality: either take an author’s ideas in toto as absolutes. Or, reject them entirely.

What’s most reasonable and realistic is a middle ground. I, for example, count myself as a GTD enthusiast. Yet there are many suggestions in David’s book that I don’t implement. They just don’t make a difference for me. I adhere to the spirit rather than the letter of his ideas.

Clarify core distinctions

Seth Godin notes that the core content in any business book — he calls it the “recipe” — can usually be reduced to 2 or 3 pages. The rest of the book is the “sell” — persuading you to actually do something differently.

The recipe often boils down to a small list of concepts that are truly original, uniquely presented, or especially useful. When it comes to GTD, for instance, here’s what I ultimately take away:

  • Your mind is for having ideas, not holding them. To keep your head clear, capture ideas as they occur to you and put them in writing, even when you’re on the run. Jot them down on index cards or “sticky notes” and toss them into your in basket. Another option is to open up a plain text file on your smart phone, tablet, or computer and make a quick note.
  • Use the “three D’s” to clear out your in basket. Take each item, one at a time, and choose wether to delete it, do it now, or defer it to a later time. Note deferred items on your calendar or a list. (Some people add a fourth “D” — delegate.)
  • Distinguish between projects and next actions. Projects are outcomes that require more than one action to achieve. Next actions are physical and visible — things that you can actually do today, such as making a phone call, running an errand, or sending an email.
  • Do a weekly review. Keep a list of all the active projects in your life and then ask: What is the very next action I will take to move each project forward? (David recommends that you keep many lists, but projects and next actions are the only ones I use.)

Your content has a center of gravity, an irreducible essence. What is it? Put that recipe in writing, and keep it short. Then do your readers a favor by placing it in prominently in your book — as an introduction, summary, or both.

Encourage readers to experiment

Four crucial words are missing from most instructions that I read — your mileage may vary (YMMV). The larger and more complex your method, the more these words apply.

The Tiny Habits program from BJ Fogg perfectly embodies this message while delivering a powerful and focused set of instructions. BJ offers a tested recipe for creating a new habit. He also cautions that you may go through several tries before settling on a behavior that actually sticks.

This is the kind of message that readers can run with. Our goal is a balance of rigor and permission. Achieving it is no small feat.