Teaching Myself to Fail: Failure as Data Acquisition

Summary:

Failure is a necessary consequence of growing and doing something new.
You should approach failure the way scientists collect data. A failure is just another piece of data that's informative about how you should advance your overall goal.

This is a guide on how to get good at failing, something most people struggle with. This is worth pursuing. Being good at failing is underrated.

Everyone hates failing

Around this time last year, I had dinner with a group of highly accomplished young professionals. Many of them had PhDs and the others were pretty high up the corporate ladder. At one point in the evening, one of the corporate guys' story about overcoming his fear of heights by learning how to skydive led to a round of "share your fear". There was one thing everyone seemed to have in common: a fear of failure. I was both relieved and surprised to hear this - relieved because it wasn't just me who feared failure and surprised because these highly accomplished people don't look like they have had to deal with failure much. I wonder whether being so accomplished makes the prospect of failure look even more daunting. Or perhaps there's nothing special about these people despite glitzy titles and flashy suits.

I have spent much of my teenage years and early twenties seeing failure as something to avoid. I have avoided putting yourself "out there" because of a fear of rejection, shame, embarrassment - in short, a fear of failure. What's worse, at some point I got really good at convincing myself that hiding is better than taking any risk of failure.

But here's the thing: failure is the necessary consequence of growing and doing something new. Not experiencing any failures is perhaps the only failure you should be afraid of, because it means everything you do is driven by the desire to avoid it. And avoiding what you think constitutes a failure really doesn't bring any benefits; only missed opportunities, regrets, and endless unanswerable "what if...?", "if only...", and "I wonder what would have happened if...?" These are questions you don't want to be asking yourself when you're 80.

Fail like a scientist

Knowing that you should embrace failure is one thing. Actually doing things that put yourself in the risk of failing is another. How do you bridge this gap between understanding and doing?  Something that has helped me close this gap is to approach failure the way scientists approach unsuccessful experiments.

Most scientific endeavours begin with some broad goal (e.g. to find a cure for disease X). To achieve this goal, scientists construct various hypotheses. These are theories about the social or physical world that they want to test out (e.g. vaccine Y is a cure for disease X). To test the hypotheses, scientists collect data. These are observation points required to examine whether the theory is correct (e.g. the data might be health outcomes of patients who got the vaccine). Scientists then draw conclusions from these data (e.g. if 80% of the patients still get sick, then they reject the hypothesis that vaccine Y is a cure). In the scientific approach, a data point or outcome - regardless of whether it's a failure (a sick patient) or a success (a cured patient) - is valuable because it yields new information that tells us whether we should reject our hypothesis. Failures may seem like momentary setbacks but they move us along the process of achieving the goal by eliminating hypotheses that we now know won't help get us to the goal.

If you adopt this scientific approach, then failing is merely the process of acquiring data to reject certain hypotheses that you want to test, either about yourself or the world. Failure is not the gap between where you are now and where you thought you'd be by now - that's the space of life. The key thing about the scientific approach is that "unsuccessful" outcomes are rejections of hypotheses (that vaccine Y is a cure), not failures to accomplish the overall goal (to find a cure for disease X). If anything, unsuccessful outcomes inch us towards the goal by reducing the potential pool of hypotheses. And so, a willingness to fail is the greatest feedback system you can create for yourself.  

Examples

What would adopting this scientific attitude to failure in my personal life look like? One thing most people probably want is a fulfilling career. Here's what being willing to fail in order to get those things might look like:

  • Goal: to find a fulfilling career
  • Hypothesis: Career X might be fulfilling.
  • Data: sample careers through internships and placements.
  • Potential outcomes: Success - fall in love with job; Failure - end up hating the internship and having to quit early.

Another thing most people want is an enriching romantic relationship.

  • Goal: to be in a loving and enriching long-term romantic relationship.
  • Hypothesis: Person X might be someone that I could have this relationship with.
  • Data: Dates and conversations with person X
  • Potential outcome: Success - have a wonderful time and arrange another date -> don't reject hypothesis; Failure - personality clash, nothing to talk about -> reject hypothesis.

We conventionally think of unsuccessful outcomes (a bad internship, a boring date) in terms of failure. But they're really just data points that inform you about what you don't like rather than things you do like: "anti-experiences". And if knowing what you don't like is as important as knowing what you do like, then maybe we should stop favouring pro-experiences over anti-experiences.

Why don't we experiment more in life?

This iterative trial and error process has long-recognized value in science. However, most adults fail to do this in our daily lives - this includes the PhD scientists at the dinner party. It's that gap between understanding and doing, between theory and practice. We understand that experiments are fact-finding missions that push us towards greater understanding in the field of science, but we don't approach our lives with this degree of scientific curiosity. Why is that? Is it because it would make life seem too clinical and scientific? Is it because life is too precious to experiment with? Or is it just because we're scared of the outcomes of these experiments?

The fact that we don't like experimenting in life is all the more strange considering that experimentation was our natural state as children. Alison Gopnik, a psychology professor at UC Berkeley, argues that children think like little scientists by continually constructing and then revising theories about the structure of the world and the way the world works, including the social world. At the heart of learning is theory revision. Children hold theories of the world, we test those theories against experience to see if they are true, and we change our beliefs accordingly. Theory revision is a part of growing up and becoming you. Christine Legare, a psychology professor at UT Austin, provides evidence of the mantra "there's no success without failure". Legare says that children learn most effectively when they encounter evidence that is inconsistent with their prior experience and when they consider alternative explanations. Put in terms of our "fail like a scientist" framework, inconsistency (a failed outcome) and explanation (exploring other hypotheses, like other potential jobs or partners) promote theory revision (movement towards the overall goal).  

How to adopt a scientific approach to failure

I hope I've convinced you that being okay with failing is a good skill to master. Here are some practical tips on how to do it.

Step 1: Diagnose your attitude to failure

What happens when you find out you've made an error? Do you shut down, go inward, and try to make excuses for that mistake? If so, you have a negative definition of failure. Or do you laugh it off, own up to your mistake, and try to figure out where you went wrong? If so, you have a positive definition of failure.

Step 2: Find your goal and hypotheses

Think about all the goals you're trying to achieve and the corresponding hypotheses that you need to test out to achieve that goal.

Step 3: Imagine what failing would look and feel like

How would life look like if you didn't achieve this goal in the next 6 months/year? How would it feel if you didn't achieve this goal in the next 6 months/year?  

Step 4: Collect your failures

Start collecting your failures. In a similar way to how we're taught to make goal lists, make a list of failed outcomes/anti-experiences. It might say something like this: I want to have 3 failed internships, where failed means I learn that I don't like that job. Or: I want my book proposal to be rejected from 20 agencies this years. You might want to create a "CV of failures" in addition to your main resume. The idea here is to gamify failure so that the number of failed outcomes is a metric of effort.

Step 5: Fail publicly

Share your failures with the world. One of my favourite podcasts is Elizabeth Day's "How to fail" where she gets guests to share 3 failures that shaped them who they are. The bottom line is that failures provide us with an architecture of success. If anyone tells you that this is woo-woo new age-y junk, show them that "proper" academics do this too. One of the most delightful academic experiences I've had recently is stumbling on a CV of failures by Johannes Haushofer a professor at Princeton. Haushofer says that he created this document to give people some perspective: "Most of what I try fails, but these failures are often invisible, while the successes are visible."

What Haushofer describes is a classic publication bias problem. On social media, we curate and edit our feeds so that we often only see people's good days. In academic journals, editors are more likely to publish studies with positive results than those with null results. We don't get to see the failures. But the failures make the successes. By sharing these failures with the world, you're doing your bit to normalize failure. And you're actively adopting the identity of a person who is okay with failing, maybe even someone who's proud of their failures.

Good luck with failing. It's totally underrated.