We’ve been running Techstars for almost a decade now. Every time we select ten or so companies for one of our accelerator programs, there’s some kind of group dynamic at play. Multiple smart people are in a room debating which of the 1,000 or so companies we’re considering should be funded. And those rooms are full of very smart people.
A while back, some Harvard and MIT researchers looked at all of the Techstars selection data and reviewed how all of those companies did over time. We have always scored companies on a scale of 1 through 4 with 4 being the highest. This ensures that there is no “middle score” and forces some polarization. It’s a bit of a simplification, but what the research found was that the average group score contained much less “signal” than what the higher scorers indicated. In other words, if the group of smart people ranked a company as follows:
Person #1 – 4
Person #2 – 3
Person #3 – 2
Person #4 – 1
Then the average here would be 2.5, which is a relatively low score. You might naturally want to exclude that company from further consideration.
However, the best way to score this for Techstars selection purposes turns out to be by counting the number of fours only. The only scores that would beat this one would have had multiple people scoring it a 4, like this:
Person #1 – 4
Person #2 – 1
Person #3 – 1
Person #4 – 4
Here the average is the same as the previous example, at 2.5. But the signal is “two fours” here, which is a much stronger signal than in the previous example. The research showed that had we been doing that all along, our outcomes would have been even better. So today we pick our companies from amongst those with the highest number of 4s, and not based on the average scores.
As humans, we have a tendency to score things and then respect the opinions of everyone in a weighted fashion. When selecting startups or looking for truly innovative ideas, you have to fight this impulse. This matches well with my own experience over a dozen years of picking startups – my Uber and Twilio picks were instinctual gut reactions from when I was a lone angel investor just learning how to run the first Techstars venture fund. Many other smart people told me Uber was a dumb idea. They would have scored it a 1, bringing the average way down. I also remember telling other investors how silly Twitter was when I first saw it. I would have scored it a 1 also. Averages kill instincts when it comes to startups. Others have learned the same.
A smart group of people with high trust for each other learns to look for signal from those who are most enthusiastic, and learns to ignore the idea of averages. It’s likely that the people voting the top score have some intuition or insight, regardless of the number of naysayers. Learn to respect the outlying data points in any group evaluation, and you’ll make your own team “group smart”.
