A lot of people in tech talk about measurable outcomes. Set a goal, figure out how you can tell you’ve succeeded, and channel your efforts into optimizing for that metric. It’s a principle that has helped me focus well, but I’ve been thinking about how this mindset makes people revere the quantifiable and discount the intangible change that can’t (yet) be measured: slow change that happens in the space of thoughts and ideas.
Daniel Kahneman’s Thinking Fast and Slow has sold 1.5 million copies, but his full impact is arguably far greater. Anchoring and loss aversion are part of our vernacular, and people who haven’t even heard of him change their everyday decision-making based on these ideas. As my friend described (on reading Kant), “For the first half of the book I thought he wasn’t saying anything new…and then I realized why.”
The kinds of metrics we have right now (“1.5 million copies”) seem like reasonable proxies, but they don’t account for most of the work that ideas do. Most changes in thinking don’t come from Kahneman-type figures, but from nudges—reading an article on the way to work, hearing a thought in passing, shaped by collective discourse. In the class on structural change I’m taking with Roberto Unger, people complain (rightfully) that he never fills in the practical details for his imagined social/political/economic change. He’s not concerned at all with the actual barriers to implementing a new trade policy or redesigning private property and contract. He’s been called a “preposterous romantic“—but that’s precisely his value. The point is that the work he does deals in a different currency than the concrete change that we can see. He changes worldviews. He plants an inkling that there are plausible alternatives to the supposed rules of the world. We’ll probably never be able to attribute a specific policy to him; we’ll never really know all the ways people act and think differently because his ideas crossed their minds.
A good idea is a lens: it’s not just one piece of knowledge, but something that shapes how you see everything. I had this thought when talking with friends about going into industry vs. academia. A lot of people describe the allure of industry as “having more impact.” You can talk about how the feature you worked on reached X thousand people. There are lots of reasons why someone should choose industry over academia, but this seems to me like more of a platitude than a real distinguishing factor. It conflates impact with measurable impact. While industry is directed at scaling an idea, the purpose of academia (in principle) is generating novel ideas in the first place. There are no immediate visible outcomes, and few papers or professors ever reach notoriety. But influence isn’t a binary indicator (it’s not “has a Wikipedia page, or doesn’t”)—it’s impossible to know just how many people read a good paper and change the way they see their work. Imagine how that idea seeps into the way they approach their problems, and imagine how many people read those people’s work. And so on.
Metrics, if chosen correctly, make it clear what the goal is and hold us accountable to the progress in the right direction. That’s why we might want to strive towards ways to measure impact in the space of ideas. In academia, citation count as a blunt proxy for impact leads to all sorts of adverse incentives. It rewards incremental advances in a popular area as much as influential papers. Studies on e.g. citation graph-based indices might give us slightly better ways to recognize / encourage the right kind of work, along with a better understanding of how paradigm shifts and collective changes in thinking happen. But until we have these tools, thinking consciously in terms of intangible impact will get us partly there.
Thanks to Kshitij, Timmy, and Basil for feedback! :) Email me if you have thoughts/comments.
Subscribe to get updates when I write new posts.