Where Good Ideas Come From: The Natural History of Innovation by Steven Johnson is one of the best books I have read recently, by far. At first glance, this looks like one of those mediocre-to-terrible books that seem to dominate the intellectual landscape. But Steven Johnson is the absolute opposite of the idiotic Thomas Friedman (see also: Thomas Friedman, idiot), and a far cry from the pseudo-intellectual Malcolm Gladwell.
Johnson actually has expertise in the study of innovation. He’s written a book that delved quite deeply into a case study of John Snow’s invention, essentially, of epidemiology, and has written a book each on neuroscience and “the connected lives of ants, brains, cities, and software”. Not only is this book about innovation, but its creation backs up some of its insights.
Each chapter, in brief: (Alternatively, the RSA Animate-style video overview is excellent.)
1. Reef, City, Web
These three types of communities experience positive network effects from the interaction of their constituent elements. Rich networks of interaction maximize the value of, respectively, a nutrient-poor patch of ocean, a dense settlement of humans, and interconnected electronic information and tools.
Johnson keeps in mind at all times a range of scales and types, showing really how networks of lower-level elements come together to produce innovations at a higher level, and what conditions maximize that productivity. One of the few diagrams in the books is the following:
The fractal nature of these spaces seems important: the spaces can be rich at any scale.
2. The Adjacent Possible
At any time, there is a large but finite set of things that are possible to be invented or discovered that build on what is existing: this is the “adjacent possible”. Often in history, multiple people invent the same thing at approximately the same time: the telephone, for example, or calculus. Only very rarely does someone invent something that cannot be realized for a very long time; we call such people and ideas “ahead of their time”; e.g. Babbage’s Analytical Engine (general purpose computer) and da Vinci’s helicopter.
3. Liquid Networks
Networking and communication are very important, but the nature of connections must be in a balance between too much order and too much chaos. Johnson does not get into a lot of detail here, but mentions a few examples. Hunter-gatherers were too dispersed and chaotic; settling into farming communities increased connections, and innovation took off. (I posit that hunter-gatherers generally have a good life; early farmers’ lives sucked, motivating innovations. In addition, the class society of agriculture allowed specialization.)
By contrast, he believes that medieval society was too ordered, and the increased trade and communication of early merchant capitalism and the interaction in growing Northern Italian cities led to the innovation explosion of the Renaissance. From my reading of Eisenstein’s The Printing Press as an Agent of Change, there were a few renascences, only one of which, through print, had the staying power to establish itself as the Renaissance. Nevertheless, this highlights the importance of communication.
He also looks at pre-biotic conditions on primordial Earth, office environments, and a few other examples. I think this is more hand-wavy than the other chapters, but not terribly so. Regardless, it is very thought-provoking.
4. The Slow Hunch
Most ideas don’t suddenly appear fully formed like Athena from Zeus’s brow. Many of the great ideas are present as a hunch in the inventor’s head for many years, even decades, before the thinker presents it in a clearly formed, cohesive fashion. Johnson backs this up with examples from Charles Darwin, art, and his own work. As he points out, formative ideas in artistic fields are often pre-figured by multiple artists: a new genre “fades into view”.
I like this idea as a way to think about the origin of musical genres: who was the “true” progenitor of rap? Gil Scott-Heron? The Last Poets? Andre Williams? Likewise, many musicians can be considered progenitors of punk.
5. Serendipity
I had not known that “serendipity” was a word native to English, coined by Walpole in 1754:
Serendipity denotes the property of making fortunate discoveries while looking for something unrelated, or the occurrence of such a discovery during such a search. The word has been voted as one of the ten English words that were hardest to translate in June 2004 by a British translation company. [Wikipedia]
Often, a hunch is incomplete. It may need to meet up with another hunch to form a new idea, and this usually occurs serendipitously. This is one reason why rich networks in dense environments contribute so much to innovation.
6. Error
Somewhat related to serendipity – mistakes can turn out to be very useful to innovation. Experiments gone wrong lead to new discoveries (e.g. mold contaminating a petri dish lead to the isolation of penicillin). Sometimes inventors hold misguided assumptions and end up creating something that works for completely different reasons – here Johnson describes the case of the vacuum tube.
7. Exaptation
This term comes from biology. Stephen Jay Gould argued that most biological innovations aren’t de novo; instead, an organism uses an existing structure in a new way. For example, feathers were originally evolved for warmth. Only later were they adapted for flight. In a way, this is simply a specific case of what is in the “adjacent possible”. Many inventions use an existing thing in a new context: the printing press repurposed the technology of the wine press.
8. Platforms
Beavers are “ecosystem engineers”. They protect themselves by damming rivers; in so doing they change the habitat in a way that benefits many other species. Their lakes are “platforms” for uses by many others. On the technological level, Johnson gives us the example of GPS. Originally intended for the horrific purpose of enabling submarines to position themselves in order to properly target nuclear weapons at cities full of people, GPS is now used for myriad new purposes. Car navigation, geolocated tweets, and location-enabled cameras are all built on its broad, open platform.
9. The Fourth Quadrant
Johnson breaks innovations down by two axes: lone inventor versus product of a network and for-profit/market versus non-market. This gives us four quadrants. Our society’s myths tell us most inventions fall into the quadrant of the heroic lone inventor seeking the big payoff. Johnson collects a representative sample of innovations over the last 600 years, and examines them in batches of two hundred years.
From 1400 to 1600, communication was limited. That meant most inventions were lone affairs. Capitalism was primitive; most inventions were non-market. From 1600-1800, as communication increased, inventions shifted dramatically to being the product of communities or groups of people, and some shift towards market-based invention is visible. However, in the last two hundred years, the shift has been towards the “Fourth Quadrant”: network, non-market invention.
The self-reflexivity of the book
As the author points out, the creation of his book itself demonstrates many of these concepts. The idea for the book emerged as an “adjacent possible” from his work about John Snow. It lay dormant as a hunch for many years. Through serendipitous connections enabled by software he uses, and through discussions with diverse people, he was able to flesh out his ideas. The book examines innovation and structure at many different scales and disciplines (biology, economics, computer technology), adapting metaphors and ideas from each to the others.
Implications
(The following are my own thoughts and reactions to the book.)
The web has only accelerated the “Fourth Quadrant” trend. Innovation is facilitated by communication and networking; the internet has greatly increased the pace of innovation and will continue to do so. One can obtain a full stack of web-server software, from operating system to database, programming language, and libraries for free on the internet, each software package provided by a large community of people from around the world. Wikipedia is, similarly, a enormously successful innovation that is the work of many people in a non-market setting. Barriers to entry have fallen for individuals around the world, but more importantly costs of coordination have dropped for groups, allowing innovation in group production (almost entirely non-market, too).
One implication is that patents make less and less sense in a “Fourth Quadrant” world, certainly for information goods (and more and more goods are information goods – a future post). Secondly, the massive shift towards group, non-market innovation and production points us toward a post-capitalist world. Where Good Ideas Come From shows us how wrong free-market ideology is about how innovation actually works.
The Possible
I really like the phrase and the idea of “the adjacent possible”. In the political realm, I’m reminded of two aphorisms, the first rather conventional:
Politics is the art of the possible. – Otto von Bismarck
The second, a response of sorts, more inspirational:
The role of the Left is to expand the realm of the possible.
The most powerful way the status quo maintains itself is by stifling our imaginations and limiting our knowledge of alternatives. Learning and teaching the history they won’t tell us, from Stonewall to general strikes, is an important method of gaining mental ground. Experiments in prefiguration, especially if well-publicized, also help push the boundaries of the adjacent possible.
My own hunch: The Next Jump
One thing I didn’t mention in the synopsis of the book above is a couple biological examples he gives – starving bacteria increasing their mutation rate, damselflies reproducing sexually (instead of asexually) at the end of the season – of how systems under stress increase their variability. They are taking greater risks, and are actively searching for new solutions.
Humanity as a whole is in this position now. We are under stress, in danger, and are actively, if often unconsciously, seeking new solutions. In The Collapse of Complex Societies, Joseph Tainter says:
Certainly none can argue that industrialism will not someday have to deal with resource depletion and its own wastes. The major question is how far off that day is. The whole concern with collapse and self-sufficiency may itself be a significant social indicator, the expectable scanning behavior of a social system under stress. – p210
The rise of the counter-culture(s) in the last fifty years fits this pattern of exploratory behavior. This very book is a part of this search. The interesting thing here is our consciousness of it. Indeed, we seem to be forming a new kind of system, at a higher level. I’d like to read Steven Johnson’s book Emergence to see what parallels other emergent systems have to ours today. (Update: I’ve read and reviewed Emergence.)
Conclusion
Read this book. It is fascinating and well-written. You can look for it on isbn.nu.
If you can’t do that, at least watch Steven Johnson’s TED Talk on “Where Good Ideas Come From”.
Update (2011/03/29)
I was talking over the book with a co-worker who is more skeptical of it, particularly the “Fourth Quadrant” chapter. It seems incomplete and a bit tacked-on. It occurs to me that that chapter feels like more of a hunch than a fully-formed idea; that Johnson should develop it. With some more work and research it might make a great book of its own.
Fish Rock Road 