Book cover of How Innovation Works by Matt Ridley

How Innovation Works

by Matt Ridley

22 min readRating: 4.1 (3,084 ratings)
Genres
Buy full book on Amazon

In "How Innovation Works," Matt Ridley takes readers on a captivating journey through the history of human ingenuity. This book explores the fascinating world of innovation, examining how groundbreaking ideas and inventions come to life and shape our world. From the steam engine to smartphones, Ridley delves into the stories behind some of humanity's most significant technological advancements.

Ridley challenges the common misconception that innovation is a straightforward process driven by lone geniuses. Instead, he argues that innovation is a messy, collaborative, and often unpredictable endeavor. By examining numerous examples throughout history, the author reveals the complex web of factors that contribute to groundbreaking discoveries and inventions.

This book is not just a collection of interesting anecdotes about famous inventions. It's a deep dive into the nature of innovation itself, exploring the conditions that foster creativity and the obstacles that can hinder progress. Ridley's insights offer valuable lessons for individuals, businesses, and policymakers alike, providing a fresh perspective on how we can encourage and nurture innovation in our rapidly changing world.

The Nature of Innovation

Innovation is Complex and Collective

One of the central themes in "How Innovation Works" is that innovation is rarely, if ever, the result of a single person's efforts. Ridley challenges the popular notion of the lone genius inventor, arguing instead that innovation is a collective process involving many minds working together, often unknowingly.

To illustrate this point, Ridley examines the invention of the atmospheric steam engine, a pivotal development in the Industrial Revolution. While we might be tempted to attribute this invention to a single person, the reality is far more complex. Three inventors - Denis Papin, Thomas Savery, and Thomas Newcomen - all produced working models of the atmospheric engine around 1700. It's difficult to determine who was truly first or how much each inventor influenced the others.

This example demonstrates that even the most revolutionary inventions are built upon a foundation of existing knowledge and ideas. The atmospheric steam engine didn't emerge from a vacuum; it was the result of ongoing discussions and experiments in scientific circles at the time. Papin and Savery, both educated men, refined their thinking through correspondence with other inventors. Newcomen, who built the most successful version of the engine, relied on advances in blacksmithing technology to complete his machine.

Ridley extends this principle to other innovations throughout history. For instance, while Thomas Edison is credited with inventing the light bulb in 1879, more than 20 other inventors had patented similar devices in earlier decades. All of these thinkers were responding to ideas and technologies circulating at the time.

This collective nature of innovation doesn't diminish the achievements of individual inventors. Rather, it highlights the importance of collaboration, communication, and the free exchange of ideas in driving technological progress. It also underscores the fact that innovation is often an incremental process, with each inventor building upon and improving the work of others.

Innovation is Messy and Unpredictable

Another key insight from Ridley's book is that innovation is rarely a neat, linear process. Instead, it's often messy, unpredictable, and filled with false starts and unexpected turns.

Ridley points out that some of the most significant breakthroughs in history weren't the result of deliberate discovery or sound scientific theory. Instead, they developed piecemeal over time through random chance, trial and error, and practical problem-solving.

This is particularly evident in the field of medicine, where many life-saving practices emerged from risky experiments and unconventional thinking. Ridley cites the example of smallpox inoculation, a precursor to modern vaccination. The practice of "engraftment" - deliberately infecting oneself with a small amount of smallpox to gain immunity - seems barbaric by today's standards. Yet, it was a crucial step in the development of vaccines and saved countless lives.

Another example is the chlorination of water supplies. In 1908, Dr. John Leal added chloride of lime to Jersey City's water supply in a desperate attempt to combat cholera outbreaks. At the time, this was considered a radical and potentially dangerous move. However, it proved highly effective in reducing disease rates, leading to widespread adoption of water chlorination.

These examples illustrate that innovation often involves taking risks and challenging conventional wisdom. Progress doesn't always follow a straight line from hypothesis to experiment to proven theory. Instead, it can involve leaps of faith, educated guesses, and learning from unexpected outcomes.

Ridley argues that this messy, unpredictable nature of innovation is something we should embrace rather than try to eliminate. While it can be uncomfortable and risky, it's often the source of our most significant breakthroughs.

Innovation Thrives on Incremental Improvements

While we often focus on revolutionary breakthroughs, Ridley emphasizes that much of innovation consists of small, incremental improvements over time. These gradual enhancements, while less dramatic than major inventions, are crucial in refining and perfecting new technologies.

The author uses the evolution of transportation technology to illustrate this point. The development of the locomotive, for instance, didn't happen overnight. In the early 1800s, numerous prototypes were developed, each making small improvements in speed, safety, or reliability. Names like the Salamanca, the Puffing Billy, and the Sans Pareil might sound quaint now, but each represented a step forward in locomotive design. It was only after many iterations that Robert Stephenson's Rocket emerged in 1829, capable of transporting 13 tons of cargo at 30 miles per hour.

Similarly, the modern automobile is the result of countless incremental improvements over more than a century. From Isaac de Rivaz's early hydrogen-powered engine in 1807 to Jean Joseph Lenoir's petroleum-based design in 1860, to Nikolau Otto's four-stroke engine in 1876, each iteration brought small but significant advancements. Karl Benz's Motorwagen in 1894 and Henry Ford's Model T in 1909 further refined the concept, making cars more powerful and accessible to the masses.

Ridley argues that this pattern of gradual improvement is common across all fields of innovation. Even in today's fast-paced tech world, major advancements often come through a series of small updates and refinements rather than sudden, dramatic leaps.

This perspective on innovation has important implications. It suggests that persistence and continuous experimentation are just as important as moments of brilliant insight. It also highlights the value of learning from failures and setbacks, as these often provide the lessons needed for future improvements.

Innovation Isn't Always Tangible

Ridley makes an important point that not all innovations are physical inventions. Some of the most influential innovations in history have been ideas or concepts that changed how we think or approach problems.

The author uses the example of the humble potato to illustrate this point. While not an invention in the traditional sense, the adoption of the potato as a staple food in Europe was a significant innovation that had far-reaching effects on society and economics. Despite initial resistance and superstition, the idea of eating potatoes gradually spread across the continent, providing a nutritious and hardy crop that helped fuel population growth and urbanization.

Another example of an intangible innovation is the Arabic numeral system. The idea of using 1s, 2s, and 3s for counting might seem obvious to us now, but it was once revolutionary. Developed by Indian scholars around 500 AD and later adopted by Arab traders, this system didn't gain a foothold in Europe until the 1200s. The positional nature of Arabic numerals (where a digit's value depends on its position in a sequence) opened up new possibilities in mathematics, accounting, and scientific discovery.

These examples show that innovation isn't just about creating new gadgets or machines. It's also about introducing new ideas, methods, or ways of thinking that can transform society. Sometimes, these conceptual innovations can have an even more profound impact than physical inventions.

Ridley argues that recognizing the importance of these intangible innovations is crucial for understanding the full scope of human progress. It reminds us that creativity and problem-solving can take many forms, and that some of the most important breakthroughs might not be immediately visible or tangible.

Drivers of Innovation

Communication Technology: A Catalyst for Rapid Change

Ridley dedicates significant attention to innovations in communication technology, highlighting how our desire to connect and share information has driven rapid technological advancement.

The author traces the evolution of communication technology from the telegraph to modern digital networks. The story begins with Samuel Morse's invention of the telegraph in 1843, which allowed messages to be transmitted almost instantly over long distances for the first time. This was followed by the telephone in 1876 and wireless radio in the 1890s, each representing a leap forward in our ability to communicate.

What's striking about these communication technologies is how quickly they were adopted and how profoundly they changed society. Ridley notes that the first telegraph line was completed in 1844, and by 1855, there were 42,000 miles of lines in the United States alone. By the end of the 1870s, telegraph cables stretched across the Atlantic and Pacific oceans. Similarly, broadcast radio went from a single station in 1900 to being the dominant form of public communication by the 1930s.

The rapid adoption of these technologies reflects their transformative power. They changed how information spread, how business was conducted, and even how wars were fought. Each new communication technology compressed time and space, bringing distant people and places closer together.

This trend has continued and accelerated with the advent of computers and the internet. Ridley points out how quickly computer technology has improved, citing Moore's Law - the observation that the number of transistors on a microchip doubles about every two years, while the cost halves. In 1975, the average computer chip had 65,000 transistors. Today, that number is in the billions.

The internet, built on this ever-improving computer technology, has revolutionized communication once again. It has democratized information access and content creation, allowing anyone with a connection to reach a global audience. This has led to the rise of search engines like Google and social media platforms like Facebook, which have become some of the most influential companies in the world.

Ridley argues that this rapid innovation in communication technology has had profound effects on society, politics, and the economy. It has changed how we work, how we socialize, and how we consume information. It has also shifted power dynamics, giving enormous influence to those who control these communication platforms.

The author's exploration of communication technology serves as a powerful example of how innovation in one area can have far-reaching consequences across society. It also illustrates how our fundamental human desire to connect and communicate can drive technological progress at an astonishing pace.

The Role of Chance and Serendipity

Throughout "How Innovation Works," Ridley emphasizes the important role that chance and serendipity play in the innovation process. While we often think of inventions as the result of deliberate, focused effort, many breakthrough discoveries have occurred by accident or as unexpected side effects of other research.

One example Ridley provides is the discovery of polytetrafluoroethylene (PTFE), better known by its brand name Teflon. In 1938, a scientist researching refrigerants accidentally created PTFE when he stored tetrafluoroethylene gas at sub-zero temperatures. The resulting substance was unusually stable and heat-resistant. While it didn't work as a refrigerant, it found numerous other applications, from non-stick cookware to components in atomic bombs.

Another example is the development of DNA forensics. This innovation began not as an attempt to solve crimes, but as a method for diagnosing diseases. In 1977, scientist Alec Jeffreys at Leicester University discovered that DNA patterns were unique to individuals, much like fingerprints. It was only later, when local police were struggling with a difficult murder case, that this discovery was applied to forensic science.

Ridley argues that these stories of serendipitous discovery are not rare exceptions, but a common pattern in innovation. Many groundbreaking innovations begin with a chance occurrence or an unexpected insight. The key is that these lucky breaks are recognized and then developed further through experimentation and application in different contexts.

This perspective on innovation has important implications. It suggests that fostering innovation isn't just about funding specific research projects or setting particular goals. Instead, it's about creating environments where chance discoveries are more likely to occur and where people have the freedom to explore unexpected findings.

Ridley points out that throughout history, universities, trading hubs, and major cities have consistently produced novel innovations. These environments bring together people with diverse expertise, perspectives, and cultures, increasing the likelihood of chance encounters and cross-pollination of ideas.

The author's emphasis on serendipity in innovation serves as a reminder that progress often comes from unexpected directions. It underscores the importance of curiosity, open-mindedness, and the willingness to pursue ideas that might initially seem irrelevant or impractical. In doing so, Ridley encourages a more flexible and open approach to innovation, one that leaves room for the unexpected and embraces the unpredictable nature of discovery.

The Bottom-Up Nature of Innovation

One of the key arguments Ridley makes in "How Innovation Works" is that innovation often emerges from the bottom up, rather than being directed from the top down. This challenges the common notion that major breakthroughs primarily come from government-directed research or large corporate R&D departments.

To illustrate this point, Ridley recounts the story of two British airships built in the 1920s. The government commissioned both a state-run lab and a private firm, Vickers, to each build an airship. The result? Vickers produced the R100, a light, fast, and efficient aircraft that successfully completed a round trip to Canada. Meanwhile, the government lab's R101 was heavier, more costly, and crashed on its maiden voyage to Pakistan, killing 48 passengers.

This example, Ridley argues, demonstrates that direct government oversight and control isn't always the best approach to fostering innovation. While government-funded research can certainly lead to important discoveries, it often takes the ingenuity and market-driven approach of private enterprise to turn these discoveries into practical innovations.

The author cites the development of the internet as another example of this dynamic. The basic components of computer networking were indeed created by a government agency (the Defense Advanced Research Project Agency). However, the world wide web didn't become a household necessity until private firms like Cisco began experimenting with and commercializing the technology in the 1980s and 1990s.

Ridley argues that this bottom-up approach to innovation is often more effective because it's more sensitive to the needs and desires of everyday people. Government projects and large corporations can sometimes be slow to adopt new, outside-the-box ideas or may not recognize the practical potential of certain discoveries.

This is not to say that top-down innovation never works, but Ridley contends that it's often less efficient and less responsive to real-world needs. He points out that even large, established companies can fall victim to this problem. The author uses the example of Kodak, which failed to capitalize on the digital photography revolution despite one of its scientists building an early digital camera in 1975. The company's leadership couldn't see beyond their established film business, allowing smaller, more nimble companies to take the lead in the new technology.

Ridley's argument for bottom-up innovation has important implications for how we think about fostering creativity and progress. It suggests that creating an environment where individuals and small teams have the freedom to experiment and take risks is crucial. It also highlights the importance of market forces in driving innovation, as they often provide the incentives and feedback necessary to turn good ideas into practical, widely-adopted innovations.

This perspective challenges policymakers and business leaders to reconsider how they approach innovation. Rather than trying to direct or control the process from the top, Ridley's work suggests that they might be better served by creating conditions that allow bottom-up innovation to flourish. This could involve reducing regulatory barriers, encouraging entrepreneurship, and fostering a culture that values experimentation and is tolerant of failure.

Challenges to Innovation

Resistance to Change

While innovation drives progress and often improves our lives, Ridley points out that new ideas and inventions frequently face significant resistance. This resistance can come from various sources, including established industries feeling threatened, ideological opposition, or simply a general fear of change.

The author provides several historical examples to illustrate this point. One particularly striking case is the controversy surrounding margarine in the late 19th and early 20th centuries. When margarine was first invented in 1869, it faced fierce opposition from the dairy industry. Fearing competition, the National Dairy Council launched a campaign against margarine, even fabricating studies claiming it was dangerous. By the 1940s, two-thirds of American states had banned this now-common food item.

This "butter battle" demonstrates how even seemingly innocuous innovations can provoke strong reactions when they threaten established interests. Similar patterns have played out in many other industries. Ridley notes how horse breeders fought against tractors, ice-harvesters tried to stifle refrigeration technology, and some musicians initially wanted to ban radio stations from playing recorded music.

The author argues that one common tactic used to resist innovation is to sow fears about safety and security. He cites the ongoing controversy over genetically modified organisms (GMOs) as an example. Despite the potential of GMOs to provide cheaper, more nutritious food to people worldwide, groups ideologically opposed to genetic modification have lobbied hard against their production, often citing questionable evidence about their dangers.

Another way innovation can be held back is through overly aggressive application of intellectual property laws. While patents and copyrights are intended to incentivize innovation by allowing creators to profit from their ideas, Ridley argues that excessively long copyright terms can actually hinder progress. He points out that in the United States, copyright terms have been steadily extended from the original 14 years to the current life of the author plus 70 years. Such long terms, he suggests, keep good ideas locked away from potential new uses long after they've stopped benefiting the original creator.

Ridley's exploration of resistance to innovation serves as a reminder that progress is not always a smooth or universally welcomed process. It often involves overcoming entrenched interests, challenging established ways of thinking, and navigating complex regulatory landscapes. Understanding these challenges is crucial for anyone seeking to introduce new ideas or technologies.

The author's insights also highlight the importance of education and clear communication in promoting innovation. By helping people understand the benefits of new technologies and addressing concerns transparently, innovators may be able to reduce resistance and gain wider acceptance for their ideas.

Moreover, Ridley's discussion of intellectual property laws raises important questions about how we balance the need to reward innovation with the need to allow ideas to circulate freely. It suggests that policymakers may need to reconsider current approaches to ensure that our legal frameworks are truly fostering innovation rather than stifling it.

The Innovation Slowdown in the West

In the latter part of "How Innovation Works," Ridley turns his attention to the current state of innovation, particularly in Western countries. He paints a somewhat concerning picture, suggesting that innovation in many sectors has slowed down significantly in recent decades.

The author begins by asking readers to imagine the world in 2050. Will we see a medical system where gene therapy and stem cell treatments have eliminated most diseases? Will artificial intelligence pilot our cars safely and efficiently? Will nuclear fusion provide us with unlimited clean energy? While these visions might seem plausible given the rapid pace of change in the 20th century, Ridley argues that current trends make such dramatic advances less likely.

To support this view, Ridley points out areas where progress has stagnated. In transportation, for example, he notes that the average speed of commercial aircraft hasn't increased since 1958 - we're still flying at about 600 miles per hour. While there have been improvements in areas like fuel efficiency, the fundamental technology remains largely unchanged.

The business world, too, shows signs of reduced dynamism. In the United States, new businesses made up 12 percent of the economy in 1980, but only 8 percent in 2010. In Europe, the situation is even more stark: of the continent's 100 most valuable businesses, only two are younger than 40 years old.

Ridley argues that this slowdown is partly due to a shift in focus from innovation to protection of current profits. Many established industries seem more interested in maintaining their market position than in pursuing risky new ideas. This risk aversion is compounded by increasing regulation and longer patent terms, which can create barriers to new entrants and limit the free exchange of ideas.

However, the author notes that this innovation slowdown isn't universal. While Western countries may be losing their innovative edge, other parts of the world - particularly China - are seeing a boom in new ideas and technologies. Ridley points out that Chinese firms like Tencent and Alibaba are at the forefront of growth industries like social media and financial services. Moreover, Chinese universities are making significant strides in cutting-edge fields like gene editing and artificial intelligence.

This shift in the global innovation landscape raises important questions about the future of technological progress and economic growth. Ridley suggests that if Western countries want to maintain their innovative edge, they'll need to recapture the spirit that drove previous waves of invention and discovery.

The author argues that fostering innovation in the future will require several key elements:

  1. A willingness to take risks: Companies and individuals need to be prepared to pursue bold new ideas, even if they might fail.

  2. Hard work: Innovation often requires long hours and persistent effort.

  3. Open exchange of ideas: The free flow of information and collaboration across disciplines and borders is crucial for sparking new insights.

  4. Supportive government policies: While top-down direction isn't always effective, governments can create environments that encourage innovation through appropriate regulation and incentives.

  5. A dash of luck: As we've seen throughout the book, serendipity often plays a role in major breakthroughs.

Ridley's analysis of the current state of innovation serves as both a warning and a call to action. It challenges readers to consider what might be holding back progress in their own fields and what can be done to reignite the innovative spirit. At the same time, it offers hope by reminding us of humanity's incredible capacity for creativity and problem-solving when the right conditions are in place.

Conclusion

In "How Innovation Works," Matt Ridley provides a comprehensive and nuanced exploration of the innovation process. Through numerous historical examples and insightful analysis, he challenges many common misconceptions about how new ideas and technologies come into being.

Key takeaways from the book include:

  1. Innovation is a collective process: Rather than being the work of lone geniuses, most innovations arise from the combined efforts of many individuals, often working independently but building on each other's ideas.

  2. Innovation is messy and unpredictable: Progress rarely follows a straight line from theory to application. Instead, it often involves serendipitous discoveries, false starts, and unexpected applications of ideas.

  3. Incremental improvements matter: While we often focus on revolutionary breakthroughs, much of innovation consists of small, gradual improvements that refine and perfect new technologies over time.

  4. Innovation isn't always tangible: Some of the most important innovations are ideas or concepts that change how we think or approach problems, rather than physical inventions.

  5. Communication drives rapid innovation: Our desire to connect and share information has been a powerful force behind technological progress, leading to remarkably rapid adoption of new communication technologies.

  6. Bottom-up innovation is often more effective: While top-down, government-directed research has its place, many of the most impactful innovations emerge from individuals and small teams responding to real-world needs and market forces.

  7. Innovation faces resistance: New ideas and technologies often face opposition from established interests or those fearful of change. Overcoming this resistance is a crucial part of the innovation process.

  8. The pace of innovation may be slowing in the West: Ridley argues that many Western countries are seeing a slowdown in innovation across various sectors, while countries like China are experiencing an innovation boom.

Ridley's work serves as both a celebration of human ingenuity and a call to action. By understanding the true nature of innovation - its complexity, its unpredictability, and the conditions that foster it - we can better appreciate past achievements and work to create environments where future innovations can thrive.

The author challenges readers to embrace the messy, collaborative nature of innovation. He encourages us to be open to new ideas, to be willing to take risks and learn from failures, and to foster environments where creativity and experimentation are valued.

At the same time, Ridley's analysis raises important questions for policymakers, business leaders, and individuals alike. How can we create regulatory frameworks that encourage innovation without sacrificing safety or ethical considerations? How can established companies maintain their innovative edge in the face of disruption? How can individuals contribute to and benefit from the ongoing process of innovation?

Ultimately, "How Innovation Works" leaves readers with a deeper appreciation for the complex web of factors that drive human progress. It reminds us that innovation is not just about creating new gadgets or technologies, but about constantly finding new and better ways to solve problems and improve our lives. By understanding and embracing the true nature of innovation, we can all play a part in shaping a more innovative and prosperous future.

Books like How Innovation Works