What kinds of instruments uncover unintended discoveries? We find a striking pattern: most often serendipitous breakthroughs come from using tools that enhance our vision—and enable seeing what was previously invisible. Following these come tools designed to separate substances—like chromatography, electrophoresis and centrifuges; and then come other tools—such as thermometers and Geiger-Müller counters. In fact, about one third of discoveries with a serendipitous moment were made alone using our vision-enhancing tools: microscopes, telescopes, x-ray methods, spectroscopes, cathode ray tubes and other particle detectors (see Figure 2.2). Each vastly expands our visual range to the world—and sparked the unexpected. The types of questions we can answer with these tools are questions about 'what is out there' . These are often very different from the questions we can solve with mathematical and statistical methods, commonly used after we know 'what is out there' . Discoveries sparked using statistics and mathematics are less likely to involve 0.14 0.27 0.21 0.20 0.15 0.14 0.11 0 .1 .2 .3 Percentage of discoveries <1600 1600–1699 1700–1799 1800–1899 1900–1949 1950–1999 2000–2022 Evolution of science: all major discoveries over time 0.00 0.03 0.12 0.14 0.15 0.17 0.17 0.19 0.46 0.24 0 .1 .2 .3 .4 .5 Percentage of discoveries econ/social chemistry physics medicine/biol astronomy Non-NP discovery NP discovery Non-NP discovery NP discovery Non-NP discovery NP discovery Non-NP discovery NP discovery Non-NP discovery NP discovery All nobel-prize versus major non-nobel-prize discoveries - by field Figure 2.1 Discoveries sparked with a serendipitous moment are decreasing over time—and are most strongly concentrated in astronomy The data reflect science's 761 major discoveries (including all nobel-prize discoveries) (Figure a), and all 533 nobel-prize discoveries compared to 228 major non-nobel discoveries—as an independent control and replication analysis (Figure b). NP stands for Nobel Prize. When we combine all nobel-prize and other major discoveries over the same time period across these five fields, we find that the aggregate shares are 2%, 14%, 16%, 19% and 37%. serendipity, as they tend to be more deliberate and require much cognitive effort— and not just making a single new unexpected observation. We find this in fewer serendipitous breakthroughs in economics—that all used statistical methods or algebra. This is an important insight: for discoveries with a serendipitous element, we can distinguish between those we uncover slower and with more analytical effort and those we unlock faster and with less effort—especially in more visually driven fields. Astronomy is a fascinating example where vision-enhancing tools have been the catalysts for big breakthroughs. In fact, we have sparked two thirds of major discoveries with a serendipitous moment in astronomy by using optical tools—a new telescope or spectroscope. In such cases, exploring the unknown leads to discoveries that seem profoundly serendipitous, simply because we had no way to predict what we are going to find in unknown terrain—like in a dark room before we suddenly install a new light. But the only way we achieve them is by designing and leveraging new tools of discovery. This brings us to a crucial insight: when a serendipitous moment arises in the discovery process, it is often the new systematic discovery tools that enable the surprising cathode ray tube microscopemicroscope particle detector spectroscope telescope x-ray method cathode ray tube microscopemicrosc ope, electron particle detec to r spectroscope telescope x-ray method Electron Semic ond uc tor amplifier Anom alies in nickel steel alloys Antineuritic vitamin Bacteria Brownian motion Cells Colloids Combating malaria Endothelium-derived Gene cdc2 Germ theo ryGraphene Mitoch ondria Streptomyc in Holograp hic method Vitr ification water techn iqu e Synthesis of radioactive Dark matter Phase contrast microscope Planet 51 Pegasi b Soft desorption ionisation methods Synthesis of RNA and DNA Catalogue of astronomical objects Expanding universe Herschel-Rigollet comet Jupiter's moons Pulsars Quasars Saturn's ringsUranus Crysta l diffraction of electrons Raman effect Röntgen radiation el emen ts (tubercul osis antibiotic) relaxing factor Figure 2.2 Discoveries with a serendipitous moment made using vision-enhancing tools 66 T HE ENGINE OF SCIENTIFIC DISCOVERY finding through exploratory research. What is more is that these tools allow other researchers to arrive at the same results, sometimes without yet a theoretical understanding about what was discovered. So serendipity is generally about using the right tool at the right moment to unlock surprise.
and methods—concrete examples We next explore discoveries with a serendipitous moment that happen right at the time of uncovering the discovery, and those that occur during the scientific process before discovery. The most popular examples focus on serendipity that arises at the moment of discovery itself—simply because they make for more exciting stories. A well-known example is the Australian Robin Warren's discovery of bacteria as the cause of gastritis and peptic ulcers. Warren was not looking for bacteria. In fact, he was studying gastric samples when he unexpectedly spotted them. Yet working as a medical doctor at Royal Perth Hospital, he and his colleague Barry Marshall had to use a high-power electron microscope and fibre endoscope to be able to see the bacteria. They had the right instruments at the right time. But they also needed method training and medical knowhow to realise the unexpected finding is important and novel—and then publish it. This was an important medical breakthrough: peptic ulcers affect hundreds of millions of people worldwide and used to be thought of as a chronic, untreatable condition. But thanks to Warren and Marshall, we can now easily treat them. Again and again, we find that when a moment of surprise emerges in discovery, it is often the instruments we employ that make it possible (Table 2.1). In these opportunistic cases—when we harness a tool in a new way or to a new question—the tool precedes both the surprise and a theoretical explanation for it. It is after a serendipitous discovery that scientists build theories around the unexpected findings.
of chance Examining science's major discoveries, we trace the year a discovery was made by the year the new method or tool was developed that enabled it, and we then split the discoveries by those involving a serendipitous moment and those that did not. Here each new method or tool is shown as a vertical line ( │) and each discovery made using it as a dot (●). Popular examples of serendipitous discoveries—like x-rays uncovered in 1895 and the first exoplanet detected in 1995—were actually sparked by two key tools: a discharge tube developed in 1875 and a fibre-fed spectrograph created in 1993—visualised in Figure 2.3 (see also Picture 2.1 ). Another famous case is Max Planck's unintentional discovery of quanta in 1900. But Planck was only able to make the surprising discovery by intentionally building on the famous experiments on blackbody radiation made possible by precise tools like spectrometers and bolometers—and a model of linear oscillators developed in 1886. Table 2.1 Serendipitous moments in nobel-prize discoveries—sparked by tools (12 examples) Discoveries (and the tool or method used to uncover them) Moment of serendipity arising: Physics (incl. astronomy) Diffraction of electrons by crystals Clinton Davisson had an accident in his lab in 1925 when a liquid-air bottle exploded that broke an experimental tube. But continuing the experiment, he surprisingly found that the distribution of the scattered electrons had changed—by applying x-ray diffraction (developed in 1912) along with other tools such as an electron gun and galvanometer. Davisson stumbled upon the wave-like nature of electrons, providing strong evidence for quantum mechanics. Bridgman pressure apparatus Percy Bridgman studied phenomena under high pressure in 1905 when the tool he used unexpectedly broke. And he immediately began designing a new sealing device that was much more efficient—and gave rise to high-pressure physics. during the scientific process before discovery Fir st exoplanet Michel Mayor and Didier Queloz were not actually searching for planets, but for brown dwarfs. But in 1995, they detected a star wobbling (that was too small to be a brown dwarf ) by using a fibre-fed echelle spectrograph (constructed in 1993)—it was the first planet discovered outside our solar system. Optical tweezers (using lasers to move objects) Arthur Ashkin accidentally left samples of viruses open overnight in 1986. He studied these larger particles using an improved method for optical trapping—that he created in 1978 and involved a microscope with a laser beam. This enabled him to discover that the particles did not actually move around freely when close to the laser beam but were trapped in the light beam. The breakthrough allows scientists to manipulate single molecules and cells. at discovery Chemistry Method for genome editing (CRISPR) Emmanuelle Charpentier was studying the bacteria streptococcus pyogenes and made the unexpected, groundbreaking discovery of an unknown molecule in 2012: tracrRNA. She was using genome-editing methods and recreating the bacteria's genetic scissors—and she had to apply improved differential RNA sequencing (developed in 2010) together with other methods such as the PCR method and electrophoresis. Methods for mass spectrometry (soft desorption ionisation methods for studying , e.g., proteins) Koichi Tanaka used trial and error and made in 1988 'a monumental blunder, which was followed by a series of fortuitous, arbitrary decisions [when he] used the glycerin instead of the acetone' to observe ions in gas form. It was uncovered by applying an improved mass spectrometer (built in 1986) along with other tools such as laser ionisation and a microcomputer. This breakthrough method has become essential to analyse complex proteins. during the scientific process before discovery c ontinued Table 2.1 continued Discoveries (and the tool or method used to uncover them) Moment of serendipity arising: Chemistry Conductive polymers (plastic that conducts electricity) Hideki Shirakawa mentioned to Alan MacDiarmid how he added 'by mistake a thousand-fold too much catalyst' to discover an organic polymer that gleamed like silver in 1977. And he also used an upgraded bolometer (created in 1961) together with other tools including x-ray crystallography and an electron microscope. This discovery has revolutionised electronics and solar cells. Water channels (transport water in our cells) Peter Agre, 'while working on a completely different problem, stumbled across a protein in red blood cells' in 1992. It turned out to be the water channel: the discovery of how water moves across cells—critical to everything from kidney function to brain health. And he made the breakthrough using immunoblotting (invented in 1979) along with electrophoresis, a controlled study and video microscopy. at discovery Medicine Cause of gastritis and peptic ulcers Robin Warren was not searching for bacteria and his colleague Barry Marshall—when cultivating the bacteria in 1982—accidentally left the agar plates too long in incubation, but by using fibre endoscopy (developed in 1979) and an electron microscope, they could identify the bacteria they unexpectedly observed. CDC genes and cell division Leland Hartwell, 'through a very amusing bit of serendipity' with his student, happened to come across photo-microscopy methods. These were the missing link in studying the cell cycle and enabled discovering the genetic control of cell division in 1971—by applying time-lapse photo-microscopy and a Coulter particle counter (created in 1960) along with a centrifuge and statistics. The discovery advanced our understanding of cancer. during the scientific process before discovery Immune response Baruj Benacerraf surprisingly found that different guinea pigs had different responses to the same antigen. The discovery revealed the cause of different immune responses in genes in 1966—by using starch block electrophoresis (designed in 1955) together with a scintillation counter and controlled study design. Hepatitis B (infectious virus affecting the liver) Baruch Blumberg 'unexpectedly discovered an infectious agent for hepatitis B while researching blood proteins' in 1965. The observation was uncovered applying agar gel diffusion (developed in 1949) and immunoelectrophoresis, a controlled study and statistics. Discovering the hepatitis B virus and the development of the vaccine has saved countless lives. at discovery The data are based on nobel-prize discoveries—with four examples highlighted in each of the three fields. E (a) (c) (d) (b) Picture 2.1 The first telescope constructed in 1608 that Galileo used to serendipitously discover the moons of Jupiter in 1610 (a, b). A new echelle spectrograph developed in 1993 that Mayor and Queloz applied to serendipitously discover the first exoplanet in 1995 (c, d). (a, b) Reproduced from NASA; NASA/Kevin Gill. (c, d) Reproduced from José Rodrigues; ESO/M. Kornmesser/Nick Risinger. When we study science's major discoveries since 1575, we see that the year we developed methods and tools is closely correlated with the year we made the discoveries—for both serendipitous discoveries and for other discoveries. We uncover an extraordinary trend: the time window between the new method and new discovery is nearly identical for both groups of discoveries. The discoveries with a serendipitous moment emerge basically within the same average time as other discoveries (Figure 2.3 and 2.5). This is an important finding because serendipitous discoveries follow the almost identical tool-driven timeline as other discoveries—regardless if we label them serendipitous or not. The strikingly similar distribution between the two groups of discoveries provides strong evidence: new tools—not just serendipity—are the driving force behind science's discoveries that make the unexpected possible . Because we must invent and apply new methods to bring about major discoveries—whether unexpected or not—means that breakthroughs have a clear method-driven structure. This includes the small share of theoretical discoveries often driven by new mathematical methods. A prominent study published in Science, Quantifying the evolution of individual scientific impact , found that 'the highest-impact work in a scientist's career is randomly distributed within her body of work'—what the authors call a random-impact rule. The influential finding is based on analysing citations over scientists' careers including nobel-prize-winning careers. Here we study directly how science's biggest discoverers actually make groundbreaking research. This leads us to a very different finding: major high-impact research does not emerge randomly at any point in a scientist's career, it specifically arises when they apply a new method or tool that enables a new way to see the world—what we can call a new methods-impact rule (Figure 2.3). 70 T HE ENGINE OF SCIENTIFIC DISCOVERY Experiencing a surprise moment does not diminish the importance of new tools— it increases it, because tools generally enable the surprise. Despite the unexpected moment, we still need to learn and implement methods to spark those breakthroughs. So do we find the strong role of serendipity as many claim? When we look at the data, we instead find a soft role of serendipity that is powered by hard tools: there are no major discoveries achieved without applying new methods or tools (Figure 2.3); there are no laypeople, without methodological training, making nobel-prize discoveries (Figure 4.1b); and only 1% of nobel-prize discoverers are not working at a university at the time of their discovery (Figure 4.3). Winning a Nobel prize is not like winning the lottery—it has predictable features. It is something we can understand by systematically studying how we generate cutting-edge research. Discoveries soon follow after new tools and if they did not, they would be distributed more randomly—as a hypothetical distribution illustrates (Figure 2.4).