Mapping or Graphing the Discovery Process

Michael E. Gorman and Christy Nilsen

In an effort to make generalizations across a wide range of problem-solving situations, Herbert Simon and his colleagues created a kind of graph that depicts a problem-solver's path toward a goal. While the problem-solver is working, he or she is asked to talk aloud, and this protocol is used as the basis for each person's graph. This methodology was developed using simple laboratory tasks like the 'Tower of Hanoi' problem, where it provides the researcher with a fine-grained record of the problem-solving process.

But can such a mapping scheme be adapted to the staggering complexity of scientific discovery and technological invention? Interestingly, instead of protocoling actual scientists working on their discoveries, Simon and his colleagues have preferred to compare novices and experts working on complicated textbook problems.

Computational maps

To study discovery, Simon and a group of colleagues have pioneered a different methodology: computational simulations of the discovery process. This approach is best exemplified by Kulkarni & Simon, who wrote a program called KEKADA that modeled Krebs' discovery of the ornithine cycle. In effect, the program creates a kind of problem-behavior graph when you chart its progress.. These 'program-graphs' can, in turn, be compared with the steps the scientist took and also with the performance of subjects asked to solve a similar problem. The authors of these programs do not necessarily see them as mapping devices, however; they sometimes argue that they are 'discovery machines.' In other words, instead of just helping us understand science, such programs might eventually replace scientists--or at the very least, show how the scientist(s) ought to have behaved.

Shrager and Langley point out that such programs lack 'embodiment': they fail to capture the visual and tactual aspects of scientific problem-solving. For example, Krebs relied heavily on a tissue-slicing technique learned in Otto Warburg's laboratory. A program cannot, of course, simulate this extremely important aspect of science.

But Gooding has shown how problem-behavior graphs can represent the hands-on as well as intellectual aspects of Michael Faraday's discovery processes. Gooding & Addis (this volume) combine computational and graphing approaches to mapping by developing a kind of 'expert assistant' program which will help the mapper clarify concepts like 'construe'. Every time the mapper thinks a construal has occurred, the program asks for all the inputs and outputs, and uses this information to identify additional instances of construal. If the program's predictions are at variance with the mappers, then the concept needs to be clarified. Gooding and Addis show how the rigor of the computational approach can be combined with fine-grained details of the discovery process. What is less clear is whether and how the expert assistant will facilitate our understanding of the hands on' and visual aspects of discovery and invention.

Socio-technical Graphs

Shrager and Langley also critique computational simulations on the grounds that, unlike scientists, they are not 'embedded' in a network of social negotiations . Sociologists like Brannigan have argued that discovery is a social label. For example, the question of whether KEKADA 'actually discovers' is moot; what is interesting is why its creators label its outputs a discovery.

From a sociological perspective, Bruno Latour is developing a graphing methodology that depicts the way in which a budding discoverer or inventor recruits allies into a social network. Latour emphasizes creating graphs from different perspectives, to represent the network as viewed by competing actors (or 'actants', as he makes no distinction between animate and inanimate actors). Here again, we see the need for some kind of visual summary and re-organization of a complex process. In Latour's case, the map or graph is always based on a narrative; therefore, it is primarily a useful way of highlighting and summarizing a path that is already outlined in a narrative. A sharp, visual summary can facilitate comparisons among the network-building activities of different actants.

Unlike some of the new sociologists of scientific knowledge, we believe that one cannot understand the innovation process without understanding the content of the inventions and discoveries. Like the sociologists, we believe traditional studies of creativity have underestimated the extent to which social negotiations and 'hands on' practices have shaped innovation. We are still experimenting with ways of incorporating social networks into our maps, which focus far more on cognitive processes. The point to keep in mind now is that 'social' and 'cognitive' are not mutually exclusive categories. Inventors and scientists take knowledge from others, transform it, and re-introduce it into a network of negotiations. Eventually, this whole process will need to be mapped if we are to understand discovery and invention.

To review and summarize earlier efforts:

  1. Simon and colleagues have provided rigorous mapping techniques which can be applied across a wide range of problems; however, these techniques tend to omit both the embodied and embedded contexts of discovery. In other words, to achieve Simon's level of generalizability, one has to omit important details of the discovery process.
  2. Gooding and Addis are showing how a Simon-like approach can be extended to a fine-grained analysis of an actual discovery that does not ignore these contexts. However, Gooding and Addis' scheme has been developed on a single scientist; it remains to be seen whether it will lose generalizability as a consequence.
  3. Latour has provided a system for graphing the growth of socio-technical networks; however, his system is based on narrative accounts, not fine-grained studies. Furthermore, it is not clear how Latour would handle the intellectual and technical content of a discovery or invention.

Our mapping methodology attempts to address these shortcomings while retaining as many advantages of preceding mapping schemes as possible. To force us to keep generalizability in mind at the outset, we decided to map the competition between three telephone inventors However, we wanted to conduct this comparison at a fine-grained level of detail. We include primary sources in our maps--through extensive quotes in notecards as well as reproducing original sketches, to capture the visual aspects of the invention process. Therefore, the reader need not accept our interpretation of the invention process but can work to reconstruct the event for him- or herself. By including so much original material in our maps, we provide a buffer against distortion of the facts in ways an account that merely selects the "relevant" data cannot. To show all this detail on one level would make our maps hopelessly complex. Therefore, another innovative feature of our mapping system is that it is hierarchical, with lower level maps representing increasing amounts of detail until at the lowest levels we have all the data we have been able to uncover.

To explain our approach, we are going to resort to a series of analogies, no one of which accurately captures our approach but each of which captures important elements. One analogy is to the archaeological reconstruction of the history of a city like Pompeii or Troy. Dozens of stages in its development have to be reconstructed from partial foundations overlaid by later buildings, from pottery shards and incomplete written records. Similarily, we have to reconstruct stages in an invention's development from surviving artifacts, scattered records and later reconstructions that have 'overlaid' earlier events.

Our primary focus in the current study is on the inventors. We are conducting a kind of archaeology of the mind--not of the Freudian sort, where one probles the depths of the psyche, but of a cognitive sort, where one tries to reconstruct the mental processes of a discoverer or inventor. This is perilous work, because one can never be sure that one has reconstructed the actual mental processes; instead, what we settle for is a plausible reconstruction that fits the availiable. data Other archaeologists can unearth new data, or show that we have lumped sketches together that really belong to different stages or communities.

Some scholars (mostly sociologists) will even argue that our focus is wrong: what we ought to reconstruct is the development of the social networks and alliances that produce the invention. Our answer is that one needs both the cognitive and the social.

What is required, then, is a method that will allow us to trace the inventor's intellectual and social trajectories, blending them into a single story. Furthermore, we need a way of making the data and assumptions on which this story is based accessible to others, so that they can challenge us and make alterations. If we can do all of this, we will have developed a method by which a wide range of discoveries and inventions can be compared, including those produced by teams.

Virtually every upper-level box has a sub-map associated with it, and many of these sub-maps in turn have even lower and more detailed maps associated with them. The use of sublevels allows us to incorporate all of the visual and many textual materials into the map while retaining the ability to generalize and summarize. The map stores textual material in notecards linked to each sketch box, allowing the information to be organized by visual cues and also within the hierarchy. In this way, one may uncover only as much information as one currently wishes to explore while preserving additional detail for later viewings of the map.

To pursue another analogy, let us compare our kind of map to a geographical one--say, one of the new ones on CD-ROM. A person looking at a map of North America could click on Chicago and 'zoom-in' for a close-up view. Associated with this view might be all kinds of information about the city. Similarily, someone looking at one of our upper level maps can click on boxes with sketches in them to obtain increasingly detailed views of the developments that lead to this upper-level sketch.

Geographical maps can also use different shapes and colors to highlight different information, e.g., geographical contours, population centers, bodies of water, etc. Similarily, we use different shaped boxes on our maps to denote goals, conclusions, thought experiments and other features (see Mapping Symbols). A mapreader may choose to scan the map for only a particular color, symbol or kind of line. For example, one can search for national parks. Within a level on a cognitive map, the different box shapes or symbols allow the reader to scan the map and select only the thought experiments or only the conclusions. Thus, the map can contain more information than a reader may be interested in at any one viewing. In the same way, our cognitive map contains more information than the reader may wish to confront at a particular time.

Another analogy is to a flowchart, or one of Simon's problem-behavior graphs. Unlike geographical maps, but like problem-behavior graphs, our "Master Map" shows the development of ideas over timewhich flows downward, though it might just as easily flow across.

Return to the Help Page
Last Modified: Friday, 24-Jun-2005 14:14:04 EDT