CROSSROADS—Identifying Viable “Need–Solution Pairs”: Problem Solving Without Problem Formulation

1. Introduction and Overview

Problem-solving research and formal problem-solving practice begin with the assumption that a problem has been identified. The problem-solving process then involves a search for a satisfactory or optimal solution to that problem. In this article, we propose something rather different. Often, we think, a need and a solution are or can be discovered as a need–solution pair. When this is so, problem identification, if done at all, comes only after that discovery.

To illustrate what we mean, consider a firm employee walking through a trade show “just to see what is new.” The employee encounters a booth offering new payroll-processing software (a solution) that claims to address certain needs better than earlier software. The employee then mentally assesses the need–solution pair on offer for utility to the firm. “I wasn’t thinking that we had any payroll-processing deficiencies, but now I recognize that we do and also recognize that this new software may resolve them.” Alternatively, instead of this type of “new-to-me” discovery, the employee may be stimulated to identify a new-to-the-world need–solution pair: “This payroll-processing approach gives me an idea to create a better design for another situation entirely, one that I was not thinking needed improvement.”

As a real-world example of a new-to-the-world innovation conceived of in this manner, consider Bernard Sadow’s description of need–solution pair identification when inventing the rolling suitcase (Sadow 1972):

Mr. Bernard D. Sadow, now 85, had his eureka moment in 1970 as he lugged two heavy suitcases through an airport while returning from a family vacation in Aruba. Waiting at customs, he said, he observed a worker effortlessly rolling a heavy machine on a wheeled skid.

“I said to my wife, ‘You know, that’s what we need for luggage,’ ” Mr. Sadow recalled. When he got back to work [Sadow worked at a luggage company], he took casters off a wardrobe trunk and mounted them on a big travel suitcase. “I put a strap on the front and pulled it, and it worked,” he said. (Sharkey 2010)

This same process of need–solution pair identification without problem identification or formulation in advance of “solving” is used in a great deal of informal personal problem solving as well. “Look at that bike baby carrier in the store window. It looks very sturdy and safe. I wasn’t thinking I needed a baby carrier, but that one could be a very useful solution element in a new pattern of daily activity I just now envision: I could use my bike to carry my baby to preschool instead of using the car.”

In the need–solution discovery and evaluation process just illustrated, there is no independent problem identification or formulation step at all. Instead, the problem solver discovers or envisions a new and possibly useful need–solution pair and then compares it with a preexisting situation to determine if the new one is preferred. One could dub the inferior older arrangement “a problem” once viewed relative to the new arrangement. However, this problem is only specifiable post hoc. In net, what we see is a problem-solving process that begins with identification of viable need–solution pairs rather than with the formulation of a problem.

What we term a “need–solution pair problem-solving process” is presently unknown to problem-solving literature in the management and organization fields as well as formal problem-solving practice. We think it flourishes in real life but can offer no evidence beyond illustrative examples at this point. In this article, we will postulate and discuss the need–solution pair problem-solving process, proposing that under some conditions it will be superior to problem-solving processes built around the solving of problems specified ex ante. We will conclude by suggesting that these matters merit further investigation.

In §2 we briefly review core assumptions in relevant literature on problem identification and formulation and problem solving. In §3, we introduce the concept of need and solution landscapes and propose the concept of viable need–solution pairing. In §§4 and 5 we discuss the richness and complexity of real-world need and solution landscapes, and the advantages and costs of widespread landscape search. In §6 we further illustrate need–solution pair discovery. In §7 we propose next steps for need–solution pair theory development and research, and we note useful links to other research streams. In §8 we offer suggestions for improvements to present-day problem-solving practices attainable with increased awareness of the value of need–solution pair problem solving. In §9 we briefly conclude the paper.

2. Problem Formulation-Related Literature

In this section we provide a distillation of critical assumptions and choices in the problem-solving literature regarding problem formulation. Our purpose is to provide an informed contrast to the need–solution pair perspective we discuss in this paper.

First, we note that those engaged in both formal practice and research on problem solving assume that a problem must first be identified and formulated for solving, and then addressed via problem-solving processes of various types. This framing is ubiquitous. With respect to research, a statement by Volkema (1983, p. 640) is representative. He argues that solvers would be better off identifying an optimal problem formulation at the outset because of the irreversibility of the problem-solving process: “Because the amount of information needed to change a decision is much greater than the amount needed to make it initially …, reformulation of the problem becomes less likely once a particular formulation is selected and pursued. This places added pressure on decision makers to avoid premature closure and to select ‘optimal’ problem statements.” With respect to practice, Spradlin (2012, p. 85), for example, explains that “Indeed, when developing new products, processes, or even businesses, most companies aren’t sufficiently rigorous in defining the problems they’re attempting to solve and articulating why those issues are important. … Many organizations need to become better at asking the right questions so that they tackle the right problems.” Dictionary definitions are in line with such a sequence of events. Thus, in the Oxford Dictionary, a problem is defined as “a matter or situation regarded as unwelcome or harmful and needing to be dealt with and overcome.”¹

The assumption that problem identification or formulation precedes solving holds both for situations where a problem is fixed at the beginning of solving and remains unchanged, and situations where the initial problem specification is progressively reformulated or re-specified as problem solving proceeds (e.g., Schon 1983, Smith and Eppinger 1997, Thomke and Fujimoto 2000, Kurup et al. 2011).

2.1. Identifying the “Underlying” Problem to Be Solved

Problems can and often do precede problem solving when they press themselves upon one’s attention independent of and prior to any solution being evident (e.g., “My head hurts” or “This experiment failed”). When problems do present themselves, experts and everyday solvers alike realize that the troublesome manifestation may be only a symptom of a cause that lies deeper (see Emirbayer and Mische 1998). For this reason, in-depth exploration seeking underlying causes, i.e., the “real problem,” is routine in many fields. For example, a skilled doctor visited by a patient complaining of a headache will routinely consider possible underlying causes, rather than simply treating the headache. Engineers and scientists also seek underlying causes: “Yes, that valve is corroded and must be replaced, but we must understand why the valve corroded and solve that underlying problem, too.” “Yes, that physics experiment failed to provide the answer we expected: what is the underlying reason?” Students of problem solving also understand this. As Schwenk and Thomas (1983) and others explain, problem formulation is an activity that not only involves problem finding, but also in-depth exploration for underlying causes for the problem as observed. The underlying cause discovered then becomes the problem to solve. When solved, this will also solve the surface manifestation initially observed.

Practical methods have been developed by many to encourage problem formulators to look below surface symptoms (e.g., Ishikawa 1968, 1976; Volkema 1983). Thus, Ishikawa developed and promoted cause-and-effect diagrams (sometimes called “fishbone diagrams”) to help problem solvers to organize their searches into deeper causes that might underlie a problem. His technique and variations have been widely used in industrial process design and elsewhere. In cases when failures (i.e., problems) of a specific type would be predictably extremely costly, as in a nuclear reactor failure or airplane malfunction, analytic methods have been developed to search for likely underlying causes of failure before any actual failure occurs. Fault tree analysis is an example of such a method (e.g., International Electrotechnical Commission 2006).

2.2. Formulating Problems for Solving

Once a problem has been identified, it is then commonly “formulated” for solving. In this regard, Simon (1973) drew a crucial distinction between well-structured and ill-structured problems, noting that only well-structured problems were suitable for algorithmic solution. He also pointed out that problems that are initially ill structured are converted into well-structured ones by the efforts of problem formulators (Simon 1977, p. 309): “In general, the problems presented to problem solvers by the world are best regarded as ill-structured problems. They become well-structured problems only in the process of being prepared for the problem solvers. It is not exaggerating much to say that there are no well-structured problems, only ill-structured problems that have been formulated for problem solvers.”

The core of problem solving via algorithmic search is creating a problem statement that contains a well-structured problem specification. Once that has been accomplished, algorithmic search can be applied. An example of an algorithmic method applicable to many well-structured problems is NK landscape search (e.g., Levinthal and Posen 2007, Ghemawat and Levinthal 2008). NK search begins with a well-structured problem statement. A precisely describable solution landscape is then searched by first casting rough-mesh digital nets over the whole landscape to discover the approximate locations of desirable solution “peaks.” This is followed by the casting of progressively finer-mesh nets over promising areas to discover a more satisfactory or even optimal solution. Rules such as “breadth-” or “depth-first” for visiting edges and vertexes in a graph have been formulated and examined for their ability to reach certain goals or results within constraints (for a comprehensive review of algorithmic search, see Corneil 2004).

In management, a particular emphasis is put on the economics of search—the efficiency and effectiveness of search strategies and procedures (e.g., Fleming and Sorenson 2004, Laursen and Salter 2006, Garriga et al. 2013). Often, in the case of managerial problems, solvers do not expect to find the optimal solution, given problem complexity and the level of resources available to conduct the work. Under these conditions, “satisficing search” algorithms are used to identify any solution deemed satisfactory, where no distinction is being made among alternative satisfactory solutions (Simon 1978, Greiner 1996, Greiner et al. 2006).

Also within the management literature, researchers have studied how problem formulation activities are and should be organized (Lyles and Mitroff 1980, Baer et al. 2013, Ben-Menahem et al. 2015). Thus, Mintzberg et al. (1976) found that, under conditions of high uncertainty in decision making, problem solvers work very incrementally, iteratively formulating problems and rapidly following up with search for solutions. When the identified problems are very uncertain or ill structured, Pich et al. (2002) argue, problem solvers should explore a variety of solution paths independently and then select the best path after careful ex post analysis. More generally, in what might be called literature on the “management of the unknown,” Pich et al. (2002) and Abbott (2005), among others, suggest ways of coping with unforeseen influences that inevitably arise as novel problem-solving projects unfold.

2.3. The Difficulties of Problem Formulation

Obtaining problem solutions that function successfully when applied requires that problems as formulated adequately represent the real-world situation. Scholars understand that problem formulation is difficult, and that formulations used may often be defective. Frequently, this is because the formulations do not contain all relevant problem-specific and contextual information. When this is so, the result can be failure when the solution developed is applied to the intended real-world contexts.

Incomplete or inaccurate information can affect a problem-solving process when a problem is initially identified or formulated, and/or when it is reformulated to convert it from an ill-structured to a well-structured form. Simon (1977, pp. 239–240), explains:

Consider the Missionaries and Cannibals problem. [This is a classic problem used in Artificial Intelligence research. The puzzle posed follows: three missionaries and three cannibals must cross a river using a boat that can carry at most two people. If missionaries are present on a river bank, they cannot be outnumbered by cannibals. Also, the boat cannot cross the river by itself without people on board.] As it is usually discussed in artificial intelligence research, it is a well-defined problem. The objects are specified, together with the legitimate moves, constraints, and starting situation. The state space is, in fact, very small, consisting of only 10 allowable states …

Contrast this “laboratory” Missionaries and Cannibals problem with a real-life problem. In the real-life problem, the number of persons the boat will hold may only be determinable by trying it, or at least examining it; and the fact that the boat leaks may only be discovered after it is loaded. The real environment may permit alternative ways of crossing the river—by a ford a few miles further up, on a bridge, or by airplane. The actors may be joined by several more missionaries (or cannibals!) while the action is going on.

In real life there is not a single, static, well-defined problem, but a constantly changing problem whose definition is being altered by information that the actors recover from memory and by other information obtained from the environment’s response to actions taken …

von Hippel and Tyre (1996b), and Tyre and von Hippel (1997) demonstrated the consequences of problem formulations that transmitted inadequate information to problem solvers. They studied the two novel process machines designed and constructed in response to fixed specifications/fixed problem statements. They found that, when the machines developed according to these specifications were placed into factory service in the real world, they failed repeatedly and required design modifications to function as intended. The most common underlying cause for the field failures was found to be a lack of full information about the real-world operating environment in the original problem statement. As Simon noted, this problem is not easily rectifiable by making a fixed problem statement more complete. Information needed by problem solvers that a problem statement “should” contain often changes as problem-solving work progresses.

Wallin et al. (2013) and Sieg (2012) show that the problems associated with inflexible initial problem statements grow more severe when solution seekers are separated from solvers by organizational barriers. This was the case in the situation explored by Tyre and von Hippel (1997) just discussed: Machine function specifications (problems) were formulated by a production group in a machine user firm and outsourced to an independent machine design firm. This also is commonly the case for problem-solving tournaments, where a fixed problem is posed by a “problem owner” to external solvers, and awards are given for the best solution to that fixed problem (Wallin et al. 2013).

Finally, note that some “wicked” problems cannot be formulated at all because those with a stake in the formulation have nonoverlapping criteria for acceptable solutions. This situation is especially acute when solutions acceptable to one set of solution users create inescapable negative externalities to other potential users. For example, the enjoyment of national forests by individuals practicing motor sports can create a serious noise pollution problem for those seeking quiet enjoyment. If externalities are severe, and they can be when conflicting uses both require access to the same common resource, there may simply be no acceptable formulation that can be agreed upon (Rittel and Webber 1973).

3. Viable Need–Solution Pairs

We have seen from our review of core assumptions that the study of problem solving in the management and organization fields generally assumes that formulation of a problem to be solved precedes the search for a solution. Recall that this holds both for situations where a problem is fixed at the beginning of solving and remains unchanged, and situations where the initial problem specification is progressively reformulated or re-specified as problem solving proceeds (e.g., Schon 1983, Smith and Eppinger 1997, Thomke and Fujimoto 2000, Kurup et al. 2011). We have also seen that problem formulation can be problematic.

In this section, we open the way to a new view of the problem-solving process that may avoid problem formulation altogether. To do this we must first make our terminology clear. In psychology, economics, and marketing, needs are clearly distinguished from wants. Needs reflect a state of felt deprivation of some basic requirement for living such as food, shelter, belonging; wants are desires for satisfiers of needs. As the underlying motivator for wants is needs, we use that term in our discussions here. Although needs are often considered an individual-level construct, they are potentially collective at the levels of groups, communities, and organizations (see Hamilton 2003). For example, the need for transportation from home to work can be individual as well as collective.

We propose that it is conceptually useful to think of a pool of need-related information as the contents of a need landscape that, along with the contents of a solution landscape, are drawn upon for problem formulation and solving. As has been noted, and as we will further see, these pools of information can be very rich and complex. They may contain tacit elements as well as encoded information held in the minds of individuals. They may contain information about a setting or context of use not known to anyone, such as the state of seaworthiness of a boat. For example, the contents of a need landscape for a surgeon would include the working environment plus professional and personal experiences and views and information, all content that could potentially be used in problem solving.

The metaphor of “landscape” makes it intuitive to represent both need and solution landscapes as three-dimensional surfaces on which the position of a point on the x and y axes specifies the location of information related to a specific formulatable need or solution. On the need landscape, the z-axis “height” of a point represents the benefit to be attained by satisfaction of the need at position x, y. On the solution landscape, the z-axis “height” of a point represents the cost required to provide the solution located at position x, y.

Problem solving then consists of making a link between a specific point on a need landscape and a specific point on a solution landscape. We term these linked points a “need–solution pair.” This pair is viable if the benefit derived from a solution is equal to or higher than the cost of providing it in that pairing. For example, consider the thick arrow in Figure 1 that connects a specific point on the need landscape with a specific point on the solution landscape. Suppose the selected location on the need landscape represents an individual’s state of thirst, and the selected location on the solution landscape represents a specific source of drinkable water. Then that need-solution pair is viable if and only if the reward associated with that particular point on the individual’s need landscape (i.e., the intensity of the need to slake thirst) is equal to or higher than the cost of providing the solution (i.e., potable water) to the individual at issue from the particular source pointed to on the solution landscape. Traveling to drink pure water from a mountain spring may not be a viable pairing to the individual’s need in the case of urgent thirst, but getting a less-delicious glass of water from the kitchen tap may be.

Next we observe that “formulating a need” and “formulating a problem” are really a single process. The distinction is between content that falls within the need landscape and content that falls within the solution landscape as a consequence of the formulation choices made. Consider the following example of successive need or solution statements:

I need special lumber for my construction project.

The lumber I need is available in Helsinki.

I need to arrange a lumber-buying trip to Helsinki.

I will arrange to travel to Helsinki.

I will arrange to buy the lumber.

I will arrange to transport it to my construction site.

In this sequence as given, the partition between need and solution landscapes (i.e., the gap in the sequence of statements above) is the formulation of the need to arrange a lumber-buying trip to Helsinki. Problem-solving steps then follow. However, the division can equally well be located between any other two adjacent statements. Thus, we could also set our formulated need statement to be the first item in the listed sequence: “I need special lumber for my construction project.” Successive steps in the problem-solving process are then carried out on the solution landscape: “I find I can buy the lumber I need in Helsinki. I must travel to Helsinki to arrange the purchase. I will arrange travel to Helsinki. I will buy the lumber, and transport it to my construction site.”

4. Value of Rich Landscape Search

As we have said, the traditional practice of separating need- or problem-formulation steps from problem-solving steps converts what could be a single continuous activity into two modules, separated by a problem statement. This practice does have value as a coordination mechanism: A problem statement can be used to transfer a task to other entities (i.e., individuals or separate groups of solvers) supplying them with a fixed goal and important related information for their problem-solving work (March and Simon 1958, Baldwin and Clark 2000, Terwiesch et al. 2002). This is common in firms, for example, when individuals in the marketing research department formulate a need for a new product, create a problem statement, and then transfer the work of developing a responsive product to developers.

However, formulating a single fixed problem also has major costs: it cuts problem solvers off from much need landscape richness. As a result, what could be better solutions for the solution user become inaccessible—because they are “on the other side” of the problem-formulation/problem-solution divide. For example, consider a firm that uses chemical X in its production process. The firm might formulate a problem requiring solution thus: “We need a way to produce chemical X for significantly less than our current cost of $Y per kilogram.” Solvers can then conduct a search on the solution landscape thus defined, for points that create a viable pairing with that fixed problem. However, the problem could also be formulated so as to shift some need landscape information into the solution landscape, as in the following: “Reevaluate our production process and the role of chemical X in it.” The enriched solution landscape might then include options like eliminating the need for chemical X entirely by developing an altogether different and cheaper production process.

The benefits of rich landscape search have been discussed and documented by several scholars to date. Raymond (1999, p. 41) explains the value of rich search in both need and solution landscapes in his proposal of a Linus’s law of software debugging: “Given enough eyeballs, all bugs are shallow.” In software, discovering (need landscape) and repairing (solution landscape) subtle code errors or bugs can be very costly (Brooks 1979). However, Raymond argued, the same task can be greatly reduced in cost and also made faster and more effective when it is opened up to a large community of software users. Raymond explains: “Each [user] approaches the task of bug characterization with a slightly different perceptual set and analytical toolkit, a different angle on the problem. … So adding more beta-testers … increases the probability that someone’s toolkit will be matched to the problem in such a way that the bug is shallow to that person” (Raymond 1999, pp. 43–44). In other words, each diverse solver has information regarding a somewhat different section of the need and/or solution landscapes. As a result, a richer landscape search can be conducted via the aggregated efforts of the group.

Jeppesen and Lakhani (2010) documented the benefits of rich solution landscape search in fields beyond software. They studied the outcomes of problem-solving challenges (i.e.,specific formulated problems) issued by the crowdsourcing firm InnoCentive. They found that many problems that this firm’s clients had found difficult to solve using their standard, in-house R&D staff were easily solved by some individuals among many recruited to search the solution landscape with diverse expertise sets. Each diverse solver deeply knew, in other words, a somewhat different section of the solution landscape. The result was that a larger fraction of a rich solution landscape was searched (nonalgorithmically) by a diverse group of solvers than had been done by in-house staff, and viable need-solution pairs were found as a consequence (Jeppesen and Lakhani 2010; also see Penin and Burger-Helmchen 2012).

5. Complexity of Rich Landscape Search

Given the evidence of efficacy described above, one might think that a straightforward solution to the difficulties created by present methods of problem formulation would be to simply widen landscape search enough to provide access to all possible, viable need–solution pairs. However, the richness of information contained in both need and solution landscapes can often make this impractical.

As illustration of need and solution landscape richness, and the impracticality of “full search,” consider a problem-formulation method involving a search for underlying causes. The “5 whys” method was developed by Sakichi Toyoda for use in the Toyota Motor Corporation (“5 Whys”). Toyoda specifies that one must ask “why” 5 times to get to what he terms the root cause of a problem.² An example follows:

My auto will not start. (the initial problem statement)

1. Why?—The battery is dead. (first why)

2. Why?—The alternator (a battery-recharging mechanism used in autos) is not functioning. (second why)

3. Why?—The rubber belt driving the alternator has broken. (third why)

4. Why?—The alternator belt was well beyond its useful service life and was not replaced. (fourth why)

5. Why?—The car was not maintained according to the recommended service schedule. (fifth why, a root cause) (“5 Whys”)

Via this illustrative example, one can see that each level of problem statement points to a different location on the need landscape. If one focuses only on the first-level problem statement, for example, it reflects a need to start the car. The solution would clearly be to recharge or replace the auto battery. Solving that formulation of the problem would indeed create a functioning car in the short term, but the answer to the second why reveals that this solution would not be viable for very long. Without the alternator running, the battery would soon discharge again, and the car would again not start. As the third why reveals, the alternator cannot be restored to function without installing a new drive belt, and so forth down through the levels.

Of course, arbitrarily deciding that five successive whys as presented in this example is sufficient to get to a root cause or real underlying need greatly understates both the depth and breadth of possibly useful inquiry. With respect to depth of inquiry, one might, for example, continue via a sixth why to ask why the car was designed using a failure-prone belt to drive the alternator in the first place. At that level, for the first time possible needs to redesign the car and/or the corporate innovation process become part of the need landscape considered.

With respect to the potential breadth of both need and solution landscapes, consider that we can begin with the same initial observation (“My car will not start”) and carry the identification of needs and related solutions in an entirely different direction. We may, for example, ask a series of why questions related to the function the car is to provide. Why do I need to use the car at all on this occasion, and what alternatives might there be? Why do I need the car to start? I need to drive to the city center. Why? I have to attend a meeting at work. Why do I need to take a car to the meeting? … This line of diagnostic inquiry within the need landscape leads one to discover a totally different set of solution options, having to do with alternate ways of substituting for the function of the ailing car. Perhaps one can borrow a neighbor’s car. Perhaps one can take a train or a cab or a bike or go on foot. Perhaps one can phone in to the meeting instead of attending in person, and so forth.

Note that each of the many ways a given problem can be specified and solved may require different information that will be useful for problem solvers depending upon the need inquiry and solution path taken. If, in the example given just above, problem formulators or problem solvers decide to redesign the automobile alternator to do away with wear-prone alternator drive belts, detailed auto design information will be very valuable to problem solvers. On the other hand, if problem formulators or solvers decide to focus on alternative ways to get to the meeting independent of the nonfunctioning auto, then information on local train schedules will be very useful, and auto design information will be irrelevant. In effect, an individual or firm with a need cannot be sure to supply potentially relevant information about the need landscape to solvers without first knowing the “best” solution path, or the one that problem solvers will in fact take. Additionally, as noted earlier, these matters can be anticipated to only a varying and generally limited extent before actual problem solving (Simon 1977, von Hippel and Tyre 1996b).

6. Direct Discovery of Need–Solution Pairs

To this point we have explored some practical difficulties with the problem formulation followed by the problem-solving paradigm that are common in management and organizational theory, as well as in practice. In what we think is a novel contribution to the problem-solving literature, we propose that one way to engage in need and solution landscape search is to not formulate a problem before search. Instead, one may simply scan one’s mind and/or the environment for need–solution pairs that might fit one’s own context. Next, these pairs can be tested against need and solution landscape information for viability. In this case, problem formulation comes only after discovery of a potentially viable need–solution pair, if, indeed, problem formulation is done at all. (When the need and/or solution component of pairs has tacit elements, it can be impractical to formulate the problem or solution judged viable and implemented in encoded form even if one wishes to.) As we said in §1, we think that this process is frequently used in practice, but it has not yet been described as a problem-solving process to our knowledge.

Recall our §1 example of a firm employee walking through a trade show who discovers new payroll processing software (solution) that claims to solve certain problems (needs) relative to earlier software. The employee then mentally assesses that need–solution pair for viability within the firm after discovering that pair. “I wasn’t thinking that we had any payroll-processing deficiencies but now I recognize that we do (thus recognizing a problem worth solving only after encountering a potential solution worth implementing). Further, this new software may solve the problem. It looks better than the system we are using now.” Additionally or alternatively, the employee may be stimulated to identify a new-to-the-world need–solution pair: “This payroll-processing approach gives me an idea to create a better design for another situation entirely, one that I was not thinking of as problematic.”

Problem solving via need–solution pairs may involve a problem statement that is, however, treated as a disposable, alterable part of a need–solution pair rather than as a fixed objective. For example, suppose you say to me, “I would like to go out for dinner tonight. What restaurant shall we choose?” I might respond, “Let’s go to the skating rink instead. If we get hungry, we can grab something at the snack bar.” Note that in my response I offered you a complete need–solution pair and note also that my response implicitly changed your awareness and problem statement to a different one. I broadened the “Where shall we go for dinner?” problem you initially posed to “What shall we do tonight?” In other words, I regarded your initial problem statement as a variable. You would not be astonished or offended that I did this: it is within the bounds of ordinary social behavior. Indeed, you might flexibly counter by offering still another need–solution pair for assessment, saying: “Well, the skating rink will be crowded tonight. Since it is Friday, let’s drive up to the mountains instead and go hiking tomorrow. The weather is supposed to be beautiful this weekend.” By doing this, you, too, regarded my problem statement as a variable, implicitly changing “What shall we do tonight?” to “What shall we do this weekend?” and offering still another need–solution pair for evaluation.

A great advantage of a need–solution pair problem-solving process is that the potentially relevant need and potentially useful solution come packaged together. One need not then invest effort in formulating a problem or searching either a complete need or solution landscape. One simply tests the proposed need–solution pairing for viability against the points in the relevant landscapes that the identified need–solution pair happens to touch.

7. Next Research Steps

At the most general level, we think the field of problem solving should reconsider what we see as a currently pervasive assumption that the problem-solving process begins with the formulation or identification of a problem. It may be difficult to do this: the very term “problem solving” seems to contain an implicit assumption that one starts with a problem. However, in this paper we have argued that there can also be great value in such a reconsideration. We have proposed the direct discovery of need–solution pairs as an approach in which problem formulation may not be required, and there may be others.

Note that we are not proposing that the identification of need–solution pairs is a panacea. Indeed, one might argue that the conventional procedure of exhaustive search of a well-defined solution landscape to solve a prespecified problem will be much richer than the possibly serendipitous discovery of viable need–solution pairs. However, on the basis of our earlier discussions, we suggest that the conventional sequence only gives an illusion of richness. As we have seen, the actual solution landscape segment selected for search via a formulated problem, relative to the full landscape, is likely to be very narrow indeed. In net, given the probably typical complexity of need and solution landscapes, all problem-solving approaches are likely to leave many potentially viable need–solution pairs undiscovered.

To begin research on the concept of the need–solution pair as a problem-solving strategy, we suggest two related research directions. First, it will be important to explore whether present-day problem-solving practice already contains a significant element of direct search for viable need–solution pairs. If need–solution pair identification without problem formulation is indeed more efficient or effective than problem formulation followed by problem solving, it is very probable that people both individually and within organizations are already frequently doing this form of solving, although perhaps not consciously. If the practice is not found, our theoretical conjecture should be rejected. Second, if need–solution pair solving is being practiced, it would be worth studying how it is being done in detail, and how it can be done better. Pursuant to both objectives, we must identify and draw upon extant streams of theory and research that can offer assistance.

7.1. Determining the Extent and Value of Need–Solution Pair Problem Solving

Problem solving via identification of viable need–solution pairs, as we have described it, has a clear associated marker: identification of such a pair without prior formulation of a problem. To determine how often this practice is followed will require studies specifically designed to identify that marker. Consider that, often, studies of problem solving begin with a list of formulated problems: for example, how does department X in firm Y prioritize and solve a listed set of problems that they have specified or that has been assigned to them for solution in period Z. Clearly, problem solving without problem formulation is likely to go undetected or to be undercounted in such studies. As an alternative approach, we propose basing studies upon samples of solutions (changes) implemented rather than problems formulated. For example, one could begin with a list of the solved problem outputs produced by department X over a period of time. Then one could work back from each solved problem to explore the solving method used to devise or discover what was eventually regarded by department X as a viable need–solution pair.

Today, there are increasingly economical ways to examine how people explore or browse because of the availability of data on searches conducted digitally. Such data cover the path of movements people make when searching in a physical or virtual landscape, e.g., traces people leave in content browsing on the Internet or eye-tracking patterns they display when they conduct their search (Cutrell and Guan 2007). One may also ask people at the start of their searches why they are searching, or what they are searching for and then, by following the evolution of the search, identify whether and when novel need–solution pairs emerge, with related purchases then documenting their viability from the perspective of the searcher. Scholars have found that browsing patterns do or can predict future purchasing behavior, a possible indicator that this kind of research can work (see Hui et al. 2009).

With respect to comparing the potential value of need–solution pair search with solution space search based upon formulated problems, we conjecture that, under conditions where need landscapes are complex and many and diverse need–solution pairs are potentially viable, a problem-solving process based upon a search for viable need–solution pairs is likely to be effective and efficient. In contrast, under conditions where only a small portion of the need landscape can give rise to a viable need–solution pair because of organizational, technical, temporal, or other constraints, formulation of problems before solving may well be the most efficient way to focus solver efforts on the relevant portion of the need landscape.

It may be possible to compare the two processes by running experiments based upon the variations in the content of need and solution landscapes. For example, recall that we have argued that problem formulation can hide or prevent access to potentially viable need–solution pairs. To understand the extent of that problem, it may be possible to perform experiments to progressively shift the need landscape/solution landscape partition by systematic variation in problem formulation. For each formulation, more or less of the need landscape is transformed into the solution landscape. Search for successful solutions can then be conducted and compared with the outcomes of unconstrained searches for viable need–solution pairs.

7.2. Exploring Conditions for Effective Need–Solution Pair Search

Next, it will be useful to better understand the nature and effectiveness of need–solution pair search. At this point, it is not clear whether that process is intuitively done well (i.e., “as natural as breathing”). If so, we will still want to better understand how it works. Alternatively, if it is difficult to do well, there will be benefit from research to explore how effectiveness and efficiency could be improved. Below, we suggest what we think are some useful starting points for further research for both of these purposes.

First, it is clear that human agency considerations will have an impact on individuals’ and groups’ ability to discover viable need–solution pairs. (In sociology and philosophy, “agency” frequently refers to the capacity of an agent (a person, other living being) to act in a specific context, using its abilities and resources. For example, human beings act with intention and pursue goals in specific contexts (Emirbayer and Mische 1998).)

In that regard, recall our §1 example of Bernard Sadow, the inventor of the rolling suitcase. This individual was employed by a suitcase producer. For this reason he is perhaps more likely than an ordinary suitcase-transporting tourist facing identical stimuli to bring to mind viable need–solution pairs with respect to suitcases. Similarly, in our §1 example of the individual happening to notice a safe and sturdy bike baby carrier in the store window, recall that that person is described as a parent who has the task of transporting a child to preschool. This contextualization of the activity imbues the person with mindfulness, intent, and selective attention relevant to the need–solution pair identified. An example of an experiment to explore the importance of human agency and context is to systematically vary subject agency and/or need–solution landscape information presented to experimental subjects. One could then study related variations in those subjects’ abilities to discover viable need–solution pairs.

It is also the case that individuals are likely to have greater “power” to identify a particular need–solution pair as a function of their individual personalities and other characteristics (e.g., cognitive skills). Thus, research has shown that the basic personality trait “openness to experience” characterizes someone who is intellectually curious and tends to seek new experiences and explore novel ideas (Barrick and Mount 1991, Zhao and Seibert 2006, p. 261). Such individuals are also described as creative, imaginative, reflective, curious, and untraditional (McCrae and Costa 1985, George and Zhou 2001). They tend to produce creative ideas (Martinsen 2011). Other individual characteristics, such as cognitive styles, divergent and holistic thinking, the use of metaphors, and analogical reasoning are also linked to the production of creative ideas (Koestler 1964). Emotional states likely matter, too. Positive affect can lead individuals to explore and seize opportune solutions to satisfy their needs, rather than shy away from what they perceive as “difficult problems” (Adler and Obstfeld 2007).

Further, individuals’ ability to conceive of novel need–solution pairs is likely to be affected by their prior experience with objects and concepts related to those pairs. It has been shown that, due to an effect called “functional fixedness,” subjects who use an object or see it used in a familiar way are strongly blocked from using that object in a novel way (Duncker 1945, Birch and Rabinowitz 1951, Adamson 1952). It has also been shown that experimental subjects familiar with a complicated problem-solving strategy are unlikely to devise a simpler one when this is appropriate (Luchins 1942). These effects have also been demonstrated in real-world research settings (Allen and Marquis 1964). Relatedly, individuals display the highest creativity when they apply problem-solving approaches that they do not routinely use (Dane et al. 2011). Still further, organization- and team-level factors (e.g., climate, openness) also have been shown to shape creativity (Amabile et al. 2005, Amabile and Pillmer 2012).

Research will be required to determine whether these individual-, group-, and organization-level factors matter a lot or a little with respect to the effective and efficient identification of viable need–solution pairs. Once that is understood, further work can determine how to apply these insights to make the identification of viable need–solution pairs more effective and efficient. For example, if functional fixedness is a major impediment, guidance toward envisioning novel need–solution pairs can also be more systematically supplied. Thus, toolkits that can be helpful offer lists of possibly useful associations for use by need formulators and problem solvers. TRIZ (Altshuller 1998) is an example. It is a toolkit that is based on the study of patterns of invention in the global patent literature and on the premise that basic types of problems and solutions are repeated across industries and sciences. It offers a knowledge base and model-based technology to enable a problem solver to systematically “try out” success-associated possibilities that might otherwise not have been envisioned.

Success might also be more likely if solver individuals or teams are chosen to already have both need and solution landscape information, encoded and nonencoded, in their minds. Individuals so equipped could more cheaply do searches involving both landscapes relatively inexpensively via thought experiments. This is why individuals who, for example, themselves have a need (reside in a real need landscape) and themselves also have solution expertise are such powerful developers of novel need–solution pairs. An example is software users who directly experience a need for improved software and are also expert programmers (Raymond 1999, von Krogh et al. 2012). A more general example is user innovators of all types who directly experience needs and discover that they also have the expertise required to solve them, often at low cost (von Hippel 1994, 2005, Chap. 5).

7.3. Integrating Existing Literature Streams

We suggest that research related to identification of need–solution pairs can usefully draw upon and also reciprocally enrich other research streams. Here, we note how research on serendipity, gestalt solution recognition, and emergent design development may be useful to the study of need–solution pairs.

The term “serendipity” refers to “accidental sagacity.” Horace Walpole first devised this “very expressive word” in 1754 by referring to a Persian fairy tale The Three Princes of Serendip in which the three were “always making discoveries, by accidents and sagacity, of things which they were not in quest of …” (Walpole 1960, pp. 407–408). As he further noted, “You must observe that no discovery of a thing that you are looking for comes under this description” (Walpole 1960, p. 408). Serendipity, as it is used today, is often considered a very valuable mode of scientific discovery. Merton, having experienced his own happenstance discovery of the word, transformed it into a concept in the domain of the sociology of science and cogently argued for the inclusion of serendipity as a scientific method alongside purposeful discovery by experimentation (Merton 1968, Merton and Barber 2004). In the social sciences, grounded theory development emphasizes the value of serendipitous discovery of findings for constructing novel theory (see Glaser and Strauss 1967, Bourgeois 1979).

Sometimes, as researchers have found, a serendipitous event triggers a new solution approach to a formulated problem already being worked upon, and the records of many successful discovery processes often show important serendipitous events (Campbell 1960, Fioratou and Cowley 2009). A recent study of scientists involved in interdisciplinary research shows that when these individuals become sufficiently and positively “surprised” about the value of unexpected content, they will often redirect their ex-post solution- search on new paths or even restructure their problem specification (Foster and Ford 2003, p. 334).

Serendipitous events can also trigger new need–solution pairs, viable solutions to problems that had not previously been formulated in a solver’s mind. The discovery of the first synthetic dye by William Henry Perkin came about in 1856 quite serendipitously during the search for synthetic quinine to treat malaria (Banerjee 2014). In more recent times, the cosmetic effects of Botox were serendipitously noticed by a patient who was being treated with that drug for an entirely different medical problem. The general value of this serendipitous discovery was then recognized by and acted upon by the patient’s doctor (Carruthers and Carruthers 1992, Coondoo and Sengupta 2015).

Research on serendipity is also directed at improvements to practice. Some researchers are exploring how problem solvers can more frequently and effectively make and recognize serendipitous discoveries (Van Andel 1994, Denrell et al. 2003). With respect to increasing the sagacity that enables those encountering happy accidents to recognize their potential utility, some conceive of serendipity as a structure, a capability, an opportunity, or a resource, rather than an event, involving conscious efforts to turn chance into luck (Denrell et al. 2003, de Rond and Thietart 2007, de Rond 2014). The work we previously mentioned on human agency, functional fixedness, richness of contextual information, and the like clearly relates to enhancing or reducing that capability for specific solvers with respect to specific events.

Research on serendipity also explores how to artificially increase the level of possibly valuable accidents that problem-solvers can then assess and exploit. Thus, computer scientists have experimented with systems that increase the “chance” element in information retrieval by using data-mining systems in ways that offer unexpected links to new and interesting information (Beale 2007, Liang 2012). Some approaches found that it is even more useful to “augment user skills” in their information seeking by visualization and artificial intelligence techniques (Beale 2007, p. 421) or by supporting the actual strategies that users said that they used in “making my own luck” (Makri et al. 2014, p. 2181). Rich browsing venues (i.e., internal and external knowledge repositories, professional journals, scientific conferences, business conferences, trade shows, and so forth) can also serve these functions (Bathelt et al. 2004).

Of course, it is also likely that research on need–solution pairs can reciprocally inform research on serendipity. By studying serendipity and serendipitous events as a trigger for combining and evaluating points on need and solution landscapes, scholars can systematically explore, for example, the characteristics of landscape points and combinations that appear to be most susceptible to such events.

Another research field that contains findings and thinking useful for those exploring need–solution pairs is the gestalt school of psychology. This school argues that the basis of problem solving is restructuring, a type of process in which the problem solver gains “insights” about a problem and thus comes to see the requirements of a problem situation in a new way (Ohlsson 1984, p. 65; for an interesting historical perspective, see ter Hark 2010). Gestalt psychologists distinguish “solutions that pop into mind” from solutions identified by purposeful search. A central assumption in gestalt theory is that a problem has underlying structural characteristics that the problem solver can manipulate in order to discover a solution (Wertheimer 1959). Recent work suggests that pop-up solutions tend to result from parallel processing of experience and the constraints people perceive in a problem specification (Novick and Sherman 2003). Again, this research stream can offer valuable insights, via both examples and theory, regarding the mental processes that may underlie need–solution pair identification.

A final example of a research stream that may usefully contribute to the study of need–solution pair solving is found in the design literature. Here most of the formal methods tend to both specify the design problem a priori and fix the solution set before problem solving begins (Simon 1979). Such specification ideally allows designers to partition the solution set into subsets, wherein they can conduct more constrained search for the “best” designs. Researchers and practicing designers both argue that such tight a priori problem specifications and solution-set approaches are rarely applicable to design situations addressed by real-world designers (Schon 1983). As a more appropriate approach, Hatchuel and Weil (2002) developed the concept-knowledge (C-K) theory of design. This downplays problem formulation in the design process and seeks to explain the systematic expansion of the solution space by generating new knowledge about hitherto unknown solutions (Le Masson et al. 2010). This kind of work can be insight generating with respect to both how viable need–solution pairs emerge and how the efficiency and effectiveness of that process might be increased.

Many additional research streams will doubtless be found that can contribute useful inputs to the study of need–solution pair solving. However, it is also important to note that some apparently related streams, upon closer examination, will be found to deal with fundamentally different matters. Thus, in the seminal garbage can model of organizational choice, already formulated problems and solutions are put into a “garbage can” and linked and acted upon via managerial decisions (Cohen et al. 1972, Padgett 1980, Fioretti and Lomi 2010). The utility of this research to studies of need–solution pair solving is currently limited precisely because inputs to the garbage can are preformulated problems and solutions rather than precursor information such as that contained in need and solution landscapes.

8. Next Steps in Practice

The eventual outcome of research such as that described in the previous section will be an understanding of the value of need–solution pair problem-solving, and improved methods to do it. What can practitioners do today, while researchers are engaged in such activities? We suggest that it is possible to see present aspects of problem-solving practice that can have the effect of more broadly exploring both need and solution landscapes simultaneously, and to utilize these with a clearer understanding of that fact. In this section we will suggest some practical steps to utilize present-day problem-solving methods in a way that is sensitive to the value of the discovery of need–solution pairs.

8.1. Need–Solution Pair Search Triggered by Broadly Formulated Problems

A hybrid of solving a problem as formulated and opening the way to the discovery of novel need–solution pairs exists when problems are purposely formulated very broadly, leaving room for richer variation than a more narrowly specified problem would allow. For example, a firm manager might implicitly or explicitly formulate a very broad problem statement such as, “Our firm is willing to produce anything we can sell at a profit with our existing production machinery and distribution channels.” They then look to similarly situated firms to see what those firms are producing at a profit as a form of discovery focused on the identification of potentially relevant and viable need–solution pairs. Entrepreneurs not encumbered or empowered with preexisting capabilities can look even more broadly: “What can the basis for a profitable business be for me? Let me search widely for what others are doing successfully.” As a second step in each case, the firms test each of these solutions (a successful need–solution pair for someone, but not necessarily for them) against their own need and solution landscapes, to see if those solutions form a potentially successful pairing for them as well.

Formal problem-solving methods exist that centrally involve a search for successful need–solution pairs guided by broad and flexible problem formulation. These include positive deviance studies and lead user studies. If, for example, one has the broad goal of helping a community in the developing world to improve family health, the positive deviance study approach (e.g., Krumholz et al. 2011, Bradley et al. 2012) would be to search for families in the community who stand out as positive deviants with respect to good family health. The assumption is that such individuals have somehow found and are using a need–solution pairing that is successful for them and that fits conditions found in the community. Researchers then seek to understand what these deviants are doing that is linked to better health. Then, if that need–solution pairing looks viable for others, they may attempt to diffuse it as a best practice in the community (see Szulanski 1996).

Firms doing lead user studies execute a similar approach (von Hippel 1986). For example, a lead user search for collision avoidance methods to be applied to automobiles will be purposely broadened to encompass how anyone or anything successfully causes or avoids collisions with other individuals or things. This can lead to, for example, an exploration of how blind individuals successfully navigate crowded sidewalks, and how military munitions successfully seek and execute collisions with targets—and how targets successfully deploy countermeasures. The focus in these studies is always discovery of successful need–solution pairs that may also fit the need–solution landscape of the firm engaged in search. The next step is to see if any of these pairs actually do have a viable fit to points on both the need and solution landscapes of that solution seeker or if they can be modified to fit (Lilien et al. 2002, Churchill et al. 2009).

8.2. Iteratively Reformulate Problems to Discover Need–Solution Pairs

As was noted in the literature review, it is also possible to start problem solving with a formulated problem, and then iteratively adjust it as problem solving proceeds in order to increase the chance of discovering viable need–solution pairs (e.g., Kurup et al. 2011). In each cycle, a point on the solution landscape (i.e., a solution) is tested against the intended point on the need landscape for viability. If the fit is not good, the problem formulation is changed and solving recommences. The trial-and-error cycle continues until an acceptable need–solution pairing is found or created (Marples 1961, Simon and Simon 1962, Allen 1966, von Hippel and Tyre 1996b, Thomke 1998, 2003, Hsieh et al. 2007, Nelson 2008).

Another example of such a method is the rapid prototyping method of product development, as practiced by software development firms and in many other fields as well. In rapid prototyping, problem solvers respond to initial user need specifications by quickly developing and delivering to users (usually within weeks) an inexpensive, easy-to-modify, working model that provides much of the functionality contained in the initial request. The users then apply the prototype in their own environment using their own data. Based upon what they learn in the trial, users then modify their need specification (Gronbaek 1989, pp. 114–116). A revised prototype incorporating the modified problem formulation is then quickly developed and sent to the user. The trial-and-error learning and iterative problem-and-solution reformulations by developer and user are repeated until a successful need–solution pairing is found.

In the software field, it has been found that rapid prototyping methods are not only less costly than traditional, noniterative methods such as the waterfall development approach based upon a fixed problem, but that they are able to “better satisfy true user requirements and produce information and functionality that is more complete, more accurate, and more meaningful” (Connell and Shafer 1989, p. 15). Various implementations of these basic operating principles are today collectively termed “agile software development,” where the emphasis is on frequent and intense interaction between developers and users. For example, one variation involves a software user and a software developer jointly and frequently inspecting the emerging product (e.g., Ferre et al. 2001). In these reviews, both need statements and solutions developed are topics for review and change (for a review see Conboy 2009).

9. Conclusion

Years ago, Herbert Simon (1973) stated that the only well-structured problems were ill-structured problems formulated for problem solving. We now think it useful to add that the only ill-structured problems extant are those identified as problems to be solved. In this essay we have explored the additional option of discovery of viable need–solution pairs without ex ante identification or formulation of a problem. This option may be frequently used in practice today and may be valuable under conditions yet to be assessed.

Problem solving through search for viable need–solution pairings is a generic process that can be conducted by individuals, teams, and organizations. It can be pursued in-house, and/or can be outsourced via crowdsourcing calls. We propose that it will be important to learn how it works and can work in these different contexts and governance regimes (Felin and Zenger 2012, Lakhani et al. 2013, von Krogh et al. 2012). It is well known that a broad range of contextual factors such as organization culture and structure shape the effectiveness and efficiency of organizational problem solving (Lyles and Mitroff 1980, MacDuffie 1997, Marengo et al. 2000, Cohendet and Simon 2007, Le Masson et al. 2010). Clearly, if need–solution pair identification is proven to be valuable, there is a great deal of work to do to understand both how it works and how it can be most effectively implemented in the light of existing and new understandings. We hope that others will find it interesting to join us in considering and exploring these matters.