A Comparative Analysis of Cognitive and Structural Approaches in Innovation Studies

Cognitive and structural approaches to organizational actions and behavior are sometimes, if not always, seen as contradictory.

To dive into this, let's look at three organizational level theories: one structural, one cognitive, and one that combines both. For each theory, I'll point out their assumptions, mechanisms, and the studies they rely on to explain agenda setting and search. I'll also highlight when these theories clash or work well together.
Cognitive and structural approaches to organizational actions and behavior are sometimes, if not always, seen as contradictory – this idea can be summed up by comparing Chandler's 'Strategy follows structure' idea (1962) with the opposite effect where cognition affects decision-making through mechanisms like meaning creation (Weick, 1979). Chandler argues structure determines strategy, but Weick suggests the opposite. I'll summarize both views, focusing on their assumptions and mechanisms, and then discuss their implications for agenda setting and search processes, followed by a third view that tries to combine the top-down and bottom-up approaches.

Chandler (1962)

Chandler's idea is that structure follows strategy. When market structure or environment changes due to tech advancements, a firm's strategy and structure might change too. Certain structures can lead to success, while others might hinder it (Eggers & Park, 2018).

Chandler says strategy is “the determination of the basic long-term goals and objectives of an enterprise, and the adoption of courses of action and the allocation of resources necessary for carrying out the goals,” whereas structure is “the design of organization through which the enterprise is administered” (1962: 13–14). Changes in strategy usually respond to opportunities or needs from external changes, like tech innovation. New strategies lead to new structures.

The study of the M-form structure dates back to Chandler’s work. Most studies in strategic management find that the M-form is linked to higher performance (e.g., Hill, 1985; Hoskisson & Galbraith, 1985; Hoskisson, 1987; Hoskisson, Harrison, & Dubofsky, 1991). More recent research shows M-form efficiency depends on internal factors (Hill & Hoskisson, 1987; Hill, Hitt, & Hoskisson, 1992; Hoskisson & Johnson, 1992).

To benefit from vertical integration or related diversification, a cooperative internal structure is needed. For unrelated diversification, a competitive internal structure is required. So, theories and methods to assess the impact of strategy on structure are mixed.

• Assumptions: Chandler’s theory assumes strategy shapes structure, not the other way around. Mintzberg and Rumelt argue the current organizational form can also limit strategic change, and structure can follow trends. The theory also overlooks managers' willingness and ability to impact structure, apart from structural constraints on strategy. It ignores cognitive and social aspects of decision-making, which can lead to conflict, power struggles, and legitimacy issues that can block structural changes.
  In terms of agenda setting and search, the theory assumes problems facing organizations are clear-cut, not leaving room for sensemaking or interpretation by decision-makers. Search processes are assumed to look for the best alternative, ignoring managers' bounded rationality and focusing on intended actions in a given situation, with these intentions directly impacting strategic direction and structural adaptation.
• Mechanisms: Chandler sees strategic decisions and resulting changes as the primary mechanism for organizations to change their structure. Managerial structure itself becomes a mechanism to facilitate firm responses, like adapting to technological changes. Regarding agenda setting and search, this means firms set agendas and search processes without structural constraints, defining strategies seen as optimal and changing organizational structure to fit these strategies.
• Empirical Literature: Most of this literature studies different organizational forms with performance. But many studies diverge from the theory’s predictions, focusing on structure as an enabler or inhibitor of decision-making and organizational adaptation. For example, Siggelkow and Levinthal (2003) suggest the degree of centralization affects a firm's ability to explore and adapt. They propose that temporary decentralization may be beneficial long-term, depending on environmental turbulence (Siggelkow & Rivkin, 2005). Recent work has moved beyond vertical integration to assess other structural aspects that may affect adaptation (e.g., Troyer, 2017). These empirical inconsistencies led to a focus on managerial cognition in explaining strategic decisions like agenda setting and solution search through top-down cognitive processes.

Cognition Interpretation Research

Traditional cognitive theories focus on conscious processes that operate on symbol structures (e.g., language) stored in long-term memory. People represent the world with internal mental models or schemata, forming inferences from them. The schemata are 'cognitive structures that represent organized knowledge about a concept or type of stimulus' (Fiske and Taylor, 1984, p. 140). Here, organizational problems are not given but interpreted, as organizations are information processing systems (Lant & Shapira, 2000), deriving meaning from external and internal phenomena and feedback to create a shared mental model of the situation.

Different organizations will yield different information scanning results, and interpretations of information will vary.

• Assumptions: Enactment and meaning creation assumptions imply organizations will interpret the same environment differently, biasing their strategic decision-making. Weick sees 'reality' construction as a social interaction process, not just within top managers' heads but through interactions throughout the organization.
• Mechanisms: Leaders can symbolically manage search initiations for change and agenda setting. Shared mental models enable collective action by resolving conflicts and interpretive issues, as interpretations differ across people. Rules of interactions and symbolic meanings of actions can substitute shared mental models in predicting decision-making. Framing plays a crucial role since how a problem is framed affects subsequent mental models and interpretations of actions.
• Empirical Evidence: Bartunek's (1984) study of ideological and structural change in a religious order shows that leadership and environmental events trigger organizational change, but the influence is moderated by the organization's interpretive scheme, contrasting Chandler’s approach. However, Donellon (1986) and Weick & Roberts (1993) found that groups could take organized action without shared beliefs about taking action. Communication mechanisms achieve collective action or facilitate change even without shared mental models, indicating a need for a theory that accounts for information acquisition and communication processes impacting both strategy and structure in ways not predicted by the other two theories.

Cognitive research assumes cognition lies at the social level, making it socially and individually constructed. But common mental models are not necessary for initiating change, search, or agenda setting – individual cognition and organizational structure also play roles (a top-down view where structure impacts change). These gaps in explaining search behavior and change can be better addressed by combining a top-down information processing view (Simon, 1958) of organizations with the interpretation and creation of mental models (Weick, 1979) for enabling agenda setting, search for solutions, and subsequent organizational change.

Information Processing Theory

The top-down view of information processing sees structure as key to distributing and aggregating information and decision-making, linking individual decision-making with the sociocognitive properties of organizations and organizational structure. This theory departs from the others by seeing structure as a key mechanism for agenda setting, search, and change. Informed by Thompson (1967) and Lawrence and Lorsch (1967), this perspective focuses on differentiation and integration as solutions to problems from uncertainty and environmental demands. Differentiation is the “state of segmentation of the organizational system into subsystems” (1967: 4), and integration is the “quality of the state of collaboration among departments” (1967: 11). Drawing from contingency theory, this perspective aims to fit between its environment and structure. Structural choice is limited, and a lack of fit between structure and environment leads to poor performance (Burns & Stalker, 1961; Galbraith, 1977), contrasting the bottom-up approach that denies a true environment exists and relies on actors' interpretations.

• Assumptions: People tend to notice schema-relevant stimuli, understanding and remembering things as expected, leading to consistency because people interpret and cognitively process input based on internal motivations or goals. Cognitive capacity is fixed and limited, with knowledge in specific memory locations that must be accessed. So, what agendas people set and what solutions they search for are influenced by their common mental models and cognitive limitations, not reflections of real organizational structure-fit with the environment (Fiol, 2005). Incorporating the top-down approach, structure's role is to increase the organization’s information and handle internal complexity and environmental uncertainty (Gulati, Lawrence & Puranam, 2005). This view believes environments can differ for different organizations and subunits of the same organization.
• Mechanisms: Information processing theories assume decision-makers are boundedly rational and satisfycing due to limited information and cognitive ability. These limitations are partly imposed by the environmental and organizational structures that decision-makers are in. Decision-makers make more legitimate decisions (Institutional theory), are affected by their network (Network theory), and learn from feedback (adaptation), implying structure defines strategy (contrary to Chandler, 1962). Formal roles, standard operating procedures, and communication channels routinize decision elevation up the chain of command, enabling efficient information processing (Galbraith, 1974; Khandwalla, 1974).
• Empirical evidence: Knudsen and Levinthal (2007) described how different organizational structures perceive and search in an NK landscape, finding that screening ability and organizational structure complement each other. The less able individual evaluators are, the more attractive hierarchical forms are, while more able ones prefer polyarchy. The relative rates of Type I and Type II errors in an evaluation process are critically affected by organizational decision processes. Firms search for solutions and adapt by trial and error, with local search shaped by organizational structure (Joseph et al, 2017). Deviations from local search are more associated with complex tasks, especially in advanced search stages (Billinger, Stieglitz, and Schumacher, 2014). Structural mechanisms (like hierarchy, centralization) prevent managers from making suboptimal choices, improving overall search quality (Rivkin and Siggelkow, 2003).

A common theme is the lack of a unifying prescriptive ‘one good’ structure. Under extreme conditions, like high environmental complexity and turbulence, it's crucial to assess all alternatives, either through centralized coordination at the top or mutual adjustment across decentralized departments (Siggelkow & Rivkin, 2005). While top-down processes primarily inform search decisions, cognitive research also addresses agenda setting, unlike top-down approaches that focus less on problem definition and agenda setting. For example, Barreto and Patient (2013) found that a proximate shock makes subunit managers focus on parochial interests, making the shock’s “loss features” more salient for problem formulation or agenda setting. Similarly, Dutt and Joseph (2018) found that corporate managers are less likely to exhibit uncertainty avoidance and more likely to respond to regulatory uncertainty by attending to renewable sources of electricity.

The bottom-up cognitive research agenda, combined with the impact of structure on strategy, forms a powerful view of search and agenda setting processes. Though this research focuses on differentiating organizational structure features, a significant amount also focuses on integrating behaviors, resulting in similar actions. Though this review excludes that due to time limits, it's still important. To wrap up, Chandler (1962) predicts that firms form strategy and change structure to match it, focusing on diversified forms for competitive advantage. The bottom-up cognition view rejects a given environment, seeing decision-makers enacting their own environments, with structure taking a back seat. A combined top-down and bottom-up approach of information processing and cognition highlights individual and social cognition, applying Simon's (1958) approach to organizations, predicting no one good design but design as a factor of internal and external organizational contingencies, subject to individual perception and interpretation.

How do these explanations work together to explain innovation or change? When do studies say change is more or less likely to succeed?
According to Chandler (1962), changes in strategy mainly respond to opportunities or needs from external changes, like tech innovation. New strategies lead to new structures. So, this view focuses on how innovation or change impacts structure, not how structure, strategy, or sensemaking lead to change. For example, the new competitive landscape in many industries, described by Bettis and Hitt (1995), emphasizes flexibility, speed, and innovation in response to the fast-changing environment. In contrast, studies on organizational cognition focus on change as an outcome of cognitively situated decision-making. Various studies find that cognition and identity, as well as attentional focus, when incompatible with new technology, hinder the acquisition and assimilation of new knowledge or assets (Benner and Tripsas, 2012; Danneels, 2011; Eggers and Kaplan, 2009; 2013; Kaplan, 2008a). Cognition may lead to offline or online search, a point that top-down perspectives ignore since, for them, given limited cognitive ability, online search is less costly and more fruitful (Winter et al, 2007). Current research designs in top-down research mean unobservable online research can't be accounted for (Posen et al, 2018). Cognitive biases can also affect organizational behavior in pursuing change (Tyler & Steensma, 1998).

Bottom-up processing views explore the interrelationships between different structural dimensions and outcomes like strategic processes and innovation (Mintzberg, 1973; Fredrickson, 1986). For example, Maciejovsky (2015) found that a hierarchical structure impedes the upward flow of innovation ideas. Reitzig and Sorenson (2013) reported that the failure to adopt an idea or innovation can arise from in-group bias among employees of an organizational subunit. Portfolio diversification increases risk-taking, potentially increasing innovation by managers with a diverse portfolio (Eggers & Kaul, 2018). In contrast to cognitive approaches, which assume equal information access for participants, this approach suggests that those with information access and those without may cause friction, as managers may disagree about necessary changes (Jordan & Audia, 2012).
However, top-down perspectives focus more on organizational structure as a closed-form solution, viewing it as hierarchical authority (Joseph et al, 2017), and haven't adequately combined bottom-up and top-down views for a comprehensive understanding of their interactions. While institutional forces are crucial drivers of inertia (and sometimes change), we rarely encounter Simon’s notion that an organization—and its structure—infuses its members with institutional conventions, rules, and roles (Simon, 1947; March & Simon, 1958).

How do institutional theories about power, legitimacy, and isomorphism explain the drivers and resisters to innovation or change?
Institutional theory looks at what makes organizations similar, or how they mimic each other. In this view, organizations in a structured field respond to their environments, which consist of other organizations responding to their environments. While selection acts in early years, isomorphism— a process that forces resemblance across organizations facing the same set of environmental conditions—takes over. A key mechanism of change in these theories is legitimacy. Organizations implement change to gain legitimacy, not efficiency, contrasting the perspectives discussed above. In search and agenda setting, and even organizational change and innovation (Roberts and Amit, 2003), search can be institutionalized—routine search activity conducted even without specific triggers (Dosi, 1988; Patel & Pavitt, 1997). Pursuit of legitimacy leads to organizational isomorphism, where different organizations initiate similar change procedures. Isomorphism can manifest in three ways: 1) Coercive: from political influence and legitimacy problems, with formal and informal pressure from more powerful organizations and cultural expectations. Even if ceremonial, they still shape organizations. 2) Mimetic: uncertainty drives imitation, with technological uncertainty, goal ambiguity, and ritual aspects adopted to gain legitimacy, modeling after perceived successful organizations. 3) Normative: from professionalization, state through cognitive biases, and networks allowing rapid diffusion. Isomorphism behaviors can benefit innovation by learning from others' experiences, reducing innovation risk. Organizational actors can influence institutional change, positively affecting innovation by following different institutionalized rules. Philippe and Durand (2011) suggested firms might selectively conform to one dimension of an industry norm while deviating on another, gaining discretionary power while reaping benefits. Organizations might benefit from mimetic learning when striving for institutional legitimacy, resulting in isomorphism, but it may not always lead to poor performance, although it mostly limits innovation.