A Comparative Analysis of Cognitive and Structural Approaches in Innovation Studies

Cognitive and structural approaches to organizational-level actions and behavior are sometimes, if not always, viewed as contradictory.

a. To evaluate this statement, compare and contrast three organizational level theories, one structural, one cognitive, and one that bridges cognitive and structural. For each of these theories, identify their underlying assumptions, mechanisms and the empirical literature that they rely on to explain: 1) agenda setting, and 2) search. In your comparisons and contrasts, be sure to identify when and whether the various theories are contradictory or complementary.

Cognitive and structural approaches to organizational-level actions and behavior are sometimes, if not always, viewed as contradictory – this statement can be best summarized by invoking the ‘Strategy follows structure’ paradigm (Chandler, 1962) against an opposite bottom up effect of cognition as moderating the effects of structure on decision making via mechanisms such as meaning creation(Weick, 1979), hence structure determines strategy rather than the other way around as argued by Chandler (1962). I now summarize the two views focusing first on their assumptions and mechanisms and then implications for agenda setting and search processes, followed by the third view which will seek to address a unified top-down and bottom-up approach to agenda setting and search.

Chandler (1962)

Structure follows strategy (Chandler, 1962). As market structure or environment changes because of technological changes, the adequate strategy and structure of a firm may also change. That is, certain forms of organizational structure may facilitate success, whereas other forms of structure may hinder (Eggers & Park, 2018).

According to Chandler, strategy is “the determination of the basic long-term goals and objectives of an enterprise, and the adoption of courses of action and the allocation of resources necessary for carrying out the goals,” while structure is “the design of organization through which the enterprise is administered” (1962: 13–14). Changes in strategy are mainly responses to opportunities or needs created by changes in the external environment, such as technological innovation. As a consequence of change in strategy, complementary new structures are also devised.

The study of the M-form structure has a long tradition in the field of strategy that dates back to Chandler’s (1962) seminal work. A majority of empirical studies in the strategic management have found some evidence that the M-form is generally associated with higher performance (e.g., Hill, 1985; Hoskisson & Galbraith, 1985; Hoskisson, 1987; Hoskisson, Harrison, & Dubofsky, 1991). More recently, strategy researchers have provided an important clarification regarding the link between M-form and performance by pointing out that M-form efficiency is dependent on internal contingencies (Hill & Hoskisson, 1987; Hill, Hitt, & Hoskisson, 1992; Hoskisson & Johnson, 1992).

To enjoy the benefits from vertical integration or related diversification, an internal cooperative structure is needed; on the other hand, to capture the benefits from unrelated diversification, an internal competitive structure is required. Hence, the theorizing and methods used to assess the impact of strategy on structure are mixed.

Assumptions: When it comes to the effects of strategy and structure – there are an important number of assumptions regarding managerial choice and agency in this work. First, the theory does not pay attention to the opposite effect. For example, Mintzberg argued that the current organisational form can also be regarded as constraining strategic change. Rumelt concluded that structure also followed fashion.

Second, the theory ignores manager’s willingness and ability to impact structure apart from structural constraints on strategy itself. Specifically, cognitive and social aspects of decision making which result in conflict, power in organizations (dominant coalition) and issues of legitimacy as posed by institutional theory that can inhibit structural adaptation.

Third, from the point of view of agenda setting and search, the theory assumes the problems facing the organizations as a given and does not leave room for either sensemaking or interpretation of the situation by decision makers in particularly affecting how they define agendas, the problem shortfalls itself as well as even search processes. Search processes are assumed to look for the best possible alternative (ignoring manager’s bounded rationality) and focusing on managerial actions as intended in a given situation - and this intention is directly assumed to impact strategic direction and structural adaptation.

Mechanisms: In Chandler’s (1962) view, strategic decisions and the resulting change is the primary mechanism through which organizations change their structure in accordance with planned activities. Further, managerial structure itself becomes a mechanism to facilitate firm responses such as adaptation to a technological change.

In terms of agenda setting and search, this implies that firms set agendas and search processes irrespective of structural constraints and define strategies that are perceived optimal, and change the organizational structure to accommodate these strategies.

Empirical Literature: As noted above, the bulk of this literature has taken an approach to study different organizational forms with performance. However, many empirical accounts diverge from the theory’s predicts and instead focus on structure as an enabler of inhibitor of decision making and organizational adaptation. For instance, Siggelkow and Levinthal (2003) theorized the degree of centralization affect a firm’s ability to explore and adapt. They proposed that temporary decentralization may be beneficial in the long run. Such effect of centralization would further depend on the degree of environmental turbulence, where decentralization would be the better structure in turbulent environment (Siggelkow & Rivkin, 2005). And, very recently, some empirical work has moved beyond vertical integration to assess other aspects of structure that may affect adaptation (e.g. Troyer (2017). These empirical inconsistencies led to the growth of managerial cognition in explaining strategic decisions such as agenda setting and solution search through the top-down processes of cognition.

Cognition Interpretation Research

Traditional cognitive theories are symbolically oriented, concerned with conscious processes that operate on symbol structures (e.g., language) stored in long-term memory. People represent the world in terms of internal mental models or schemata from which they form inferences. The schemata are 'cognitive structures that represent organized knowledge about a given concept or type of stimulus' (Fiske and Taylor, 1984, p. 140). From this point of view, organization problems are not a given but have to be interpreted since organizations are information processing systems (Lant & Shapira, 2000) which derive meaning from the external and internal phenomena and feedback to create a shared mental model of the situation that not only makes individual cognitive sense but also surpass social sharing issues to be shared be relevant communities as commonly constructed amongst their members.

In such a view, it is likely that information scanning activities would yield differing results, and that interpretations of information would differ across organizations.

Assumptions: Enactment and meaning creation assumptions in this approach imply that organizations will have differing enactments of the same environment, hence they strategic decision making will be biased by this. Further, Weick viewed the way in which 'reality' is constructed as a social interaction process. That is, interpretation does not occur within the heads of top managers, but rather is a social process that occurs through interaction throughout an organization.

Isms: Search initiations for change (and hence even agenda setting) can be symbolically managed by leaders in this view. Shared mental models are an important mechanism that enable collective action by resolving conflicts, and interpretive issue – since interpretations differ across people. However, rules of interactions and symbolic meanings of actions can substitute shared mental models in predicting decision making. Framing also plays an important role since how a problem is framed, formulated or presented will impact the subsequent mental models and interpretations of actions.

Empirical Evidence: Bartunek's (1984) in a study of ideological and structural change in a religious order documents the changes in interpretive schemes within this organization and the relationship between interpretive change and organizational restructuring. She found that although leadership and environmental events are key triggers to organizational change, the influence of these factors is moderated by the interpretive scheme of the organization – in contrast to Chandler’s approach suggested above.

However, Donellon (1986) and Weick & Roberts (1993) found that groups could engage in organized action without having developed shared beliefs about taking action. Communication mechanisms were critical to achieving collective action or, communication beliefs that facilitate change even when mental models are not shared – calling the need for a theory which takes into account such processes of information acquisition and communication that impact both strategy and structure in ways not predicted by either of the above two theories.

Further, this research assumes that cognition lies at the social level – which makes it socially and individually constructed. However, as pointed in the above paragraph, common mental models (i.e. consistent mental models) are not necessary for initiating change and search or agenda setting – individual cognition plays a role as does the structure of the organization itself (i.e. a top down view where structure also impacts change), instead of ding impact by change (Chandler, 1962), or being less important than mental models and interpretations (Weick, 1979)/

These shortfalls in the literature explaining search behavior and change can be addressed better by incorporating a top down information processing view (Simon, 1958) of organizations juxtaposed with the interpretation and creation of mental models (Wecik, 1979) in enabling agenda setting, search for solutions and subsequent organizational change.

Information Processing Theory (incorporating the top-down and bottom-up views of attending to, processing, and interpreting information)

The top down view of information processing posits structure as key mechanism to distribute and aggregate information and decision making; and also establishes a link between individual decision making and the sociocognitive properties of organizations and organizational structure. Clearly departing from the above two theories which consider the opposite role of structure, and a role where structure is put in the back seat driven by mental models – in this theory structure acts a key mechanism for agenda setting, search and change.

Informed by the work by Thompson (1967) and Lawrence and Lorsch (1967), which established two central design concerns—differentiation and integration—as solutions to the problems resulting from uncertainty and environmental demands. Lawrence and Lorsch defined differentiation as the “state of segmentation of the organizational system into subsystems, each of which tends to develop particular attributes in relation to the requirements posed by its relevant external environment” (1967: 4); they defined integration as the “quality of the state of collaboration that exists among departments that are required to achieve unity of effort by the demands of the environment” (1967: 11).

Drawing from contingency theory this perspective achieves to strive a fit between between its environment and its structure. The scope of structural choice is therefore limited, and a lack of congruence or fit between structure and the environment results in suboptimal performance (Burns & Stalker, 1961; Galbraith, 1977). This again, is in contrast to the bottom up approach which refuses that a true environment lies somewhere and only believes in actor’s interpretations of the environment.

Assumptions: In line with the bottom up approach, people tend to notice schema-relevant stimuli, and they tend to understand and remember things the way they expect them to be leading to cross-situational consistency because people interpret and cognitively process input data based on an internal set of motivations or goals. According to this perspective, cognitive capacity is fixed and limited since knowledge lies in the content of a particular memory location that must be accessed. Hence what agendas people set and what solutions they search for is a factor of their common mental models and cognitive limitations and not reflections of real organizational structure-fit with environment (Fiol, 2005).

Further, incorporating the top down approach of information processing, role of structure is to increase the organization’s information and to deal with internal complexity and environmental uncertainty (Gulati, Lawrence & Puranam, 2005). Hence, this approach squarely believes that environments can be different for not only different organizations but for also subunits of the same organization.

Mechanisms: As such, information processing theories assume decision makers to be boundedly relational and satisfycing in the face of limited information and cognitive ability. These limitations are in part, imposed by the environmental and organizational structures that decision makers are situated in. Decision makers will make more legitimate decisions (Institutional theory), will be affected by their network (Network theory) and will learn from feedback (adaptation). All this implies structure defines strategy (in contrast to Chandler, 1962).

Formal roles, standard operating procedures, and communication channels routinize the elevation of decisions up the chain of command and enable efficient information processing (Galbraith, 1974; Khandwalla, 1974).

Empirical evidence: Knudsen and Levinthal (2007) described how different organizational structures perceive and search in an NK landscape and found that screening ability and organizational structure exhibit a high degree of complementarity. The less (resp. more) able are individual evaluators, the more attractive are organizational forms that tend toward hierarchy (resp. polyarchy) - and the relative rates of Type I and Type II errors in an evaluation process are critically affected by how the organization’s decision processes are structured.

Firms search for solutions and adapt by trial and error and local search is shaped by organization structure (Joseph et al, 2017). Deviations from local search were more strongly associated with complex tasks, especially in the advanced stages of search (Billinger, Stieglitz, and Schumacher, 2014). Structural mechanisms (in the forms of hierarchy, centralization) that prevent managers from making suboptimal choices and thereby improve overall search quality (Rivkin and Siggelkow, 2003).

One common theme of this research is the lack of a unifying prescriptive ‘one good’ structure. Under extreme conditions—for instance, high environmental complexity and turbulence—it is crucial that the firm be able to assess all alternatives regardless of whether this goal is achieved through high levels of centralized coordination at the top of the organization or through mutual adjustment across decentralized departments (Siggelkow & Rivkin, 2005).

While the above research focuses on top down processes to primarily inform search related decisions, research on cognition also addresses the topic of agenda setting, unlike top down approaches which focus less on problem definition and agenda setting. For example, Barreto and Patient (2013) find that for subunit managers, a proximate shock is more likely to activate parochial interests and hence to make the shock’s “loss features” more salient as an object of problem formulation or agenda setting. Similarly, Dutt and Joseph (2018) find that corporate managers are less likely (than subsidiary managers) to exhibit uncertainty avoidance and more likely to respond to regulatory uncertainty by attending to renewable sources of electricity—as reflected in the firm’s ultimate agenda.

The bottom-up cognitive research agenda, combined with the direct work of the impact of structure on strategy as outlined above combines to form a powerful view of both perspectives to give a scholars a better understanding of the search and agenda setting processes. Though the above research focuses on differentiating features of the organizational structure in enabling search and agenda setting, a significant amount of research also focuses on integrating behaviors resulting in similar behaviors. Though due to the limits on timing that review is excluded from the current discussion, it still important to note its importance.

To conclude, the Chandler (1962) approach predicts that firms will form strategy and change structure to match that strategy -0 focusing on diversified organizational forms as a source for competitive advantge, while the bottom up cognition view rejects the idea that there is a given environment and that decision makers enact their own environments, hence structure takes a back seat in this perspective which locates situational cognition as the main mechanism driving search and agenda setting. In contrast, a combined top-down and bottom up approach of information processing and cognition throws light on the individual as well as social cognition and applying Simon (1958) approach to organizations predicts no one good design but design as a factor of internal and external organizational contingencies facing the organizations which are simultaneously subject to individual perception, attention and interpretation.

b. How do these explanations complement each other when explaining innovation/change? In general, under what conditions do organization studies portray change to be (a) more likely vs. (b) less likely to succeed?

According to the Chandler (1962) perspective, Changes in strategy are mainly responses to opportunities or needs created by changes in the external environment, such as technological innovation. As a consequence of change in strategy, complementary new structures are also devised. Hence this view focuses on how innovation or change impacts structure rather than how structure/ strategy/ sensemaking result in change. For example, new competitive landscape in many industries, as described by Bettis and Hitt (1995), gives rise to relentless pace of competition, emphasizing flexibility, speed, and innovation in response to the fast-changing environment.

In contrast, studies focusing on organization cognition have focused on change as an outcome of cognitively situated decision making. Various studies have found that cognition and identity as well as attentional focus when incompatible with a new technology hinder acquisition and assimilation of new knowledge or assets (Benner and Tripsas, 2012; Danneels. 201;, Eggers and Kaplan, 2009; 2013; Kaplan (2008a).

Cognition may further result in offline or online search - a point that top down perspectives ignore as for them given limited cognitive ability, online search is less costly and more fruitful (Winter et al, 2007) while current research designs in top down research mean that unobservable online research cannot be accounted for in current studies (Posen et al, 2018). Further cognitive biases can also affect organizational behavior in pursuit of change (Tyler & Steensma, 1998).

When it comes to bottom up processing views, researchers have explored, both theoretically and empirically, the interrelationships between several different structural dimensions and such outcomes as strategic processes and innovation (Mintzberg, 1973; Fredrickson, 1986). For example, Maciejovsky (2015) found that a hierarchical structure impedes the upward flow of innovation ideas. Along similar lines, Reitzig and Sorenson (2013) reported that the failure to adopt an idea or innovation can arise from in-group bias among employees of an organizational subunit. Further, as portfolio diversification increases risk taking, it has the potential to increase innovation by managers with a diverse portfolio (Eggers & Kaul, 2018).

Further in contrast to cognitive approaches which assume equal information access to participants, this approach poses that those with informational access and those without, may cause friction, since managers may disagree about whether organizational changes are required (Jordan & Audia, 2012)

However, this research has focused more on the top down perspectives exclusively and organizational structure is modeled as a closed-form solution and viewed as a hierarchical authority (Joseph et al, 2017), and has failed to adequately combined the bottom up view with the top down view to arrive at a comprehensive understanding of the interactions between the two. While institutional forces are an important driver of intertia (and sometimes change), we still seldom encounter Simon’s notion that an organization— and, in particular, its structure—infuses its members with institutional conventions, rules, and roles (Simon, 1947; March & Simon, 1958).

c. Describe how institutional theories about power, legitimacy, and isomorphism account for the drivers and resisters to innovation/change.

Institutional theory seeks to answer an opposite question – what makes organizations the same – or, how do organizations change in a way that they mimic each other? In this view, organizations in a structured field respond to their environments which consists of other organizations responding to their environ While selection acts in early years of the organizations, Isomorphism: constraining process that forces resemblance across organizations that face the same set of environmental conditions

A key mechanism of change in these theories is legitimacy – organizations implement change in the pursuit of legitimacy and not efficiency – in contrast to all the perspectives discussed above. In the specific processes of search and agenda setting and even organizational change and innovation (Roberts and Amit, 2003) – search itself can be institutionalized - routine search activity conducted even in the absence of particular triggers (Dosi, 1988; Patel & Pavitt, 1997). Such pursuit of legitimacy leads to organizational isomorphism where different organizations initiate similar change procedures.

Isomorphism can manifest itself in three ways: 1) Coercive: from political influence and legitimacy problems, formal & informal pressure exerted by (more powerful) organizations that other organizations depend on and by cultural expectations. Even though they may be ceremonial but still shape organizations. 2) Mimetic: uncertainty drives imitation, technological uncertainty, goal ambiguity, ritual aspect-adopted to gain legitimacy, model after organizations perceived to be successful. 3) Normative: from professionalization, from state through cognitive biases and networks allow for rapid diffusion.

Isomorphism behaviors can be good for innovation as organizations learn from the experience of others and can diminish the risk of innovation. Organizational actors can also influence institutional change itself which can positive effects for innovation within the organization as it starts following different institutionalized rules from before. Further, Philippe and Durand (2011) suggested that firms might selectively conform to one dimension of an industry norm while deviating on another (in their case, compliance with a socially approved goal versus level of procedural commitment) so as to gain some discretionary power while still reaping benefits. In conclusion, organizations might benefit from mimetic learning when striving for institutional legitimacy, resulting in isomorphism – but that may not always lead to poor performance even though it mostly limits innovation.