Skip to main content

Welcome, the Hub connects all projects


Finding Value and Meaning in the Concept of Partnership


This paper reports on findings from the National Science Foundation sponsored Research, Evaluation, and Technical Assistance (RETA) project (NSF 02-061 Award #: 0231904):

Alternative Approaches to Evaluating STEM Education Partnerships: A Review of Evaluation Methods and Application of an Interorganizational Model

Gordon Kingsley - Principal Investigator

What is a partnership? This seemingly simple question evokes an astonishing variety of responses that can have profound implications for the formation, operation and outcomes of a partnership. In this study we develop a model of partnership with the goal of linking embedded partnership relations to educational outcomes. The academic literature around ideas of partnering leads to an organizational or community building view of partnerships. However, in an electronic Delphi panel conducted as part of the Georgia Tech Math Science Partnership (MSP) Research, Evaluation, and Technical Assistance (RETA) project, four distinct ways of conceptualizing partnership were found. Many respondents used entity-based conceptualizations in which the partnership is comprised of memberships, boundaries, and consists of formal and informal organizing structures designed to achieve specific functions. Other respondents used a process-based conceptualization in which relationships are built up over time to enhance levels of trust and cooperation. A third group employed an agreement-based conceptualization with predetermined goals aimed at improving performance in science, technology, engineering, and mathematics (STEM) education. A final view was of partnerships as a venue or opportunity to interact. The findings are drawn from a four round electronic Delphi panel of 32 STEM education professionals who have experience working with and organizing math and science education partnerships that link universities and K-12 schools (as required under MSP).

Finding Value and Meaning in the Concept of Partnership1

     Partnership is a topic that has drawn the attention of many fields of inquiry most of which are closely coupled with professions. Reflecting the complexities of modern work environments practitioners, consultants, evaluators, and researchers have been challenged with the need to model and assess the performance of work involving multiple organizations. The need to understand how partnering works, and whether it influences the outcomes of work, has led to significant investments by government and the private sector in the topic. Building upon this trend, our Research, Evaluation and Technical Assistance (RETA) project is aimed at developing a model and theory of partnership. Here, we present the results of an electronic Delphi study which explored the topic of partnership with a panel of 32 STEM professionals who have had extensive experience in organizing and running a school-university partnership aimed at math and science education. Panelists were asked to identify and discuss the elements of the partnerships with which they have been involved and the factors that facilitated or hindered outcomes. Panelists defined the term partnership in distinctive ways. We examine the relationship between panelist conceptualizations of partnership and the elements of a process model of partnering.
One of the challenges in studying partnership is the lack of a common understanding of the term. The various literatures on partnership compound this problem by having little cross- pollination across fields of inquiry. Like blind men with elephants, each field has tackled different aspects of partnerships.

     For example, organization studies are primarily interested in trying to explain the reasons for cooperation and variance in the forms of organizational relations that fosters or hinders cooperation and coordination. This focus has a long history in the private management literature under the heading of inter-organization relations (IOR) which covers a broad range of relations of which partnership is but one form. In an influential work Oliver (1990) identified several distinctive determinants of IOR’s (including partnerships) including the need for exchange, the need for knowledge and technical expertise, and the need for collaboration to secure strategic positions within a field of organizations.

     This contrasts sharply from the public management literature where the term partnership is closely linked to various reform agendas aimed. Here partnership is an all-purpose remedy serving as the answer to many different reform movements (Osborne, 2000). Teisman and Klijn (2002) summarize three features of partnership for public management reform: a) for an individual actor to achieve goals requires activities by other actors; b) the resources and knowledge for achieving goals are distributed across multiple actors; and c) the systems and processes that develop under partnership are complex because they are dependent upon the negotiations of participating actors. Most importantly, there is a strong flavor of program evaluation associated with partnerships in this literature as public agencies, like public schools, are subject to intense accountability standards.

     The education literature on school-university partnership is more closely related to the uses found in the public rather than the private sector. Relatively little attention is given to the factors that drive the creation of a partnership. Rather partnership is offered as a potential remedy for a class of public education problems. The focus in the public sector is upon the implementation and outcomes associated with partnering. Consequently, the questions driving the interest in partnership in the public sector are evaluative in nature. This contrasts sharply with the private sector literature where the interest in partnerships is largely a strategic question of understanding when and how organizations partner.

     Models of partnerships in the education community tend to be descriptive of programs being promoted or evaluated. Goodlad (1988) describes partnerships as collaborative events that grow out of networks of professionals creating a platform for programs, activities, and projects. Tushnet (1994) examines the work of partnerships through the programmatic foci, distinguishing between professional development, curriculum development and client participation as ways of classifying partnership activities. Firestone and Fisler (2002) find that, unlike MSP, partnerships may be a platform for fundraising activities supporting programs and activities sanctioned by participants.

     These models resonate with the goals of the Math and Science Partnership (MSP) program (NSF 02-061) aimed at strengthening K-12 education under the “No Child Left Behind Act”. The program explicitly requires partnerships between universities and K-12 institutions with an expectation that a two-way exchange of ideas and methods will occur. In particular it is expected that the university faculty will bring an increased knowledge of specific content areas, (e.g., mathematics, chemistry, biology) while the K-12 teachers are expected to bring a greater knowledge of current pedagogy. The program assumes that regular classroom teachers will have learned techniques for reaching students but perhaps lack confidence in or detailed knowledge of specific content areas.

     That the MSP2 should be “partnership driven” suggests that a thing called partnership drives the initiative and that it lies between or includes universities and K-12 institutions. We might expect such an entity to have boundaries (things on the inside and on the outside), aspirations or goals, and a tendency to strive for continued sustainable existence. The MSP policy mandates the creation of partnerships that are goal based, encourage the use of evidence and scientific research to develop and improve programs, and that drive “institutional change” while promoting partnership “sustainability”. In order to better understand partnerships in the context of the MSP program we developed a logic model of the partnering process from the available literature.

     We drew on organization, public management, and education literatures to develop a model of the partnership process. For partnership formation we borrowed from the IOR literature, which focuses on the formation of relationships between organizations (e.g., Whetten, 1981). From this literature we drew formation variables like embeddedness and strategic needs (Grannovetter, 1992a). Partnership formation was hypothesized to be a product of embedded relationships among individuals and organizations coupled with an over arching strategic need. Greater overlap of these two conditions is hypothesized to increase the likelihood of partnership formation and may impact partnership operation. Several researchers from different fields have argued that mutual goals are more likely to be met when partnerships are built on embedded relationships (Goodlad, 1994; Gulati & Gargiulo, 1999; Sanders & Epstien, 2000). For the operation and outcome elements of our model we drew from parts of the public management and education literatures that focus on program implementation, operation, evaluation and outcomes (Boyer, 1981; Firestone & Fisler, 2002; Goodlad, 1988, Obsorne, 2000). Partnership operations and programs are then expected to produce outcome measures of process and performance. A rival pathway exists in our model that assumes that partnering has no impact on outcomes. Changes in educational outcomes may be the result of environmental or other factors outside of the partnership.

     Our interactions with the STEM community led to the realization that our model needed to capture partnership’s lifecycle and needed to include important environmental pressures on partnered organizations. The inclusion of environmental variables will allow us to control for the context in which the partnerships operate. Following the first round of data collection, we added elements of coercion (e.g., derived from legal mandates, restrictions embedded in conditions in aid) derived from our panelists’ open-ended responses.

     In the first round of the panel study we learned that our panelists would often speak from a number of perspectives. Sometime they spoke of or for individuals or workgroups. Other times they spoke for their organizations. In order to capture these differences and to be more systematic in our later analysis of the impacts of education partnerships we developed a set of hierarchical partnership interaction levels that include individual, work-group, organizational, partnership, and policy components.

     The complete logic model incorporates these five levels of analysis. We expect that the more detailed logic model will provide a better context for detailed discussions of the impacts and interactions of partnering. Elsewhere, Provan and Milward's analytical approach explored individual, agency, and network level impacts on outcomes (K. G. Provan & Milward, 1995). And, a broader application of Yin suggests looking at individuals, organizations and their network relationships (intermediate units) and the environmental conditions (total system) (2003).

Unit of Analysis Examples
Individual students, teachers, administrators, program officers or staff
Work Group small work groups, administrative grouped program implementation groups
Organization university, K-12, other
Partnership MSP
Policy federal policy

     Within this hierarchical model we can explore the basic question of whether or how partnership structure is related to desired policy outcomes. The key strength of such a hierarchical approach to analysis is that with it, we can explore bridges between and cross-level impacts of the variables in our model (e.g., individuals or organizations can impact partnerships, the environment can impact individuals or organizations, either of which might impact outcomes). We are able to explore the individual, workgroup, organizational, partnership, or policy context of interventions, any of which potentially have greater explanatory weight in a particular case.

About the Model

     This is a hierarchical model of partnership formation, operation and outcomes in an educational context. It is assumed that there may or may not be multiple interactions across levels and stages. It is further hypothesized that partnership creates linkages through these levels. Two sets of hypotheses about partnerships are explored in the larger study. First that something about the way partnerships form or are structured will impact performance outcomes. Second, interactions occur between levels of analysis such that they form bridges in ways that would not occur outside of a partnership. Our primary rival hypothesis is that other pathways exist and that partnerships provide no additional linkages. For this study, the linkages in red are the hypothesized feedback loops that transfer knowledge from one stage of the partnership to the next. It is important to note that the levels of analysis with respect to learning required at various stages of the model may change. Further, there may be learning at several levels that impacts policy outcomes.

The First Grey Box - Pressures on parent organizations

     The concept of embeddedness derived from organizational theory refers to the ongoing relationships among actors. Strategic needs are either internal to an organization’s core mission or are externally imposed by laws, regulations or public demand. When participants repeatedly referred to rules and financial constraints, the importance of adding and including coerciveness in the model was made evident in the early stages of e-Delphi. This collection of pressures, lead to partnership formation while environmental factors may contribute to or hinder partnership formation.

The Second Grey Box - The lifecycle of a partnership

     Partnerships form, operate, and dissolve. Once formed partnerships conduct operations. If these fail or stop we have direct dissolution. However, if programs are implemented, we can identify indicators of process and of impacts, which can then be evaluated.

Red Boxes

     The red typed boxes (H1-H4) and arrows show the hypothesized learning linkages. We can test to see if these theoretical linkages within MSP partnerships exist in fact. However, as our design is quasi-experimental, we must also consider any possible rival explanations for measures changes in process or performance (H0).

Research Methods

The GA Tech Modified Electronic-Delphi

     The electronic Delphi or ‘expert panel’ portion of the Georgia Tech RETA project was initiated in 2004 and was completed April 7, 2005. The panel study was designed to serve as a model building exercise: modeling alternative conceptualizations and modeling alternatives to evaluation. Our modified electronic Delphi is based on the Rand Corporation's paper and pencil Delphi method developed by Dalkey and Helmer and refined through the 1950's and 1960's (Dalkey, 1970). The method involves an iterative survey of experts with the intention of developing a better understanding or consensus on problems, approaches, or future trends. In recent times the computer has been used to facilitate the Delphi method (Turoff & Hiltz, 1982).

     This computerized approach has been used in many ways beyond the traditional structure of consensus seeking Delphi protocols. A policy Delphi is not looking for consensus but seeks to develop pro and con arguments about policy issues and their resolutions (Turoff, 1970). This technique allows a panel of experts to contribute elements to a complex situation with the intention of building a composite model. Our exercise in model building with the Delphi takes a slightly different approach as we started with a model developed from the literature and then began testing the components and looking for missing elements and interactions.

Participant Selection

     The sample frame for this study was 300 STEP education professionals drawn from three sources: online searches of PI’s working on STEM education partnerships, nominations (both of self and others) in response to presentations of our partnership research, and lists of attendees to educational conferences on STEP partnerships. To be eligible, nominees had to have experience running multiple partnerships that linked universities with K-12 schools for the purpose of improving math and / or Science education. This group of professionals generated a total of 133 nominees.

     The potential panelists were contacted by telephone and e-mail over the summer of 2004 to determine their availability and willingness to participate in the project. Telephone interviews were conducted of each potential panelist to gather demographic and background information. As the sample was put together for the purpose of modeling partnerships and evaluation we screened for people with experience on a number of partnerships or projects from a variety of backgrounds. Following the interviews, the 32 final panelists were selected based on their level of experience, diversity, and availability.

     The panel consists of 15 women and 17 men from 30 states. There were 29 white one African American, and two Native Americans on the panel. They averaged 12.7 years of experience with STEM education and half have worked as K-12 classroom teachers. Most of the panelists have graduate degrees and about half hold Doctorates. We were concerned about the diversity of the panelists. Further assessment of the source lists and nominations found that the respondents were representative of individuals charged with organizing partnerships. While the panel consisted of 32 participants, only 30-31 of them participated in any given round.

     The intention of this portion of our study is to develop a range of possible models of partnering for further research. Thus, we make no claims that all partnerships or for that matter all MSP’s are represented by this panel. Since completing the panel survey we have identified an unanticipated source of bias in our sample. By using experience as our first selection criteria we created a panel made up of experienced professionals many of whom have initiated their own MSP’s. Since the panelists all have a great deal of experience, they also tend to be senior level persons in their respective organizations. Thus, one early hypothesis we planned to test, that partnerships formed from the top-down would differ from partnerships that were initiated from the bottom up could not be fully explored.

About the Survey

The Delphi panel consisted of the four “rounds” of online surveys that were intended to take about four hours each to complete. The surveys were administered over a period of about six months during the fall of 2004 and winter of 2005. Each round of the study called upon the respondents to approach the topic of STEM partnerships from a different perspective. In round one, the respondents were encouraged to be in a descriptive mode as they discussed their history and general experience of working with STEM partnerships. In round two, respondents were encouraged to be in a conceptual-judgment mode as they were asked to react to key factors identified in the research literature (and in our initial model) as critical elements in the formation and successful operation of partnerships. In round three, respondents were encouraged to be in a judgment-evaluation mode as they were asked to offer peer review assessments of proposals for forming STEM partnerships as if they were on a National Science Foundation review panel. In round four, we pursued three objectives. First we revisited issues from earlier rounds in which had found disagreement. In some cases, there were differences of opinion among panelists but, we also needed to probe differences in opinion from one round to the next where individual responses had shifted with the participant’s frame of reference. Next we asked questions that panelists expressed an interest in pursuing further. Finally, we explored some additional questions about evaluation and learning within partnerships.

Analysis and Validation

     The approach to analyzing qualitative data in this study derives from Peirce’s “abduction” model (Kelle, 1997; Levin-Rozalis, 2000). In this approach, hypothetical relationships in the data are assumed, tested, modified or discarded in a back and forth deductive and inductive approach to analysis. Hypothetical models can be derived from the literature, the data, and even from hunches. In this study a formal logic model (Yin, 2003) was developed from the literature. This model was modified after the first round of data collection to include boxes for coercive and environmental variables. The coding procedure started with assembling the data according to the logic model and then proceeded to identification of variables from the data that are consistent or inconsistent with each category. We used the NVivo computer program as an analysis tool for the qualitative data as it allows for several unique approaches to testing relationships. The qualitative data were combined with the quantitative survey data (based on likert questions asked during each round) as appropriate. Questions in each round were designed to address one aspect of the major conceptual elements of linkages between concepts. Each question coupled open ended and likert scale items for comparative purposes. Our results were validated by triangulating the existing literature with our quantitative and qualitative data.


     This research was designed to develop a model of partnership that included explanatory elements drawn from our expert panel. We seek to better understand why organizations choose to work together and what impacts these relationships have on STEM education. The first step in evaluating the results of this study is clustering the data according to our model’s large concepts. The data collected in each round reflected both different approaches to looking at partnerships and different parts of our model. In the first round, the panel responded to questions from an individual perspective primarily about partnership formation. The panel shifted to an organizational perspective and explored both formation and operation of partnerships in the second round. The third round placed the panel in the role of evaluator and challenged the participants to identify linkages and interactions while the fourth round tied things together.

     The starting point for this study was to explore the meaning of the term partnership. We wanted to make sure that we had a common understanding of the core meaning of the term among our panelists. It was anticipated that there would be differences in the focus of the partnerships (e.g., content area, service population) but we wanted to nail down the boundaries and scope of the thing we are calling a partnership. This was important because we wanted to capture with our logic model the range of partnering structures, activities, interactions, and processes that contribute to partnership outcomes. However, we found that the members of our panel have four different understandings about what constitutes a partnership. Further, the overwhelming majority (93%) did not define partnerships as a things so much as agreements or a processes.

     In the first round, we wanted to gauge the impact of two categories of variables thought to impact partnership formation, embeddedness and strategic needs. The results tell us two different kinds of stories. The first is about the relative importance of various elements of these two variables. However, a second story explains more generally how the panelists approach partnering. Both organizations and individuals can form partnerships but our panel reported a tendency toward individual relationships and away from strictly organizational relationships. In addition, the organizational relationships that do exist are often embodied in very specific sets of actors. We also noted that the most critical initiating conditions for forming partnerships involved mutual benefits rather than enhancing organizational identity.

     The implementation of a partnership and its programs was discussed in two ways. First there is the building of the partnership itself. The panel noted the importance of trust and most often cited two years as the time it took to build relationships such that programs might be designed and implemented. As a partnership matures, it can begin to focus on specific programs. Programming shifts attention from partnering and partners to the client (e.g., students, education community) and to specific needs or goals (e.g., improving standardized test scores, student retention, teacher training, curriculum development, or school reform). Among strategic needs only mutual goals was rated highly important. Other potential needs (e.g., enhancement of reputation, individual professional promotions, finances, etc.) rated less important to partnership formation.

     The hypothesized importance of embedded relationships was explored in two ways. Likert scale items were used to test the relative importance of embeddedness variables drawn from the literature (e.g., trust, length of relationship, frequency of interaction) while open-ended questions were used to elicit additional ideas from the panelists’ experience. The panel rated reputation, trust, and good communication highly important but length of relationship, potential for public relations, or gains in organizational prestige were less important. The panel suggested financial restrictions and other conditions in aid as well as government mandates as additional potential coercive divers of formation. However, when tested in round two these were not rated as important to the panel as embedded relationships.

     In addition to the positive questions about partnership formation, we asked our panel of experts what mistakes might be made along the way. The panel identified several things that sound like good ideas but that their experience has shown can cause problems (e.g., too much focus on process, involvement of very high level executives, meetings, rigid authority, extra partners, leadership training social events / retreats). One point several panelists noted was that although regular meetings seem like a great idea, they and more social events could provide opportunities for trouble. A more strategic approach to meeting, when specific agenda items need to be handled may be more productive. Also, rookies often believe that having a common vision among partners is important to partnerships. Instead, the panel pointed out that time spent chasing a common vision might be better spent on focusing on specific achievable goals. All partners need not share all goals as long as the separate goals compliment one another.

     Our panel provided additional advice on what constituted good partners the primary one being that they are advocates for STEM education. Champions are seen as important motivators within partnerships. Experience, education, administrative skill, and connections to local community organizations are reported to be much less important to being a good partner than passionate advocacy. However, turnover in partnership staff is seen as high and as a significant cost to partner organizations.

     The typical frequency of interactions with partners is at least weakly for the individual or some member of their organization and all partner organizations typically get together monthly (60%) or yearly (30%). The most reported working relationship is one that is reciprocal, meaning that partners work in a collaborative fashion. One third of the panel said that their interactions with partners almost always deal with partnering activities while two thirds report that their interactions with individuals with their partnerships rarely or almost never refer to partnering activities and usually refer to regular work. This suggests that much of the interaction within partnerships is done by people who regularly work together.

     Another measure of organizational interaction is the question of who is in control. We asked the panel a series of questions related to power and control (e.g., Is a single person in charge? Is a single organization in charge? Is power and control equally divided?). The responses produced a bimodal distribution about half of the panel voting for singular controls and unequal power and half voting for decentralized control and equal power. The distribution broke along gender lines most men seeing their partnerships more centrally controlled while most women saw power and control in their partnerships as more shared.

     In round three, we asked the panel to play the role of evaluators and to review two proposals for STEM partnership projects. The first was a large statewide initiative to develop a new science course. The project involved a large number of organizations with little history of interaction, was well financed, and had a statewide mandate in response to a new state law that passed in response to a specific academic need. The second proposal was built on established relationships, around a general recognition of a weakness in student achievement, and involved an unorthodox approach to improving educational outcomes by standardizing local curricula in response to student hyper-mobility.

     The panel universally disliked the first proposal citing the lack of relationships and rejecting the top-down mandated approach. They did not believe that a working partnership would form nor did they think the project would succeed. This is further evidence that mandates (and other potential coercive variables) do not drive partnership formation. Despite the untried nature of the second proposal’s content the panel thought it likely to succeed and that a working partnership would continue.

     When reading these scenarios as evaluators, the panel was more interested in organizational relationships and their work history than in individual relationships. This differs from round one in which, responding from an individual perspective, most panelists emphasized interpersonal relationships. How the work would be distributed was of great concern in the statewide proposal. Determining appropriate evaluation criteria was very important in both cases. In neither case were the costs associated with interactions of large numbers of people cited as significant limits to success though they were seen as more of a limit in the statewide case. In both cases, our panel reported that it was likely that the outcomes predicted in the scenarios would be impacted by factors outside of the partners’ control. The panel reported much greater agreement on the likelihood of organizations learning new things and that learning would result in some organizational transformation in the case of the unorthodox intervention.

     In a discussion of outcome measures, the panel reported that their partnerships were highly successful at providing professional development for teachers, doing curriculum development and raising money. Improving academic performance, retention of teachers and students, and generating good public relations, and improving pedagogy at colleges and universities were considered less likely to be achieved.


Stepping Through Our Logic Model – Partnership Formation

     Our process model was developed assuming that partnerships would form among partners within existing embedded relationships and around a specific strategic need set in a particular environment. However, we were made aware of an important problem when we read the responses to the question that asked the panelists to define a partnership in the context of the MSP program. Our Delphi panel gave us four conceptually different definitions of partnership. These differences continue to be important because when we asked the panel about partnership formation we had certain expectations about what exactly was being formed.

     Our logic model of the partnership process assumes that a thing called a partnership is being formed. Given the research literature, we expected that all members of our Delphi panel held similar entity-based conceptualizations of partnership in which memberships, boundaries, and formal and informal organizing structures designed to achieve specific functions, played a major role. However, this was not the case. Our panel did not describe clear boundaries (who was in or out), a definitive scale (partnerships might include individuals, single schools or whole districts with little distinction), or well-defined routines and procedures for their partnerships. Only two of the panelists viewed partnership in this entity-based way and their organizations were specifically set up as bridging organizations and service providers.

     Firestone and Fisler, citing Goodlad, (1988), state that “the term partnership implies several meanings and may suggest ultimate outcomes to be achieved, but its core definition points to a deliberately designed and formalized agreement between different kinds of organizations without specifying purposes” (2002, p. 450). From this perspective, a partnership is an agreement. The important elements of success should thus be found in the delivery of agreed upon activities or programs to teachers, school administrators and students according to the terms and conditions of the agreement. About one third of our panelists described an agreement-based conceptualization of partnership with predetermined goals aimed at improving STEM education.

     A third group, nearly two thirds of the panelists related a process-based conceptualization of partnership in which relationships are built up over time to enhance levels of trust and cooperation. Brinkerhoff describes this type of partnership as “a dynamic relationship among diverse actors, based on mutually agreed objectives, pursued through a shared understanding of the most rational division of labor based on the respective competitive advantages of each Partner” (2002, p. 14). In this view a great deal of attention is paid to patterns of communication and the forms of reciprocity that develop between partners. Brinkerhoff posits two explanatory dimensions; mutuality, which encompasses ideals of partnering and enhancement to organizational identity, which provides the motivation for choosing partners (2002). Goal setting and activities are an integral part of the partnering process.

     A final, minority view was of partnerships as a venue or opportunity to interact. Rose argues that there exists a “third space” that lies between the state or public and the family or individual (Rose, 1999). Within this space there is an opportunity for interaction that is neither coerced by authority nor based solely on personal influence. It is a space based on mutually beneficial alliances in which negotiations can take place to develop goals, set agendas, and to coordinate actions (Whetten, 1981). From this perspective specific goals or activities are less important than the potential value added of the collaboration. The venue-based conceptualization is a place-based view of what Sirotnik and Goodlad describe as a seemingly unattainable ideal partnership consisting of “mutually collaborative arrangements between equal partners working to meet self-interests while solving common problems” (Sirotnik & Goodlad, 1988).

Examples of Four Conceptual Definitions of Partnership from the Delphi Panel

Entity Based
“An independent non-profit whose mission is to enhance the capacity of people, companies and organizations to develop and apply science and technology and compete responsibly in the global marketplace.”

Venue Based
“A partnership provides the opportunity to bring together individuals often with diverse multiple expertise and with similar interests to address issues of importance to the partners in their endeavor to enhance STEM initiatives”

Process Based
“Partnership means the networking between education (or a particular level or discipline of education) and another entity, such as government or industry, in order to enhance the value of one or both partners and/or provide information between partners.”

“Working together in an organized way to reach mutual goals which would be hard or impossible to reach by yourself.”

"Partnership is a collaborative working relationship in which each partner respects the expertise of the other. The partners plan together and work together to achieve mutual goals, sharing information and decision-making responsibilities."

Agreement Based
“A partnership is an agreement made by individuals on behalf of their respective organizations, to define joint goals that will likely improve science/math teaching and learning in the region and to commit resources toward reaching the common goal.”

“A mutual agreement among partners to achieve specific goals. Each organization has identified clear roles of responsibility and is accountable for carrying out these activities and responsibilities.”

     We do not view these categories as mutually exclusive types. All our respondents had provided is their dominant discourse for defining partnerships. They were able to discuss questions about process even if their definition was agreement or entity-based but this has led us to explore how important these definitions are. It also taught us that respondents might not share a common language of partnering.

     Given the apparent ambiguity surrounding the term partnership among our panelist’s we will still explore the lifecycle of a partnership from the perspective of our most fully developed logic model. Something that our panel identified as a partnership was being formed. Therefore, whatever the specific conceptualization of what a partnership is as an ideal, our panelist could still describe the process by which it got started. And, by exploring their ideas about the lifecycle of an MSP we might better understand the ways in which partnering impacts STEM education.

     While embedded relationships can be positive or negative, Granovetter argues that partnerships are built from a set of embedded relationships that encourage organizations to partner (Grannovetter, 1992b). Therefore, in the first round of the Delphi study we asked a series of likert and open-ended questions about the importance of a variety of embeddedness variables to the formation of the panelists’ previous and current partnerships. We found that measures of embeddedness like trust, a potential partner’s reputation, open communication and the presence of a passionate individual were indeed important to the formation of a partnership. However, other measures of embeddedness, length of relationship, organizational gains, and coordination potential were not considered to be critical.

     Burt argues that organizations enter partnerships for rational reasons (e.g., to gain resources, increase control or coordination, to gain knowledge or expertise) (1992). However, among these potential strategic needs, only having a mutual goal rose to the level of critical importance to our panel. The implication here is that needs do really not drive people to partner but that they partner with people who have similar goals. This evidence supports Gulati’s argument, that a purely rational approach to choosing partners is less important than the existing environment and socially embedded set of relationships (Gulati, 1998). Our panel told us that good partners are first and foremost advocates for STEM education. Experience is a distant second in importance. Further, diplomas don’t matter. Surprisingly, local community groups are also rated low on a scale of good potential partners.

     Due to responses from the first survey, we added coercion variables to our model. However, responses to questions in later rounds suggest that they are relatively weak motivators for formation further strengthening the view that partnership formation is not so much driven but taken on as an opportunity. Grant requirements, conditions in aid, rules, mandates and federal policy goals do not play as large a role in the partnering activities of our panel as we had expected. This may be at least partially explained by the broad range of funding sources used to finance math and science initiatives. However, it is clear that the MSP program’s policy inducements are more carrots than sticks.

Partnership Operation

     In this study we attempted to distinguish between collaborative partnering activities, coordination and program implementation. In order to identify the unique advantages of partnering, we are trying to distinguish between the regular interactions and work of the partner organizations and work that is unique to the partnership. Therefore, we asked the panel how often their interactions with other partners dealt with partnership related matters. One third of the panel reported that their contacts with partners usually related to partnership activities. However, two thirds reported that their interactions rarely or never related to partnership activities that were not a part of their regular work. This leads us to wonder, what about these partnerships is different from the way these organizations would ordinarily interact?

     We asked a series of questions related to power and control among the partners. In these questions we wanted to explore our panels’ experiences with other individuals and organizations involved in partnerships. We initially anticipated a difference, hypothesizing that typically one organization might be the more controlling or that a single individual would be both a driving force and in charge. However, our panel was split with about half reporting more centralized control and disproportionate power, while the other half reported an equal sharing of control and balanced levels of power.

     This sharp division led to a search for differences among the two groups that might explain why the two groups reported such dissimilar experiences. We looked at their home organizations wondering if university and K-12 partners had different viewpoints but found that no discernable pattern. We checked age, education levels, job descriptions and experience hypothesizing that more senior panelists might be in positions that gave them greater control but still found no relationship between seniority and the differing views about power and control. We finally checked the demographics of our panel and found that men in the panel were reporting much more centralized control, an imbalance of power in which a single organization had the majority of control, and single individuals were likely to make most decisions. The women on the panel reported more balanced power among organizations, that decision-making was more likely to be shared equally, and that they strongly disagreed that power was invested in a single individual.

     Our panel reported that administrative activities like meeting, administrative support and regular staff are critical to the success of partnership operations. However, structured agreements, deadlines, clear dispute resolution procedures, and recruitment of new partners are viewed as less important. Further geographic proximity was not cited as a significant aid to
partnership operation in most cases. These findings suggest that much of the work of the partnerships is being done in a compartmentalized fashion. Thus, implementation and administration of partnership related programs or activities may not require broader interorganizational collaboration.

Connecting Outcomes to Partnering and Programs

     In this study, we make a potentially important distinction between partnering operations and programs. Programs are designed, implemented and evaluated with specific tasks and expected outcomes. A successful program can be seen as one that is effective and efficiently and equitably delivered. However, the process of implementing a program assumes that the choice of program is already made. Therefore, looking at alternative programs is not something that would be done in that context. Within in a broader partnership context there is an opportunity to debate options and make choices. However, It has been found elsewhere that taking opportunities to explore alternatives in the face of performance based initiatives is often overlooked (Moynihan, 2005).

     Single loop learning involves measuring a singular linear process. Within partnering, it is trying to find a better way to implement the current program. It is comparable to a process of continuous quality improvement. Double loop learning however, goes a step further and asks, why are we doing the process in the first place? Are we doing the right thing? Should we try something else? From a broad policy perspective, we have a set of experiments within the MSP program. Some may work better than others, producing changes that we can document. In the future we can choose among the successful experiments to model future interventions.

     We engage in double loop learning when we take an opportunity to choose among options, to change what we are doing and try something else if we are not seeing progress. This type of learning involves a collaborative interaction in a place where goals and options are evaluated new choices are made based on evidence gained through practical experience. We asked the Delphi panel to relate examples of what they learned through their partnerships and how they used that knowledge.

     When asked about the success of a partnership, our panel cited having a passionate individual championing the project as the most significant factor. Key personnel interactions were cited as more important to the success of the partnerships, than were the interactions of top executives. There was a tone of coordinated activity but little description of collaborative interactions. Further, there was a great deal of difficulty in describing how a programmatic intervention delivered a specific outcome. Changes in an intervention driven outcome measure, (e.g., improved student test scores) are difficult to trace back to a specific education program. Environmental noise makes drawing a clear line of causation close to impossible. Therefore, it is difficult to determine what effect they have or to determine if an intervention developed and implemented within a partnership is different from or in any way better (or worse) than one developed by an expert in isolation.

     This is not surprising as authors from several fields have pointed out that studies of inter-organizational relationships, of which partnerships are one type, rarely evaluate outcomes (Gulati & Gargiulo, 1999; Kingsley & Melkers, 2000; Provan & Milward, 2001). Many studies focus on measures of process because such data is easier and more straightforward to collect. However, it is possible that the difficulty sorting through the environmental noise masks a lack of clear logical linkages from activities to outcomes.

     A third of the panel reported that their MSP’s focus on teacher training while two thirds report that they pursue a more holistic approach. Yet, improving teacher content knowledge and pedagogy are the successes most often cited by the panel. Only one third consider improving student outcomes as likely. The panel also reports curriculum development and raising money among their successes. Student achievement, public relations, and community involvement were all cited as less likely to be achieved.


     Organizational transformation can be seen as an institutional change resulting from a “jolt” to the system though a breakdown in current institutional arrangements coupled with the development of new innovations and theories, which then spread to partnered organizations to be reinstitutionalized in some new form (Meyer et al., 1990). The National Science Foundation has provided a jolt to the education system through the MSP program. The use of partnerships as a policy tool implies that they are believed to provide a value added to educational programs. However, our panel had a hard time describing a complete network of connections from partnership formation to program outcomes.

     It is not at all clear that what people say a partnership is, translates into clear consistent patterns of activity. The interactions, collaboration and mechanisms for decision-making and coordination we might expect to find in a multiorganization partnership are difficult to identify. MSP’s are not clearly defined as entities. However, they do not seem to be complete networks either. There appear to be holes in the network, possibly with boundary spanners who can help with coordination but who may turn out to be filters instead. In future work we need to identify the locus of the collaborative partnering activity.

     Panel participants reported reciprocal relationships, which argues for the presence of collaboration, but failed to link various stages of the partnership process. Our panelists could make distinctions in formation activity, but would then take formation and leap to outcomes without a clear description of the partnering operations in between. Two thirds of the panel describes partnership as a process but they do not talk about goal setting or decision-making. They do not report clear stories about learning driving changes yet they claim interaction and learning. Evaluations described at the program level miss questions that capture the potential value added or costs generated by partnering interactions. There are perhaps pockets of collaboration embedded within coordinated activities. Yet, something still seems missing. A key task for future research will be to identify the locus of collaborative partnering activity within an MSP as several possible arrangements exist. It is possible that there are many loci, that they may be linked or not, or there may simply be disconnected pockets of partnering.

     In addition to exploring the collaborative activities of the MSP projects, we need to further explore the gender differences that were identified in the questions about power and control. It is unclear if the differences are an artifact of the language used in the questions, a difference in the perceptions of power among men and women, or a real difference in the partnering activities of our panelists. We hope to get a better sense of this as the coding and analysis of the qualitative data continues. However, as these distinctions may play a role in the implementation of partnership activities, we will be looking for similar differences in our upcoming case study work.

     The most important finding from the Delphi panel thus far has resulted in the breakdown of our entity-based assumptions about policy-induced partnerships (PIP’s). This has led to a search for a way to model the partnership process that does not begin with an entity at the center. If MSP partnerships are more like sets of processes and agreements than they are like venues and entities then we need ways to model and explore partnership phenomena that allow for more flexibility than our current logic model. Borrowing some ideas from social network analysis we are envisioning a set of actors and activities with certain attributes that are in some way connected. By following the operating path of these connections we might develop a better understanding of the connections and interactions among partnership formation, operations and outcomes.


Boyer, E. L. (1981). School / college partnerships that work. Current Issues in Higher Education, 1, 4-10.

Brinkerhoff, J. M. (2002). Partnership for international development: Rhetoric or results. Boulder, Co: Lynne Rienner.

Burt, R. S. (1992). Structural holes: The social structure of competition. Canbridge, MA: Harvard University Press.

Firestone, W. A., & Fisler, J. L. (2002). Politics, community, and leadership in a school - university partnership. Educational Administration Quarterly, 38(4), 449-493.

Goodlad, J. I. (1988). School-university partnerships for educational renewal: Rationale and concepts. In K. A. Sirotnik & J. I. Goodlad (Eds.), School-university partnerships in action: Concepts, cases, and concerns (pp. 3-31). New York: Teachers College Press.

Goodlad, J. I. (1994). Educational renewal. San Francisco: Jossey-Bass.

Grannovetter, M. (1992a). Economic action and social structure: The problem of embeddedness. In M. Grannovetter & R. Swedberg (Eds.), The sociology of economic life. Bolder, Co: West View Press.

Grannovetter, M. (1992b). Problems of explanation in economic sociology. In N. Nohria & R. Eccles (Eds.), Networks and organizations (pp. 25-56). Boston, MA: Harvard Business School Press.

Gulati, R. (1998). Alliances and networks. Strategic Management Journal, 19, 293-317. Gulati, R., & Gargiulo, M. (1999). Where do interorganizational networks come from? American Journal of Sociology, 104(5), 1439-1493.

Kelle, U. (1997). Theory building in qualitative research and computer programs for the management of textual data. Sociological Research Online, 2(2).

Kingsley, G., & Melkers, J. (2000). The art of partnering across sectors: The importance of set formation to network impacts. In L. J. O'Toole, Jr., H. G. Rainey & J. Brudney (Eds.), Advancing public management: New developments in theory, methods, and practice. Washington, DC: Georgetown University Press.

Levin-Rozalis, M. (2000). Abduction: A logical criterion for programme and project evaluation. Evaluation, 6(4), 415-432.

Meyer, A. D., Brooks, G. R., & Goes, J. B. (1990). Environmental jolts and industry revolutions: Organizational responses to distant change. Strategic Management Journal, Special Issue, 93-110.

Moynihan, D. P. (2005). Goal-based learning and the future of performance management. Public Administration Review, 65(2), 203-216.

Oliver, C. (1990). The determinants of interorganizational relationships: Integration and future directions. Academy of Management Review, 15, 2, 241-265.

Osborne, S.P. (2000) Public-private partnerships: Theory and practice in international perspective. London: Routledge.

Provan, K. G., & Milward, H. B. (1995). A preliminary theory of interorganizational networks: A comparative study of four community mental health systems. Administrative Science Quarterly, 40, 1-33.

Provan, K. G. & Milward, H. B. (2001). Do networks really work? A framework for evaluating public-sector organizational networks., Public Administration Review (Vol. 61, pp. 414): Blackwell Publishing Limited.

Rose, N. (1999). Powers of freedom. Cambridge: Cambridge University Press.

Sanders, M. G., & Epstien, J. L. (2000). The national network of partnership schools: How research influences educational practice. Journal of Education for Students Placed at Risk, 5(1), 61.

Sirotnik, K. A., & Goodlad, J. I. (Eds.). (1988). School-university partnerships in action: Concepts, cases, and concerns. New York: Teachers College Press.

Teisman, G.R. and Klijn, E-H. (2002) Partnership arrangements: Governmental rhetroic or governance scheme? Public Administration Review, 62, 2, 197-205.

Turoff, M. (1970). The policy delphi. Journal of Technological Forecasting and Social Change, 2(2).

Turoff, M., & Hiltz, S. R. (1982). Computer support for group versus individual decisions. IEEE Transactions on Communications, 30(1), 82-90.

Tushnet, N.C. (1994) Does it really take a whole village to raise a child?: Mixing metaphor and meaning in the Educational Partnerships Program. American Educational Research> Association Annual Meeting, New Orleans, LA, April 1994.

Whetten, D. A. (1981). Interorganizational relations: A review of the field. Journal of Higher Education, 52(1).

Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA:


1 This paper reports on findings from the National Science Foundation sponsored RETA project, Alternative Approaches to Evaluating STEM Education Partnerships: A Review of Evaluation Methods and Application of an Interorganizational Model, headed by Gordon Kingsley (NSF 02-061 Award #: 0231904 Award Group: Research, Evaluation, and Technical Assistance). return

2 The MSP program has five key features: Partnership-Driven - Partnerships between universities and K-12 institutions are required. Teacher Quality, Quantity and Diversity - The projects are intended to enhance the quality, number and diversity of math and science teachers. Challenging Courses and Curricula - The projects are intended to provide courses and curricula that improve the math and science understanding of students while teaching a range of problem solving and analytical skills. Evidence-Based Design and Outcomes - Programs are to be designed based on the best current research and link evaluation with indicators of partnership success. Institutional Change and Sustainability - The core partners are to commit to strengthening teaching practices on the university side while K-12 institutions commit to providing an environment in which teachers, administrators and other staff can grow for the long term. Further, the partnership (and its impacts) should last beyond the funding of the MSP. return