Intermediary Agencies in Canada:

 

 

 

 

 

Research Policy Studies:
Between Science and Higher Education

 

Amy Scott Metcalfe, PhD

Department of Educational Studies

The University of British Columbia

 

¨§¨

 

Abstract

R

esearch policy is an understudied area of higher education, but has been covered extensively in the field of science policy studies. In this essay I compare the citation histories of two books on research policy, one from the perspective of science policy studies and the other from higher education. Each book utilizes different conceptual frameworks (principal-agent theory and academic capitalism), which may contribute to the differences in their citation networks.

 

 

Introduction

A

 paradox of contemporary higher education scholarship is that as we endeavour to understand the complexities of knowledge systems we can become entrenched in disciplinary “foxholes” that obscure our view of the field around us. Case in point is a recent article by Kivistö (2005), in which he claims that “in the field of higher education research, [principal-] agency theory is still quite unknown and so far it has not been widely explored or applied” (p. 2). One of the “occasional attempts” that Kivistö mentions as an application of the theory to higher education research, an article in the Journal of Public Policy by Braun (1993), is a classic article in the area of research policy studies. The article, “Who governs intermediary agencies? Principal-agent relations in research policy-making,” (Braun, 1993) has been cited 21 times according to the ISI Web of Science, a rather high number of citations.[1] Yet, the problem is not “insufficient” theorization as Kivistö claims, as most of the authors citing Braun also utilize principal-agent theory. Instead, the problem lies in the division between “science policy” and “higher education policy,” as the articles citing Braun appeared primarily in four journals that sit at the juncture between the two fields of study: Minerva; Research Policy; Science, Technology and Human Values; and Scientometrics. It is likely that these articles, which focused on research policy studies, were not recognized by Kivistö as being “higher education research.”

As scholars we are constrained by disciplinary conventions and perspectives, which can lead to an incomplete understanding of various phenomena. The study of research policy suffers from the limited exchange between science policy studies and higher education policy studies. More work on research policy is done from the perspective of science policy, and as such we have little scholarship that helps us to understand how the overall institutional and organizational setting of postsecondary education is affected by academic research. Furthermore, theoretical and methodological differences exist between science studies and higher education studies, which may make the transfer of knowledge between the two even more difficult. Can we reconcile science policy studies and higher education policy studies, especially with regard to research policy studies?

In this essay I aim to provide an example of the division between science policy studies and higher education policy studies by comparing the citation histories of two books that are well known in each area: Guston’s Between Politics and Science: Assuring the Integrity and Productivity of Research (2000) and Slaughter and Leslie’s Academic Capitalism: Politics, Policies, and the Entrepreneurial University (1997). Both books have been well-received in their respective fields, and although they have similar subject matter, the analysis is very different. This comparison shows that that although the books essentially describe the same phenomenon (the political, economic, and social structure of research with particular emphasis on technology transfer) they are cited by two distinct networks (science policy scholars and higher education policy scholars), with limited overlap. Implications for research policy scholarship are then discussed.

 

Discovering Unique Networks

B

ibliometric analysis, the systematic examination of scholarly communication, is a form of both higher education research and science studies in itself, so its application to this context is reflexive and purposive. Through the proliferation of electronic databases and the creation of the ISI Web of Science (which includes the Science Citation Index Expanded, the Social Sciences Citation Index, and the Arts & Humanities Citation Index), analysis of citation histories has become a more accessible method for researchers. There is even a journal dedicated to this type of inquiry, titled Scientometrics, and the method has been applied to research policy studies (see Rinia, 2000).

 However, the use of citation histories has not been without critique (MacRoberts & MacRoberts, 1989) or questions regarding its theoretical roots (Luukonen, 1997). Bibliometric analysis, of course, can only be a good as the dataset, and the network of journals included in the Web of Science is not complete. Many journals are not indexed in the database, and as such their articles only appear when they are cited in other journals.  This prevents researchers from knowing the complete picture of “who cites whom.” However, the ISI Web of Science does provide a glimpse into the world of scholarly communication and permits a rudimentary analysis of trends in academic publishing.

When one examines the citation histories of Guston’s book on boundary organizations (Guston, 2000) and Slaughter and Leslie’s book on higher education funding (Slaughter & Leslie, 1997) it is clear that there are distinct audiences for each. Both books develop a theoretical framework for studying aspects of research policy and they both specifically examine technology transfer, so they are comparable works in that sense, but they seem to speak only to the members of their own disciplinary “foxhole.” Using the analytical capabilities of ISI Web of Science (a citation index), I found that these books have had remarkably dissimilar citation histories given their common subject matter. Below I present the results of two bibliographic inquiries. First, I present the network of common authors who cite the two books, which is intended to explore the degree of overlap of individuals who have recognized these books in their own work. Second, I present the network of common journals that contain these citations.

 

Citation Histories—Authorship

W

hile citations are not a proxy for intellectual agreement (a person might cite a book only to claim it as inadequate for his or her purpose), I felt that on the whole an examination of specific citations would at least demonstrate the amount of field convergence between the two books. I entered each book into the “cited reference search” function of the ISI Web of Science in two separate searches.[2] Because the database is built upon journal publications, only journal citations of these books would appear in the results. The query resulted in 226 citations for Academic Capitalism and 39 for Between Politics and Science. The total number of citations is not relevant here, as I am more interested in the overlap (or lack thereof) in citations between the two books. I entered the citing authors into a spreadsheet and then analyzed the data with a social network software called UCINET. I entered each name independently (each author of a co-authored publication counted as a separate entry). Figure 1 is the output of the Netdraw function of UCINET for the author network.

           

Figure 1. Citation network for Slaughter and Leslie (1997) and Guston (2000).

 

            Of the 282 authors who cited Academic Capitalism and Between Politics and Science, only seven cited both books. As it happened, these citations occurred in just four publications, and in each article both the Slaughter and Leslie book and the Guston book were cited. In other words, in no instance did an author cite Academic Capitalism in one article and Between Politics and Science in another. There were many cases where authors cited one of the books in multiple publications, but nobody cited one book in an article and the other book in another article. The only joint citations were in these four publications: “The shift in academic quality control” (Hemlin & Rassmussen, 2006), “From ‘endless frontier’ to ‘basic science for use’: Social contracts between science and society” (Slaughter & Rhoades, 2005), “Bureaucrats, brokers, and the entrepreneurial university” (Vogel & Kaghan, 2001), and “Competition and pluralism in the public sciences: The impact of institutional frameworks on the organisation of academic science” (Whitley, 2003). These four articles appeared in three journals: Science, Technology & Human Values (two articles); Research Policy; and Organization. The titles of these journals are significant, as they point to the (limited) intersections between higher education policy studies and science policy studies. An examination of the journals in which authors cited the focal books follows in the next section. A more thorough discussion of these articles and their content is warranted, but is beyond the scope of this essay.

 

 

 

Citation Histories—Journals

A

 total of 117 different journals contained articles that cited Academic Capitalism or Between Politics and Science. Only ten journals contained citations for both books, although as noted above only three journals held citations for both books in the same article. Figure 2 graphically represents the journals that contained articles that cited Slaughter and Leslie (1997) and Guston (2000).

 

 

Figure 2. Journals that contained articles that cited Slaughter & Leslie (1997) and Guston (2000).

 

            These ten journals (Society; Research Policy; Organization; Minerva; Global Governance; Social Study of Science; Futures; Science Communication; Science, Technology & Human Values; Technovation) are clearly more oriented towards science policy studies than higher education policy studies. Only one journal of the group is classified in the field of education (Minerva: A Review of Science, Learning and Policy), and even it is largely concerned with science as the subtitle suggests.

 

Discussion

T

he two books in this comparison have been widely cited by other scholars, but tellingly there has been little overlap between their citation networks. This lack of overlap can be seen in the separate networks of authors and in the journals that contain the citations. Simply stated, only seven authors have cited both books, out of a total network of 282 citing authors. These seven authors found the books to be on similar enough ground to be included in the same discussion since they cited the two books in the same articles. Given the similar subject matter it is surprising that there is not more overlap, particularly as Guston cited Academic Capitalism in Between Politics and Science. Although this citation would not appear in the Web of Science (because it is a citation appearing in a book), one might think that it would lead at least a few more scholars to the work of Slaughter and Leslie.

It is interesting that Guston’s book, with chapters titled “Assuring the Integrity of Research” and “Assuring the Productivity of Research,” was not cited at all in higher education journals. Based on the citation histories it seems that the science policy process, where Guston focused his attention, is outside the analytical jurisdiction of most higher education researchers, even though intellectual property and technology transfer policies (which Guston addressed) are central to the changes facing contemporary universities.

The disconnect between the two fields of policy studies is not yet resolved by the area of “research policy,” although one might think this a natural area for knowledge sharing between science policy and higher education policy. Indeed, the journal Research Policy is not overly concerned with the academic context. Rather, Research Policy publishes articles that view universities as the birthplace of “basic” science, and as the locus of “knowledge spill-overs” that affect industrial “cluster” formation, a key part of the “innovation” cycle. In other words, Research Policy is concerned with knowledge transfer from the academy, but not the transformation of the academy that occurs through the research process.

 

Conclusion

T

he gap between science policy studies and higher education policy studies threatens to jeopardize the rigor of research topics that fall between the two areas. The science policy process cannot be understood without considering academic science as a subset of university culture, and higher education policy cannot be completely regarded without recognizing the influence of external forces such as the larger research community. Research policy is a natural fit for both areas of policy studies, and new theoretical frameworks and methodologies may develop from a shared sensibility from both fields.

In order to do this, the scholarly community will need to be more aware of and open to ideas from other fields of policy studies. Researchers can broaden their scope during the initial phases of their work as they generate literature reviews and consider various theoretical frameworks. Editors can participate in the broadening of research policy studies by selecting reviewers from various disciplines and by suggesting that author’s consider scholarship outside of their own narrow area of interest. Furthermore, faculty who teach policy studies might choose to adopt an interdisciplinary strategy, selecting readings and coursework in a more inclusive fashion.

 


 

 

References

 

Braun, D. (1993). Who governs intermediary agencies? Principal-agent relations in research policy-making. Journal of Public Policy, 13(2), 135-162.

 

Guston, D. H. (2000). Between politics and science: Assuring the integrity and productivity of research. Cambridge: Cambridge University Press.

 

Hemlin, S. & Rasmussen, S. B. (2006). The shift in academic quality control. Science, Technology & Human Values, 31(2), 173-198.

 

Kivistö, J. (2005). The government-higher education institution relationship: Theoretical considerations from the perspective of agency theory. Tertiary Education and Management, 11, 1-17.

 

Luukonen, T. (1997). Why has Latour’s theory of citations been ignored by the bibliometric community? Discussion of sociological interpretations of citation analysis. Scientometrics, 38, 27–37.

 

Macroberts M. H. & Macroberts, B. R. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40, 342–349.

 

Rinia, E. J. (2000). Scientometric studies and their role in research policy of two research councils in the Netherlands. Scientometrics, 47(2), 363-378.

 

Slaugher, S. & Leslie.L. (1997). Academic capitalism: Politics, policies, and the entrepreneurial university. Baltimore: Johns Hopkins Press.

Slaughter, S. & Rhoades, G. (2005). From ‘endless frontier’ to ‘basic science for use’: Social contracts between science and society. Science, Technology & Human Values, 30(4), 536-572.

 

Vogel, A. & Kaghan, W. N. (2001). Bureaucrats, brokers, and the entrepreneurial university. Organization, 8(2), 358-364.

 

Whitley, R. (2003). Competition and pluralism in the public sciences: The impact of institutional frameworks on the organisation of academic science. Research Policy, 32(6), 1015-1029.

 

 

 

 

 



[1] Search conducted May 16, 2006 for the period between years 1993-2006. Interestingly, a special issue of Science and Public Policy edited by Braun and Guston (2003) and dedicated to the application of principal-agent theory to research policy studies was not included in the found citations because the journal is not part of the ISI Web of Science. Thus, technological and academic barriers exist that further limit our understanding of the overlap between fields and disciplines in higher education.

 

[2] The searches were conducted in May 2006 and included the span between the year of publication for each book and the date of the search.

 

Author’s Note: A longer version of this paper was presented at the annual meeting of the Canadian Society for the Study of Higher Education (CSSHE), held at York University May 29-31, 2006.

 



Higher Education Perspectives. ISSN: 1710-1530