The Limits of the Numerical: Metrics and the Humanities in Higher Education

Aashish Mehta
Global and International Studies
UC Santa Barbara

Christopher Newfield
English
UC Santa Barbara

Participants

This Residential Research Group sought to develop a historical, cultural, and economic account of metrics in higher education. It investigated the adoption of quantitative measures of university learning, research, and value. It evaluated their educational and intellectual impacts on colleges and universities.

Numerical indicators have been reshaping higher education for decades. Most people rank colleges like sports teams, weigh the salary benefits of a bachelor’s degree, and compare the expected earnings of majoring in computer science vs. art history. The same kind of thing is happening globally: national educational systems are now being compared and governed through massive scoring projects like the OECD Program for International Student Assessment, or PISA; rankings from U.S. News and World Report directly affect how students apply to universities and how resources are spent on campus. Bibliometric techniques are having a similar impact on research funding and faculty promotion. The research group evaluated the widely varying effects of quantifying the benefits of higher education.

The group investigated several areas of special concern. One, already mentioned, was: do quantitative measures of the value of a university degree crowd out awareness of non-economic, indirect, and social benefits? Another related to learning assessments: how does testing learning affect learning itself? A third had to do with research quality: what are the impacts of bibliometric analysis and its analyses of research output?

Metrics are now used to audit the quality and quantity of outputs in nearly all organizations, and the university is no exception. A feature of audit is that “any number beats no number.” In practice, nearly everyone involved in quantitative evaluation agrees that the metrics are flawed, and yet most insist that they can always be improved and are much better than the alternatives—either qualitative assessment, which is seen as too subjective and variable, or no assessment at all. 

The “Limits of the Numerical” group both critiqued existing metrics and proposed alternatives. It was focused on three tasks. First, it analyzed how specific indicators are produced.  Second, it identified what practices and values the creation of indicators obscures in the process of foregrounding measurable features. Third, it outlined ways of describing higher education that do not diminish or distort the qualities and experiences that students, faculty, and communities care about.

The group’s study had several facets. At UCHRI at UC Irvine during Spring 2018, it brought together three literary and cultural theorists, two geographers, and two economists. “Limits of the Numerical: Metrics and Higher Education” is a project based at UC Santa Barbara, and is funded by a NEH Collaborative Research Grant (2017-2019) awarded to UCSB and Texas A&M. This project in turn is one of three strands of the “Limits of the Numerical” initiative. A second strand is based at the University of Cambridge (funded by ISRF), where the group focuses on the effects of quantification in health care. The third is housed at the University of Chicago (funded by Mellon), whose team studies quantification in climate change debates.