The Limits of the Numerical: Metrics and the Humanities in Higher Education

Participants (from left): Mukul Kumar, Aashish Mehta, Heather Steffen, Laura Mandell, Zach Bleemer, Chris Newfield, Chris Muellerleile


Christopher Newfield
UC Santa Barbara
Aashish Mehta
Global & International Studies
UC Santa Barbara



Our Residential Research Group seeks to develop a historical, cultural, and economic account of metrics in higher education. It investigates the adoption of quantitative measures of university learning, research, and value.  It evaluates their educational and intellectual impacts on colleges and universities. 

Numerical indicators have been reshaping higher education for decades. Most people rank colleges like sports teams, weigh the salary benefits of a bachelor’s degree, and compare the expected earnings of majoring in computer science vs. art history. The same kind of thing is happening globally: national educational systems are now being compared and governed through massive scoring projects like the OECD Program for International Student Assessment, or PISA; rankings from U.S. News and World Report directly affect how students apply to universities and how resources are spent on campus. Bibliometric techniques are having a similar impact on research funding and faculty promotion. Our HRI group is evaluating the widely varying effects of quantifying the benefits of higher education.

We are investigating several areas of special concern. One we’ve already mentioned: do quantitative measures of the value of a university degree crowd out awareness of non-economic, indirect, and social benefits? Another is learning assessments: how does testing learning affect learning itself? A third is research quality: what are the impacts of bibliometric analysis and its analyses of research output?

Metrics are now used to audit the quality and quantity of outputs in nearly all organizations, and the university is no exception. A feature of audit is that “any number beats no number.” In practice, nearly everyone involved in quantitative evaluation agrees that the metrics are flawed, and yet most insist that they can always be improved and are much better than the alternatives—either qualitative assessment, which is seen as too subjective and variable, or no assessment at all. 

The “Limits of the Numerical” group is both critiquing existing metrics and proposing alternatives. We are focused on three tasks. First, we are analyzing how specific indicators are produced.  Second, we identify what practices and values the creation of indicators obscures in the process of foregrounding measurable features. Third, we are outlining ways of describing higher education that do not diminish or distort the qualities and experiences that students, faculty, and communities care about.

Our study has several facets. At the HRI at UC Irvine during Spring 2018, we have brought together three literary and cultural theorists, two geographers, and two economists. “Limits of the Numerical: Metrics and Higher Education” is based at UC Santa Barbara, and is funded by a NEH Collaborative Research Grant (2017-2019) awarded to UCSB and Texas A&M. This project in turn is one of three strands of the “Limits of the Numerical” initiative. A second strand is based at the University of Cambridge (funded by ISRF), where the group focuses on the effects of quantification in health care. The third is housed at the University of Chicago (funded by Mellon), whose team studies quantification in climate change debates.

The HRI group members are as follows:

Christopher Newfield, Literature and American Studies, UCSB (Convener). Newfield’s main research is in critical university studies. He is most recently the author of The Great Mistake: How We Wrecked Public Universities and How We Can Fix Them (Johns Hopkins UP, 2016), and co-edits the blog Remaking the University.

Aashish Mehta, Development Economics, UCSB (Co-convener). Mehta studies shifting employment patterns around the world, and their consequences for economic inequality and the demand for education. His most recent papers ask what happens when many countries attempt to industrialize at the same time.

Heather Steffen, Cultural Studies, UCSB. Steffen is a critical university studies scholar working as a Postdoctoral Scholar in the Chicano Studies Institute and Department of English at UCSB. Along with her collaboration on the Limits of the Numerical Project, she is writing a book about academic labor, Useful Work: Imagining Academic Labor in the American University.

Chris Muellerleile, Human and Economic Geography, Swansea University, UK. Muellerleile studies financial markets and information economies, including the ways these two intersect. He is currently researching and writing about the ‘marketization’ of academic publishing, including open access journal publishing.

Laura Mandell, Professor of English and Director of the Center of Digital Humanities Research at Texas A & M University, tries to bridge distant and close reading methodologies through work as a digital textual editor. Her recent book is titled Breaking the Book: Print Humanities in the Digital Age (2015).

Mukul Kumar is a PhD Candidate in the Department of City and Regional Planning at UC Berkeley. Mukul’s research interests include urban, environmental, and development studies. His research on the financialization of public universities has been published by the Berkeley Journal of Sociology and the Institute for Research on Labor and Employment.

Zach Bleemer is a PhD Candidate in the Department of Economics, UC Berkeley. Zach is the director of the University of California ClioMetric History Project at the Center for Studies in Higher Education at UC Berkeley. His research examines the long-run consequences of young Americans’ post-secondary education and specialization decisions, with a side-interest in the computational analysis of structured text. Zach’s most recent paper in the Journal of Public Economics examines socioeconomic differences in students’ responsiveness to information about the high returns to college education.