Thursday, September 24, 2009

Research Excellence Bullshit

So, there's another consultation document on the Research Excellence Framework -- "the new arrangements for the assessment and funding of research in UK higher education institutions that will replace the Research Assessment Exercise (RAE)". A wonderful document indeed, literate and elegantly written, revealing much thought and reflection on the nature of the university in the best traditions of Arnold and Leavis. Of course. Still, perhaps it isn't quite what we might hope for.

Ok, ok, I jest. It isn't at all what we might hope for, though it is the sort of egregious crap we've come to expect. How about this, for example: "As an indication of our current thinking we propose the following weightings" (between different components of assessment); "Outputs: 60 per cent. Impact: 25 per cent. Environment: 15 per cent." Hold on! Impact? Impact? What's that?

Well, the document gives "a common menu of impact indicators" under various headings to help us out. Here are the headings ...

  • Delivering highly skilled people [as evidenced e.g. by "Staff movement between academia and industry, Employment of post-doctoral researchers in industry or spin-out companies".]
  • Creating new businesses, improving the performance of existing businesses, or commercialising new products or processes
  • Attracting R&D investment from global business
  • Better informed public policy-making or improved public services
  • Improved patient care or health outcomes
  • Progress towards sustainable development, including environmental sustainability
  • Cultural enrichment, including improved public engagement with science and research
  • Improved social welfare, social cohesion or national security
  • Other quality of life benefits
Right. Let me see if I understand. If you are a medieval historian, an editor of Euripides, a Shakespeare scholar, or indeed just a logician trying to understand the philosophical significance of Gentzen's work on the consistency of arithmetic, then 25% of your score in son-of-RAE is going to be for "impacts" utterly irrelevant to your projects and concerns?

I'm being unfair, you say: arts subjects at least get into the frame under the heading "Cultural enrichment". You might think so: but in fact we are told that possible indicators of that are -- I kid you not -- "Increased levels of public engagement with science and research (for example, as measured through surveys). Changes to public attitudes to science (for example, as measured through surveys). Enriched appreciation of heritage or culture (for example, as measured through surveys). Audience/participation levels at public dissemination or engagement activities (exhibitions, broadcasts and so on). Positive reviews or participant feedback on public dissemination or engagement activities." Yep, and we are also told that impact does not include "we do not intend to include impact through intellectual influence on scientific knowledge and academia".

Ah, there's a chink of light perhaps: not everyone is to be ranked for impact, if I've got it right? -- a department's return will rather involve a series of "case-studies" of impactful individuals. Well, yes, you can just see the guys and gals in M&E sitting around trying to find a smidgin of impact somewhere between them.

Brilliant. Well, I know will happen; you know what will happen; HEFCE no doubt know what will happen when traditional humanities departments come to fill in the impact case studies on which 25% of their overall rating is going to depend.

They'll have to bullshit.

Added later. My jest about the M&E contingent having a bit of difficulty cooking up an impact statement was truer than I realized. Eric Schliesser, currently at Leiden, writes in a comment on the Leiter blog that "in places where 'impact' is already playing a prominent role (say, in Netherlands and Flanders), certain subjects (e.g., analytic metaphysics,) have very little chance to receive coveted research grants (now almost the sole source for PhD funding). Yesterday, Michael della Rocca gave a terrific talk on the three-dimensionalism vs four-dimensionalism debate. It generated great discussion. But the people in attendance were hard-pressed to name a sole Dutch philosopher who is working on the topic ... Of course, other subjects (e.g., philosophy of technology, applied ethics, decision theory, semantics, logic, normative ethics, etc) have an easier time in articulating the impact factor and are generously funded."


Shalla said...

Impact: Influence; effect: e.g., the impact of Einstein on modern physics.

Not that Merriam-Webster's considered highly credible these days.

In fact, "Changes to public attitudes to science (for example, as measured through surveys)" would not have said much for Einstein in his day, clearly demonstrating the decreasing worth of dictionaries in determining definitions. Evidently, the modern authority is the survey.

Kevin Schutte said...

I hope understanding the significance of Gentzen's work isn't as hard as you imply it is. Because to me it seems really obviously very significant. Maybe you mean understanding Gentzen's work itself is hard. It probably is; I'm a little scared to try.

enzo rossi said...

David Mitchell in today's Observer:

...not every new government policy contains a glaring logical inconsistency.

Luckily for me, this one does. The article under the terrifying headline was about the proposed new system for allocating government money for academic research, the Research Excellence Framework. It wants to weed out pointless studies by favouring research that looks like it's going to be of economic or social use.

Hooray! That won't harm the comedy studies at all! When Professor Sponsored Link of the University of Twix announces that anti-wrinkle cream gives women the confidence to have cleverer children, he's not being funded by the government but by a cosmetics manufacturer trying to grab a headline.

All the "flowers/chocolate/ice cream bringing happiness/better orgasms/an enhanced sense of perspective" studies are entirely self-financing. They may add little to the sum of human knowledge; the fact that academics are reduced to them may show how eroded our respect for learning has become, but they're not a drain on the taxpayer – they all get paid for out of various multinationals' marketing budgets.

So what sort of pointless study is this new system going to weed out? Why, all the ones that don't have a solid social or economic goal, of course. The government isn't going to pay for clever people just to sit in universities indulging their curiosity. No, they should be allocated something useful to discover and then research as hard as they can in that direction. Nothing good ever got invented by accident, apart from some silly fun stuff like the slinky, post-it notes, penicillin, warfarin and X-rays. ...

Frank said...

Tenured professors should not whine.

Besides, if you want someone else's money through REF you have options. Think only of Dodgson/Carroll. He's got a ton of fun logic stuff that can be made accessible to the public. The white knight and all that. If you're clever, you could make a go with Cantor too. Think of this theological ideas, for example. So Dr. Logic, which would you rather do, whine about the philistines or, you know, do something?

Vince said...

I agree with Frank. British academics take the prize for whingeing. Here's an idea. Instead of whining -- politicians and quangocrats are philistines, quelle surprise -- why not do something about it? Charge students the amount an equivalent American university would charge (setting up the equivalent scholarships etc.) and, bingo, you're free from government interference. You don't need a PhD in Philosophy to figure that one out. You get the paymasters you deserve.

Brian said...

With all due respect to Frank and Vince, I don't think you're whin(g)ing. As I interpret it, your lament is as much (if not more so) qua citizen rather than qua academic.

I have to say, I certainly don't see the shift from government funding to student-gouging to be anything like a solution to the problem. It just shifts the kinds of economic demands placed on the university by its paymasters.

State funding is preferable precisely because it doesn't leave decisions to the Invisible Hand, except when the State decides that it should, as it has in this case. Hence the lamentation.

Vince said...

Brian, depending on students rather than the state creates a different set of demands. Plainly, these demands are destroying private American universities. Take Harvard (or Stanford or Yale or MIT or Princeton), the American analogue of Cambridge. Because Harvard charges $40k tuition per year the great majority of its undergrads are rich thickos, incomparably dumber than their Cambridge counterparts. Because Harvard has to raise its own postgraduate bursaries it sadly loses out to its state-sponsored European rivals, who whisk off some of the best international students year after year. Because Harvard professors are forced to go begging to hard-up funding bodies such as the NSF or internal university committees that have no conception of the point of research compared to the mages of HEFCE, their reseach priorities are distorted. The resulting money is arbitrarily distributed, according to the latest educational or political fad. Difficult, difficult indeed, are the conditions at Harvard. Wrecked by capitalism, professors and students alike dream of fleeing to the benevolent statism enveloping Cambridge, the brain drain only staunched by the beauty of Harvard Yard and the occasional glittering supper. The Invisible Hand has a lot to answer for.

Ed said...


Concerning your characterization of Harvard specifically, you might consult this.

Families with annual income <$60K pay *no* tuition. Even a family with an income of $180K (no immodest sum there) ends up owing "only" $18K. It is not as straightforward as everyone owing $40K.

In any case, many students (that aren't the rich "thickos" you describe) can attend Harvard for much less than the $40K you mention. Many get a price tag that is not nearly that dramatically different from Cambridge's current tuition of £3,225 ($5,160). And those who need the most help even get a better deal.

To be clear, I'm not advocating the Harvard/etc. model over the Cambridge model. But your exact picture is a bit skewed.


vince said...

Ed, Thanks. I'm fully aware of the facts you cite. Harvard is in a position to be so generous now and to subsidise even relatively well-off students because it has charged market rates for many decades and continues to do so for wealthy students. Cambridge might not be able to start off quite so generous but that's where it should aim to end up.