Thursday, September 24, 2009

Research Excellence Bullshit

So, there's another consultation document on the Research Excellence Framework -- "the new arrangements for the assessment and funding of research in UK higher education institutions that will replace the Research Assessment Exercise (RAE)". A wonderful document indeed, literate and elegantly written, revealing much thought and reflection on the nature of the university in the best traditions of Arnold and Leavis. Of course. Still, perhaps it isn't quite what we might hope for.

Ok, ok, I jest. It isn't at all what we might hope for, though it is the sort of egregious crap we've come to expect. How about this, for example: "As an indication of our current thinking we propose the following weightings" (between different components of assessment); "Outputs: 60 per cent. Impact: 25 per cent. Environment: 15 per cent." Hold on! Impact? Impact? What's that?

Well, the document gives "a common menu of impact indicators" under various headings to help us out. Here are the headings ...

  • Delivering highly skilled people [as evidenced e.g. by "Staff movement between academia and industry, Employment of post-doctoral researchers in industry or spin-out companies".]
  • Creating new businesses, improving the performance of existing businesses, or commercialising new products or processes
  • Attracting R&D investment from global business
  • Better informed public policy-making or improved public services
  • Improved patient care or health outcomes
  • Progress towards sustainable development, including environmental sustainability
  • Cultural enrichment, including improved public engagement with science and research
  • Improved social welfare, social cohesion or national security
  • Other quality of life benefits
Right. Let me see if I understand. If you are a medieval historian, an editor of Euripides, a Shakespeare scholar, or indeed just a logician trying to understand the philosophical significance of Gentzen's work on the consistency of arithmetic, then 25% of your score in son-of-RAE is going to be for "impacts" utterly irrelevant to your projects and concerns?

I'm being unfair, you say: arts subjects at least get into the frame under the heading "Cultural enrichment". You might think so: but in fact we are told that possible indicators of that are -- I kid you not -- "Increased levels of public engagement with science and research (for example, as measured through surveys). Changes to public attitudes to science (for example, as measured through surveys). Enriched appreciation of heritage or culture (for example, as measured through surveys). Audience/participation levels at public dissemination or engagement activities (exhibitions, broadcasts and so on). Positive reviews or participant feedback on public dissemination or engagement activities." Yep, and we are also told that impact does not include "we do not intend to include impact through intellectual influence on scientific knowledge and academia".

Ah, there's a chink of light perhaps: not everyone is to be ranked for impact, if I've got it right? -- a department's return will rather involve a series of "case-studies" of impactful individuals. Well, yes, you can just see the guys and gals in M&E sitting around trying to find a smidgin of impact somewhere between them.

Brilliant. Well, I know will happen; you know what will happen; HEFCE no doubt know what will happen when traditional humanities departments come to fill in the impact case studies on which 25% of their overall rating is going to depend.

They'll have to bullshit.

Added later. My jest about the M&E contingent having a bit of difficulty cooking up an impact statement was truer than I realized. Eric Schliesser, currently at Leiden, writes in a comment on the Leiter blog that "in places where 'impact' is already playing a prominent role (say, in Netherlands and Flanders), certain subjects (e.g., analytic metaphysics,) have very little chance to receive coveted research grants (now almost the sole source for PhD funding). Yesterday, Michael della Rocca gave a terrific talk on the three-dimensionalism vs four-dimensionalism debate. It generated great discussion. But the people in attendance were hard-pressed to name a sole Dutch philosopher who is working on the topic ... Of course, other subjects (e.g., philosophy of technology, applied ethics, decision theory, semantics, logic, normative ethics, etc) have an easier time in articulating the impact factor and are generously funded."

Wednesday, September 23, 2009

Schubert's Piano Trios

I buy few new CDs these days, as I already have ridiculously many (and multiple recordings of most pieces that I really care about it). But I was driving home from my aged mother's the other day, and the BBC were playing the Schiff/Shiokawa/Perényi recording of the E flat trio D 897, and I was bowled over. The double CD with the other trio, the D897 Notturno and the Arpeggione Sonata came out in 1997, Schubert's bicentennial year, but -- though I've always admired Schiff's Schubert playing -- I'd missed this record. But, still in stock at Amazon, it arrived a couple of days ago.

And it indeed is terrific. The performances could hardly be bettered it seems to me -- there's a flow to the playing and a rapport between the three that gives new life to the pieces after years of listening to the Beaux Arts' recordings. The Gramophone review agrees. (I can't imagine though why, after a decade, this hasn't been reissued in a cheaper version: it deserves to be on everyone's shelves.)

Friday, September 18, 2009

Praise for Just and Weese!

From time to time I do get more than a bit critical here about books of one sort or another: so it's good to give praise for once!

Over the last couple of days I've been reading the first volume of Winfried Just and Martin Weese's Discovering Modern Set Theory (AMS, 1996), with an eye to moving on to the second volume. Well, I just loved the style, and think it works very well. I don't mean the occasional (sightly laboured?) jokes: I mean the in-the-classroom feel of the way that proofs are explored and motivated, and also the way that teach-yourself exercises are integrated into the text. For instance there are exercises that encourage you to produce proofs that are in fact non-fully justified, and then the discussion explores what goes wrong and how to plug the gaps. My grip on set theoretic niceties is patchy enough to be find this kind of reinforcement of understanding pretty helpful from time to time, even at the elementary level of the first volume. So I'll be rather warmly recommending the book to students.

Saturday, September 12, 2009

Student evaluations

I remember, quite a few years ago, giving the same introductory logic course two years running, as far as I could tell doing as a good a job each time. But my student evaluations plummeted between one year and the next. Why? I could only put it down to the fact that the first year I gave the course in relaxed casual dress; the next year (because a committee was scheduled the same afternoons) I wore a rather serious suit. So I supposedly came across as remote, unhelpful, and harder to understand.

I was reminded of that experience -- which made me permanently a tad sceptical about the worth of student evaluations -- when I read these two scepticism-reinforcing pieces*, by the philosophers Michael Huemer and Clark Glymour. I was particularly amused (in a world-weary sort of way) by this excerpt from the former:

[There was a] study, in which students were asked to rate instructors on a number of personality traits (e.g., "confident," "dominant," "optimistic," etc.), on the basis of 30-second video clips, without audio, of the instructors lecturing. These ratings were found to be very good predictors of end-of-semester evaluations given by the instructors' actual students. A composite of the personality trait ratings correlated .76 with end-of-term course evaluations; ratings of instructors' "optimism" showed an impressive .84 correlation with end-of-term course evaluations. Thus, in order to predict with fair accuracy the ratings an instructor would get, it was not necessary to know anything of what the instructor said in class, the material the course covered, the readings, the assignments, the tests, etc.

Williams and Ceci conducted a related experiment. Professor Ceci, a veteran teacher of the Developmental Psychology course at Cornell, gave the course consecutively in both fall and spring semesters one year. In between the two semesters, he visited a media consultant for lessons on improving presentation style. Specifically, Professor Ceci was trained to modulate his tone of voice more and to use more hand gestures while speaking. He then proceeded, in the spring semester, to give almost the identical course (verified by checking recordings of his lectures from the fall), with the sole significant difference being the addition of hand gestures and variations in tone of voice (grading policy, textbook, office hours, tests, and even the basic demographic profile of the class remained the same). The result: student ratings for the spring semester were far higher, usually by more than one standard deviation, on all aspects of the course and the instructor. Even the textbook was rated higher by almost a full point on a scale from 1 to 5. Students in the spring semester believed they had learned far more (this rating increased from 2.93 to 4.05), even though, according to Ceci, they had not in fact learned any more, as measured by their test scores. Again, the conclusion seems to be that student ratings are heavily influenced by cosmetic factors that have no effect on student learning.
So now you know: bounce in optimistically, wave your hands around confidently, and you can sell the kids anything ...

And I should say that these days I always wear a suit to lecture (so I've a cast-iron excuse for any poor evaluations, of course).

Added For a bit of judicious balance, do read Richard Zach's second contribution (Comment 12 below), and the linked paper.

*Links from twitter, thanks to John Basl and Allen Stairs

Tuesday, September 08, 2009

Math logic reading list (updated)

I've spent the last couple of days reorganizing and rewriting the reading list for the Part II Math Logic paper (that's a third year undergraduate paper for philosophers). It was a rather minimalist affair, and I've taken a step or two towards its becoming an annotated study guide.

The paper is something of a Cambridge institution, pretty much unchanged in its basic syllabus since when I took it a long time ago. It rather distinctively mixes an introduction to the "greatest hits" as far as formal results are concerned, with a look at some of the philosophical issues arising.

Anyway, having had some initial comments here and from local grad students, you can now download my third shot at an updated list. All comments and suggestions for further improvement (within the current, fixed, syllabus) will still be very welcome.

Congratulations to Thomas Forster

Another logic-seminar regular hits the big time! It is good to see that Thomas's “The Iterative Conception of Set”, published last year in the new Review of Symbolic Logic was judged one of the ten best papers of 2008 by the Philosopher's Annual. Here's a link.

Grumpy old man, #42

I think I'm turning into a grumpy old man ...

[Cue suppressed laughter off stage, murmurings of "Turning? Turning? Happened years ago", etc. But I shall ignore these scurrilous interruptions.]

... and the latest cross-making irritation (especially galling for a long-time Analysis editor) is the effort by some OUP copy-editor to improve a forthcoming Analysis paper by Luca Incurvati and myself, by inter alia, removing all the contractions, replacing "don't"s by "do not"s etc.

Now, it is one thing to replace American spelling by English spelling (or vice versa), or to replace "...ize" by "...ise", for example. But to replace "don't" (one long syllable) by "do not" (two short staccato syllables) is to change the rhythm of a sentence. The use of "don't" can smooth the reading of a sentence, slightly modulating the emphasis. Has the OUP editor being paying attention to such matters? Somehow I think not. My bet is that the changes have been made without thinking, slavishly following some semi-literate "style book".

And to make such changes wholesale is to arbitrarily change the authorial tone of voice: which is just impolite (to put it mildly -- especially when some of us put quite a bit of effort into getting the tone we want).


Wednesday, September 02, 2009

School maths, from the distant past

I found myself yesterday in a small-town bookshop, kicking my heels for half an hour. Prompted by recent press discussion of the standard of A-levels (the UK 18+ end-of-high-school examination), I browsed through some books intended for A-level further maths students. I must say that they did seem really rather noddy to me, though of course it is only too easy to be seduced into the thought that things are going to the dogs!

Still, that prompted me, just for fun, to look out the papers I sat aged seventeen and a bit, to get into Cambridge, back when the world was young. So here's a small selection of some of the shorter questions:1 click to enlarge. (There were four three-hour papers with ten questions apiece: as I recall you aimed to get out at least half-a-dozen a time).

The questions do seem tougher than anything I saw in the contemporary text for further maths. But it would be interesting to know from anyone with their finger more on the pulse how many reasonably bright school kids are in a position to tackle this sort of thing these days. Or indeed -- though the answer could be depressing -- how many of their teachers.

1 I don't guarantee my proof reading in copying the questions!