Originally published at physicsfocus.

As a professional physicist – as I sometimes like to pretend I am – I would estimate that at least 70% of my working week is spent on words, not numbers. Many of the undergrads here at Nottingham don’t appear to be entirely comfortable with this when I point it out. Indeed, quite a few students have specifically told me that they didn’t do physics to write essays and that they will go out of their way, in terms of module choices and exam questions, to avoid having to work with words.

But not all of our students have such an adverse reaction to the more qualitative side of their subject.

I have been extremely impressed by very many of the blog posts and articles produced, as coursework, for a fourth-year module we introduced this year, “The Politics, Perception, and Philosophy of Physics”. The majority of the coursework pieces to date have been uploaded at the course blog, and the quality of writing is generally very high. And it’s not just me who thinks this: I was delighted when both Physics World and physicsfocus agreed to publish coursework articles submitted by students.

A key point about the students taking the course, however, is that they were forewarned, repeatedly, that the module was devoid of mathematics. I stressed, during an introduction to Year 4 modules at the start of the academic year, that they would be assessed on the basis of blog posts and articles they submitted. In this sense, they’re a self-selecting ‘sample’ and thus perhaps not entirely representative of the class as a whole.

On the other hand, all physics undergraduates at Nottingham, even those who take our Physics with Theoretical Physics course, are required to do experiments in Year 1 and to submit formal reports on their lab work. (All undergrads also, of course, submit project reports in later years.) The title of this blog post stems from my marking of a set of first-year lab reports a few weeks ago, where the same errors in writing cropped up time and time again. (It’s not the first time that this has happened in my 17-odd years of teaching at Nottingham…)

I’ve been meaning to put together a video which not only lays out what is expected from physics undergrads for their lab reports – which, to be fair, is often not quite as clear and well-defined as it could be – but also highlights those common failings that cause so much wear and tear on my red pen. I managed to finally get round to doing this, after literally years of procrastination, over the Christmas break and I’m including the video here. I’d very much welcome and value feedback from physicsfocus readers.

My concerns about the words-numbers divide are, however, much broader in scope than the niggles on structure, punctuation,[1] and grammar outlined in the video. Having taken on the role of undergraduate admissions tutor this year, I am now even more aware of the extent to which the A-level system exacerbates the arts-and-humanities-vs-STEM divide. I grew up in Ireland where our equivalent of the A-level system, the Leaving Certificate, makes both English and maths mandatory, and where a larger range of subjects (typically seven) is studied in the final two years of secondary school.

I was lucky to do not only all three science subjects and maths for my Leaving Certificate, but also French and English. And Irish. (Some might well say “Is fearr Gaeilge briste, ná Béarla clíste” but then they haven’t heard my spoken Irish. Or my English, for that matter.) There are, of course, other examples of education systems where there is a greater breadth of subjects than is typically the norm in the UK – Scottish Highers, International Baccalaureate. The A-level system, on the other hand, too often means that students end up making a stark choice between the STEM and arts/humanities pathways too early. This is a great shame because it serves to entrench the ‘two cultures’ divide that CP Snow criticised so forcefully almost 60 years ago.

Simon Jenkins, the Guardian’s resident STEM-skeptic, regularly bemoans the negative perception of the value of the arts and humanities as compared to, as he sees it, the unquestioned importance of STEM subjects to society. He was on fine form on New Year’s Day, arguing in an article, “Easy to sneer at arts graduates – but we’ll need their skills”, that “a humanistic education” produces better-rounded and more creative types who “seem better equipped to use their imagination and challenge conventional wisdom”. Last year Jenkins also provoked quite some ire by arguing that STEM graduates, particularly computer scientists, lack the ability to communicate effectively.

This may perhaps come as something of a surprise to readers of physicsfocus, but I have quite some sympathy with Jenkins’ concerns about the extent to which an arts and humanities degree has been ‘devalued’ in terms of its perceived value to society (and, by extension, to the individual graduate). I have always rather disliked articles and reports proclaiming that physics is so much more intellectually challenging – i.e. ‘harder’ – than other subjects. Yes, physics is conceptually challenging. And, yes, it’s intellectually stimulating and demanding. And yes, as I’ve discussed before for physicsfocus, it requires a heck of a lot of work and effort in order to ‘get it’. But, as Dave Farmer explains in a perceptive, important, and smart post, there are many types of intelligence, and there are many types of aptitude.

There are physicists at all career levels whose analytical maths abilities are truly remarkable. But ask some of them to write 500 words which are engaging and thought-provoking, and they’re flummoxed. Echoing the points made by Farmer, a capability with mathematics is just one type of intelligence. Attempting to quantify such a multi-faceted and complex human characteristic via an aptitude in one area, or, worse, via a single ‘IQ’ value, is as ludicrous as, errmm, reducing the value of a university to a position on a league table.

An ability to communicate effectively is essential, independent of subject, discipline, or career. University physics departments across the country have for years complained about the reduction in the rigour of A-level maths, and have introduced first-year ‘refresher’ modules in order to bring incoming students up to speed in mathematical techniques. But similar primers in written communication have not been introduced. Given the lack of subject breadth of the A-level system, and the associated absence of the development of writing skills for many STEM-focused students, one could make the argument that there is an equally pressing, if not greater, need for formal teaching of written communication skills in Year 1 of a physics degree.

Where my views diverge dramatically from those of Jenkins, however, is with his argument that arts and humanities graduates are necessarily more creative than those with degrees in STEM subjects. Science is intrinsically creative and Jenkins does his important arguments about the value of the arts and humanities a great disservice by playing down to lazy stereotypes of STEM graduates.

Equally importantly, an arts and humanities degree is no guarantee of an ability to communicate concepts in a clear, engaging, and effective style. I’ll leave you with Exhibit #1 – an excerpt from the work of Prof. Karen Barad, of the Philosophy Department at the University of California Santa Cruz. (I suspect that I’ll be returning to a discussion of Barad’s work for a future physicsfocus post).

“Multiply heterogeneous iterations all: past, present, and future, not in a relation of linear unfolding, but threaded through one another in a nonlinear enfolding of spacetimemattering, a topology that defies any suggestion of a smooth continuous manifold. Time is out of joint. Dispersed. Diffracted. Time is diffracted through itself. It is not only the nature of time in its disjointedness that is at stake, but also disjointedness itself. Indeed, the nature of ‘dis’ and ‘jointedness’, of discontinuity and continuity, of difference and entanglement, and their im/possible interrelation ships are at issue.”

Thanks to my colleague at Nottingham, Brigitte Nerlich, for bringing my attention to that quite remarkable piece of impenetrable writing, via this blog post.

_ _ _

[1] I’m a fan of the Oxford comma.

Image Credit: https://www.flickr.com/photos/darinrmcclure/6198246544 

An experiment in post-proposal peer review


Originally published at physicsfocus.

I’m a huge fan of post-publication peer review (PPPR). It’s the future of scientific publishing and it’ll be de rigeur – rather than a novelty – for the next generation of scientists. Because if that doesn’t happen, science and society are going to continue to suffer from gaping holes in the quality-control mechanism that is traditional peer review.

I’m about to describe an experiment which takes the online/public peer review process back a couple of steps from the point of publication. But before I do that, it might help if I explain just why I’m such an enthusiastic advocate of PPPR.

Over the past couple of years, and along with colleagues at Nottingham, NIST, and Liverpool, I’ve been embroiled in a rather heated debate about the validity of a substantial body of research focused on the structure of coated (aka ‘stripy’) nanoparticles. I blogged about this for physicsfocus around about this time last year, and was delighted when our paper critiquing the nanoparticle research in question was finally published in PLOS ONE a couple of months ago.

Long before the paper appeared in PLOS ONE, however, we had made it available (via the arXiv) at the PubPeer PPPR site, for what is perhaps best described as pre-publication peer review. This led to a large volume of very helpful comments (and, it must be admitted, the occasional less-than-helpful post) from our peers. The PubPeer contributions of one of those peers, Brian Pauw, were so insightful and important that he ended up being added as a co-author to the paper.

In addition to highlighting the benefits of open and public next-generation peer review, the striped nanoparticle controversy made me intensely aware of a number of shocking deficiencies in the traditional peer review system. First is the demonstrated inability of traditional peer review to always filter out junk. I don’t want to harp on about the deficiencies in the striped nanoparticle work (which is faulty, rather than fraudulent) so let’s turn to a truly shocking example of the failure of traditional peer review: the nano chopsticks farce, as Brady Haran and I discuss in this Sixty Symbols video:

Social media, in particular the Chemistry Blog and ChemBark sites (and their associated Twitter feeds), exposed the chopstick ‘breakthrough’ as a staggeringly poor Photoshop job within days of the paper being published. It was retracted just two months after its publication.

A decade before this chopsticks debacle, the nanoscience community endured the rather less cack-handed, arguably quite clever, and remarkably systematic fraud of Hendrik Schön. I firmly believe that if post-publication peer review had existed in the early 2000s that Schön’s fraud would have been identified much, much sooner than it was. (Note how quickly the PubPeer community identified problems in the then-acclaimed, but now-retracted, STAP results published at the start of last year.)

PPPR isn’t, however, all about laying bare fraudulent work. At its best it’s exactly how the scientific method should work: authors should be willing to have their work discussed, debated, and dissected by their peers both before and after – particularly after – its publication. Compare and contrast with the following response from a well-respected, influential, and – for those who care about simplistic and flawed metrics – very high impact-factor journal, after I asked whether they’d be interested in publishing our critique (which eventually became the PLOS ONE paper described above):


Or, in other words, our journal is not interested in following the scientific method.

From PPPR to PPrPR

The deficiencies in peer review of course extend to the assessment of grant proposals. As I was writing this post, a link to an article published in Nature a couple of days ago appeared in my Twitter timeline (thanks @NKrasnogor), highlighting that the ratings of Medical Research Council proposals from external referees do not correlate well with the probability of the grant application being funded. This, of course, will not come as a great surprise to many researchers.

Some time ago I suggested to the Engineering and Physical Sciences Research Council (EPSRC) that they carry out an experiment where they send the same set of proposals to entirely independent prioritisation panels (and referees), and subsequently check for correlations between the rankings of the various panels. This is particularly important given that EPSRC blacklists researchers on the basis of where their grant proposal falls on the ranked list returned by the prioritisation panel.

EPSRC hasn’t run this experiment.

I’m trying a rather different peer review experiment of my own. Late last year I discussed the possibility of open peer review of a grant proposal, rather than a publication, with PubPeer and, subsequently, The Winnower. While PubPeer facilitates open review of any publication with a DOI, The Winnower, founded by Joshua Nicholson, combines open access publication with PPPR. The Winnower kindly agreed to publish our EPSRC proposal, Mechanochemistry At The Single Bond Limit, which, for the reasons discussed in this article in Physics World, is my first for EPSRC in quite some time. With the DOI provided by The Winnower, we subsequently set up a PubPeer thread related to the proposal.

As the ‘Pathways to Impact‘ section of the proposal lays out, the entire impact case is based on public engagement (rather than, for example, commercial exploitation). A key component of that public engagement programme, should the grant application be successful, is that my colleague Brigitte Nerlich will be an ‘embedded’ sociologist within the research team. Brigitte will observe, and blog/tweet about, just how the scientific method plays out in the course of the project. It therefore makes a great deal of sense to extend the public engagement aspects of the proposed research to the grant application process itself, i.e. to incorporate post-proposal peer review (PPrPR).

Coincidentally, and fortuitously, a week or so after the discussions with PubPeer and The Winnower, Dorothy Bishop tweeted a link to an important and very relevant paper by Daniel Mietchen in PLOS Biology (not one of the journals I usually read).

The closing sentence of the abstract to this far-sighted paper is worth quoting at length:

“The article … explores the option of opening to the public key components of the [grant application review] process, makes the case for pilot projects in this area, and sketches out the potential that such measures might have to transform the research landscape in those areas in which they are implemented.”

The motivation for making our EPSRC proposal available for comment and criticism via The Winnower and PubPeer is exactly as that abstract describes – it’s a question of opening up the grant application/review process to public scrutiny. My aim over the coming months is – EPSRC and reviewers permitting – to make available, here at physicsfocus, the referees’ reports and, ultimately, the outcome of the panel ranking process.

It’s an experiment that may return a null result, of course, in that there could well be a deafening silence in response to making the proposal (and, hopefully, the subsequent reviews) publicly available. After all, I don’t believe that there are too many academics fretting about finding more reviewing to do. But then, a null result is still very often an important finding that can provide key insights.

Let’s just run the experiment and see…

Image credit: https://commons.wikimedia.org/wiki/File:In_Peer_Review_We_Trust