We are anonymous. We are legion. We are (mostly) harmful.

This revelation appeared in my Twitter timeline earlier this week:

On the same day, Nature News published a fascinating interview with Brandon Stell, the founder of PubPeer who has revealed his identity to the world:

I’ve waxed lyrical about PubPeer a number of times before, going so far as to say that its post-publication peer review (PPPR) ethos has to be the future of scientific publishing. (I now also try to include mention of PubPeer in every conference presentation/seminar I give). I’ll be gutted if PPPR of the type pioneered by PubPeer does not become de rigueur for the next generation of scientists; our conventional peer review system is, from so many perspectives, archaic and outdated. I agree entirely with Stell’s comments in that interview for Nature News:

Post-publication peer review has the potential to completely change the way that science is conducted. I think PubPeer could help us to move towards an ideal scenario where we can immediately disseminate our findings and set up a different way of evaluating significant research than the current system.

But one major bone of contention that I’ve always had with PubPeer’s approach, and that has been the subject of a couple of amicable ‘tweet-spats’ with the @PubPeer Twitter feed, is the issue of anonymity. I was disappointed that not only was it just Stell who revealed his identity — his two PubPeer co-founders remain anonymous — but that there are plans (or at least aspirations) to “shore up”, as Stell puts it, the anonymity of PubPeer commenters.

I am not a fan of internet anonymity. At all. I understand entirely the arguments regularly made by PubPeer (and many others) in favour of anonymous commenting. In particular, I am intensely aware of the major power imbalance that exists between, for example, a 1st year PhD student commenting on a paper at PubPeer and the world-leading, award-winning, scientifically decorated and oh-so-prestigious scientist whose group carried out the work that is being critiqued/attacked. Similarly, and in common with Peter Coles, I personally know bloggers who write important, challenging, and influential posts while remaining anonymous.

I also fully realise that there are are extreme cases when it might not only be career-theatening, but life-threatening for a blogger to reveal their identity. However, those are exactly that: extreme cases. It’s statistically rather improbable that all of those pseudonymously venting their spleen under articles at, say, The Guardian, Telegraph, or, forgive me, Daily Mail website are writing in fear of their life. (Although, and as this wonderful Twitter account highlights so well, many Daily Mail readers certainly feel as if their entire culture, identity, and belief system are under constant attack from the ranked hordes of migrants/PC lefties/benefit claimants/gypsies/BBC executives [delete as appropriate] swamping the country).

I am firmly of the opinion that the advantages of anonymity are far outweighed by the difficulties associated with fostering an online culture where comments and critique — and, at worst, vicious abuse — are posted under cover of a pseudonym. For one thing, there’s the strong possibility of sockpuppets being exploited to distort debate. Julian Stirling, an alumnus of the Nanoscience Group here at Nottingham and now a research fellow at NIST, has described the irritations and frustrations of the sockpuppetry we’ve experienced as part of our critique of a series of papers spanning a decade’s worth of research. In the later stages of this tussle, the line separating sockpuppetry from outright identity theft was crossed. You might suggest, like PubPeer, that this type of behaviour is not the norm. Perhaps. But our experience shows just how bad it can get.

Even in the absence of sockpuppetry, I’ve got to come clean and admit that I’m really not entirely comfortable with communicating with someone online who is not willing to reveal their identity. I’ve been trying to get to grips with just what it is about anonymous/ pseudonymous comments that rankles with me so much, having been involved in quite a number of online ‘debates’ where the vast majority of those commenting have used pseudonyms. When challenged on the use of a pseudonym, and asked for some information about their background, the response is generally aggressively defensive. Their standard rebuttal is to ask why I should care about who they are because isn’t it the strength of the argument, not the identity of the person making the comments, that really matters?

In principle, yes. But the online world is often not very principled.

Ultimately, I think that my deep irritation with pseudonyms stems from two key factors. The first of these is the fundamental lack of fairness due to the ‘asymmetry’ in communication. Dorothy Bishop wrote a characteristically considered, thoughtful, and thought-provoking piece on this issue of communication asymmetry a number of years back (in the context of the debate regarding the burqa ban). The asymmetry that’s established via anonymous commenting means that those who are critiquing an author’s (or a group’s) work are free to comment with impunity; they can say whatever they like in the clear knowledge that there’s a negligible chance of their comments ever being traced back to them.

Stell and his PubPeer co-founders claim that this is actually a key advantage of anonymity — those who comment are not constrained by concerns that they’ll be identified. But if their arguments are sound, and expressed in a polite, if critical, manner, then why the heck should they be concerned?  After all, it’s long been the case that “old school” journals — the APS’ Physical Review family of titles being a notable example — publish critiques of papers that have previously appeared in their pages. Those formal critiques are published with the names of the authors listed for all the world, or at least the readership of the journal, to see.

We should aim to change the culture so that critiquing other scientists’ work is seen as part-and-parcel of the scientific process, i.e. something for which researchers, at any career level, should be proud to take credit. Instead, the ease of commenting online from behind cover of a pseudonym or avatar is encouraging a secretive, and, let’s be honest, a rather grubby, approach to scientific criticism. I was therefore particularly encouraged by this announcement from The Winnower yesterday. It’s a fantastic idea to publish reviews of papers from journal club discussions and it’ll help to move the critique of published science to a rather more open, and thus much healthier, place.

The second, although closely related, aspect of anonymity that winds me up is that it essentially (further) depersonalises online communication. This helps to normalise a culture in which those commenting don’t ever take responsibility for what they say, or, in the worst cases — if we consider online communication in a broader context than just the scientific community — fail to appreciate just how hurtful their abuse might be. I’ve often seen comments along the lines of “It’s all just pixels on a screen. They should toughen up”. These comments are invariably made by those hiding behind a pseudonym.

As ever, xkcd has the perfect riposte…

This splendid poem also makes the point rather well. “Cuts and bruises now have healed, it’s words that I remember.

Anonymity contributes to a basic lack of online respect and too often can represent a lack of intellectual courage. When we criticise, critique, lambaste, or vilify others online let’s have the courage of our convictions and put our name to our comments.

Author: Philip Moriarty

Physicist. Rush fan. Father of three. (Not Rush fans. Yet.) Rants not restricted to the key of E minor...

54 thoughts on “We are anonymous. We are legion. We are (mostly) harmful.”

  1. Interesting post! I’m all for openness – in fact I sign all my manuscript reviews. However, I don’t think that anonymous commenters are mostly harmful. In my experience the negative anonymous comments are mostly subjective and, while hurtful, can be ultimately ignored. While for the anonymous objective comments (e.g. pointing out obvious image manipulation) anonymity is much less of an issue. It’s either right or it’s not.

    I agree that you caught a very bad one (sockpuppetry) on PubPeer, but that really appear to be the exception, rather than the rule, and I would argue (subjectively) that PubPeer would have a fraction of the impact it has had if it had banned anonymous commenting when it started.

    You do mention pseudonyms, which I find an interesting compromise. There are several instances where people have invested so much in their pseudonym (e.g. @drugmonkey) that they no-longer can espouse any old non-sense just because they are anonymous.

    Like

    1. Thanks for the comments, Jan, and sorry for not replying sooner.

      The problem with anonymous commenting is that it is going to make the wider/”universal” acceptance of PPPR as a valid form of scientific debate/criticism/critique extremely difficult. If the argument is that “Without anonymity, we won’t have comments posted” then that highlights that we have a major problem with 21st century science. (Another one.)

      During their debates on the fundations of quantum theory, Bohr, Einstein, Dirac, Heisenberg, Schroedinger et al. didn’t send each other anonymous notes. Debate and critique are absolutely core to the scientific process.

      I find it depressing and dispiriting to think that the only way that PPPR will work is if those who comment are protected by anonymity.

      Like

      1. In the time of Bohr, Einstein, Dirac, Heisenberg, Schroedinger et al. peer review of research proposals did not exist yet, in fact most of the funding was at the professors discretion rather than nowadays in the hands of anonymous peer reviewers. In that time, peer review of article was still rate where it would be possible to punish someone for being critical. These people had tenure track positions. In my institute, only the 3 professors have this. The other 40 to 50 scientists work on temporary contracts.

        So yes, if you do not like anonymity, we have to change the culture and get rid of the metric micro-management fake-competitive project-based science system the politicians and professors have created.

        P.S. I am not against anonymity. I would see it as the online equivalent of the freedom to organize, which is there to make it possible to engage without being held personally accountable, to stand up to power, and to be able to say stupid stuff in private before coming up with a message of the group, which should best not be stupid. Within science arguments should count, however, not power, thus if the system were well designed the scientific debate should not need anonymity.

        Like

  2. “When we criticise, critique, lambaste, or vilify others online let’s have the courage of our convictions and put our name to our comments.”

    Agree with your last sentence above, but what hope does one have when the grants, papers, fellowship applications, etc… are done by anonymous reviewers?

    In my opinion, anonymity is the only reason why PubPeer works and PubCommons and Journal commenting sites haven’t to the same extent.

    Science communities are very small and the last thing you need is a spat with big heavyweights in the field. When anonymous peer review is the norm it is not a level playing field. Therefore, anonymous post publication review is justified.

    Like

    1. Thanks for your comment.

      There’s a major difference between anonymous peer review and online PPPR a la PubPeer — the former is moderated by editors. I have long argued that a good compromise for PubPeer, for example, is to just have comments from those who are willing to register at the site as Peer 1, Peer 2,…Peer n. That means that their identity is known to the PubPeer “moderators”.

      PubPeer’s counter-argument is that this would dramatically decrease the number of comments. They’re probably right. As I said in response to Jan above, I find that immensely dispiriting. If we are willing to criticise someone’s work in public (another difference with conventional peer review) then we should be willing to put our name to our critique.

      As regards avoiding a “spat” with big heavyweights in the field, does this mean that in the absence of anonymous PPPR, their work simply doesn’t get challenged? That you’d also avoid publishing a paper/comment critical of their work? This is a very worrying stance and we have to get beyond this timid mindset.

      Like

  3. Philip – your piece lacks evidence, most notably in its very title. Where is your evidence that most comments (at PubPeer since this is the real focus of the piece) are harmful.

    We have plenty of examples of interesting and insightful anonymous comments. Whatever their reasons [and I really do mean this “whatever”], it is reasonable to hypothesize that in the absence of anonymity we would not have benefited from their thoughts.

    Moderation (as in the case of proposals or journal peer review) does not considerably change the game. After all, this moderation is in reality very light touch or ineffective and certainly does not guarantee the absence of conflicts of interest.

    I have therefore a lot of sympathy with the anon commenters who are asking those who would like them to unmask to stick to the content rather than looking for the person.

    Right now, the urgent priority is to bring the critique out of the closet, to share issues with problematic papers so that public money and PhD student enthusiasm are not wasted on problems which have already been identified by others.

    PubPeer provides a tool where people can choose to be anonymous or not. Its priority is to attract a lot of quality comments and it is working. It is a success due to the quality of the interface and to the option of anonymity.

    I know we – and many others including most probably the good people, anon or not, at PubPeer – share the same long term objective: a scientific culture where open debate and critique is a normal (and rewarded) part of the scholarly activity. This is not the world in which we currently live.
    Once the value of PPPR is obvious to all, including governments and funding agencies, there is a chance that we will get closer to that objective.

    Like

    1. Thanks for your comments, Raphael. I suspected we might not agree on this particular aspect of PPPR!

      I know we – and many others including most probably the good people, anon or not, at PubPeer – share the same long term objective: a scientific culture where open debate and critique is a normal (and rewarded) part of the scholarly activity. This is not the world in which we currently live.

      And my point is that anonymous commenting is not going to accelerate our progress towards this goal; it’ll impede it. This is what I meant by anon commenting being harmful — it embeds a culture where the expectation is that we critique other research under cover of anonymity. This is not a healthy way for science to evolve. “Harmful” in the sense of distorting critique of others’ work; not harmful in the sense of the comments being unreliable.

      We already hear regularly from PubPeer and other PPPR proponents that without anon commenting, the volume of critique would drop dramatically. As I state in the blog post, this is really a rather worrying indictment of our peer review culture. To echo my response to Jan Jensen above, we need to get back to the same type of open and robust debate that Bohr, Einstein, Dirac et al. engaged in as part of the scientific process.

      PubPeer provides a tool where people can choose to be anonymous or not. Its priority is to attract a lot of quality comments and it is working. It is a success due to the quality of the interface and to the option of anonymity.

      I agree entirely that PubPeer is a major success.

      I also agree that anonymity has played a large role in that success.

      I find that rather dispiriting.

      Like

      1. We largely agree except on this: “… my point is that anonymous commenting is not going to accelerate our progress towards this goal; it’ll impede it..”

        You are dispirited by the fact that anonymity played a large role in PubPeer success, but you do not offer a direction as to what could be the drivers that would lead us to a change of culture.

        I make the optimistic and testable prediction that you will be proved wrong, i.e. that the volume of pppr will augment AND the proportion of pppr that is signed will also increase with time.

        Like

        1. @Raphael (September 9th 2015 at 2:31 pm)

          I make the optimistic and testable prediction that you will be proved wrong, i.e. that the volume of pppr will augment AND the proportion of pppr that is signed will also increase with time.

          …but what is going to change the culture away from anonymous reviewing if the argument is that it’s an essential component of PPPR?

          you do not offer a direction as to what could be the drivers that would lead us to a change of culture

          I don’t explictly state what that driver is but it’s very implicit throughout the blog post. We should put our name to PPPR reviews/comments and encourage others to do so. That was, after all, the central point of the post!

          Like

      2. “This is what I meant by anon commenting being harmful — it embeds a culture where the expectation is that we critique other research under cover of anonymity.”

        But this is how most journals do pre-publication peer review today and most of us think that this is a good system. Are you proposing that we do away with anon. pre-pub review, too? I have seen the attempt by Frontiers to use named reviewers as some sort of review quality control measure, and I don’t think it passes muster. Instead, I saw a couple of examples of people I knew missing something that I find obviously wrong and thus the only “benefit” was to think that the editors selected the wrong reviewers. Now, I would like to see the reviews themselves presented along with a paper, but anonymously.

        Like

  4. Hi Phil,

    Thanks for your support for PubPeer (disclaimer – I’m speaking personally).

    Regarding anonymity, there is one important situation where I think your argument against it falls down, and that is when authors cannot be relied upon to act in good faith. In particular, if somebody has committed misconduct, the stakes will be really, really high and there is no reason at all to expect them to play fairly and resolve issues through open discussion.

    In practice, there are also many powerful researchers who would take an extremely dim view of any serious criticism, even without underlying misconduct. They will often be tempted to take the easy route of applying pressure at the source of criticism rather than addressing the criticism, especially if they have no answer to the criticism. I can understand that most people wouldn’t think that voicing a public criticism is worth dealing with a legal threat, and that’s just one of the risks they run.

    Maybe somebody should just run a survey or poll asking scientists whether they would like the option to comment anonymously.

    I don’t share your pessimism about anonymous commenting eventually leading to more open discussion. Comments and discussion generally will encourage good scientific practice, as will the trend towards data sharing. Once that feeds through, fewer people will be stuck in the situation of having cut corners in a way that they cannot admit, and much of the aggression will dissipate.

    Like

    1. Thanks, Boris (and absolutely no problem with using “Phil” 🙂 ).

      Regarding anonymity, there is one important situation where I think your argument against it falls down, and that is when authors cannot be relied upon to act in good faith. In particular, if somebody has committed misconduct, the stakes will be really, really high and there is no reason at all to expect them to play fairly and resolve issues through open discussion.

      You raise an extremely important point. I fully agree that it may often be the case that authors do not act in good faith (and we’ve certainly experienced this). But that’s still not a powerful argument for anonymous commenting. Some might say that the researcher who comments under a pseudonym/anonymously isn’t acting in “good faith”. If there are serious issues with a paper we should be willing to put our head above the parapet and say so. We have to start fostering a culture that enables this to happen. It’s just a little bit more “above board” that way.

      In practice, there are also many powerful researchers who would take an extremely dim view of any serious criticism, even without underlying misconduct. They will often be tempted to take the easy route of applying pressure at the source of criticism rather than addressing the criticism, especially if they have no answer to the criticism.

      I agree. And this is why in the blog post I raised the example of the 1st year PhD student criticising work by a major group leader. However, if that criticism is valid then it’s up to that student’s research supervisor (or other senior colleagues in the group/department) to support them. By enabling a culture of anonymity we’re effectively admitting defeat — we’re saying that those with power can’t be challenged openly. That our only recourse is to hide behind cover of anonymity. And that’s a very dangerous position to adopt.

      Of course, there is also the case where a PhD student acts as a whistleblower, highlighting either fraudulent or sloppy research practice. But even in that case, I remain of the opinion that it’s up to us, as a community, to ensure that a student/early career researcher will be sufficiently supported so as to make those criticisms openly. Here’s a good example . Note that Pedja put his head above the parapet and openly criticised the issues with the research as he saw them.

      Comments and discussion generally will encourage good scientific practice, as will the trend towards data sharing.

      I agree that PubPeer has driven a level of discussion and debate that simply does not happen in the traditional peer review system. This is great. And I sing PubPeer’s praises at any available opportunity!

      But we don’t put a cardboard box on our head to conceal our identity when we attend a conference/workshop and ask questions or debate science! And I don’t see the points you raise as being particularly strong arguments to support the continued encouragement of anonymity for online PPPR.

      Like

      1. “If there are serious issues with a paper we should be willing to put our head above the parapet and say so. We have to start fostering a culture that enables this to happen.”

        I don’t think it is just a problem of “culture”, there are some real, specific risks that need to be eliminated and I don’t see how. Who will pay to defend critics legally – nobody. So they are certainly exposed to legal risk (typically a frivolous libel suit). And how can you ensure that the anonymous grant/paper/application referee isn’t subtly working against the critic? You won’t be the editor, and you probably won’t be in that grant panel or search committee either.

        Basically, it’s simply impossible now or in the foreseeable future to guarantee any kind of safety from bad faith actors, however much one would like to foster a supportive culture. And in that situation, many people will take the lower risk option of saying nothing, unless they can criticise anonymously.

        Like

  5. @practicalfMRI (Sept. 10 2015 at 2:13 AM)

    First, it’s a shame that you felt you had to use a pseudonym to comment 😉

    I’ve explained the difference with traditional peer review before. In traditional peer review there are two major differences: (i) the comments aren’t public, and (ii) there’s a moderation process. This makes the traditional review process very different from PPPR via sites such as PubPeer.

    ” Instead, I saw a couple of examples of people I knew missing something that I find obviously wrong and thus the only “benefit” was to think that the editors selected the wrong reviewers.”

    ..and your argument is that if they had been anonymous reviewers they wouldn’t have missed this? Hmmm. Traditional anonymous peer review has many, many flaws, otherwise we wouldn’t have sites like PubPeer in the first place.

    Like

    1. @Boris Barbour (Sept. 11th 2015 at 4:56 pm)

      I don’t think it is just a problem of “culture”, there are some real, specific risks that need to be eliminated and I don’t see how.

      Culture and the potential for litigation are two distinct, though connected, issues. The central problem is that it is now the norm — regardless of the potential for litigation — for PubPeer commenters (and, indeed, commenters for just about any online forum) to be anonymous. It’s this culture that I’m challenging.

      You yourself say that the legal risk often amounts to a “frivolous libel suit”. We have to openly fight the idea that scientific critique should be open to libel suits. Sense About Science set a very important precedent: http://www.senseaboutscience.org/pages/keep-libel-laws-out-of-science.html

      “And how can you ensure that the anonymous grant/paper/application referee isn’t subtly working against the critic? “

      You can’t. But that argument can equally well be applied to any type of criticism of another researcher’s work. Are you saying, for example, that we shouldn’t author papers which are critical of another group’s work in case we piss them off and affect our chances to get a grant (in case they review the proposal)? Really?

      And in that situation, many people will take the lower risk option of saying nothing, unless they can criticise anonymously.

      I’ll say it again: I find this aversion to openly challenging published research (and to any form of risk) immensely dispiriting. It’s a sad indictment of the state of science. We can either hide behind anonymous cover or we can openly challenge issues with the scientific literature. I prefer the latter approach because it fosters a much better culture of scientific debate, but it’s clear that I’m in the minority!

      Like

      1. For now we’ll mostly have to agree to disagree, but I’ll just finalise a couple of points.

        We probably have a misunderstanding about how chilling even a frivolous libel suit is. It’s no joke. Costs mount very quickly, with both winning and recovering costs being uncertain. So some random PhD or post-doc will be entirely defenceless before a motivated litigant with superior resources, whatever the weakness of the case.

        You may know that PubPeer commenters have been sued. Had they signed their comments, they would already be in court. How do you envisage “culture” would have helped them defend the case?

        Note that I also have a reputation for openly challenging published research, but I wouldn’t feel happy encouraging others to take that risk.

        Like

    2. I know you’re joking but in case anyone else reading is interested in my perspective on this particular issue, my pseudonym is merely a catchy moniker to encourage people to send me their questions, etc. A cynic would say that I’m marketing myself, and they wouldn’t be a hundred miles off the mark. That said, it is trivially easy to determine who I am if anyone is bothered, and some of my posts on The Winnower have appeared under both my real name (final version) and my pseudonym (draft version). If it still bothers anyone I’d be prepared to send everything from my address to height, weight, DoB and shoe size. I might even send you a goofy pic!

      But anyway, back to anon commenting.

      “I’ve explained the difference with traditional peer review before. In traditional peer review there are two major differences: (i) the comments aren’t public, and (ii) there’s a moderation process. This makes the traditional review process very different from PPPR via sites such as PubPeer.

      ” Instead, I saw a couple of examples of people I knew missing something that I find obviously wrong and thus the only “benefit” was to think that the editors selected the wrong reviewers.”

      ..and your argument is that if they had been anonymous reviewers they wouldn’t have missed this? Hmmm. Traditional anonymous peer review has many, many flaws, otherwise we wouldn’t have sites like PubPeer in the first place.”

      I’d like to see the anon comments. I don’t see why the comments in trad peer review should remain confidential, they could be a very useful component of the scientific record. But I don’t feel as strongly about this as I do about anonymity in general. So I’ll move on to the signed reviews. Here I do have a very big problem with not being able to see someone’s review. All I can do is guess what someone might have caught/missed. For all I know the reviewer I know has caught a problem and the editor overruled it somehow. It’s all speculation. So my point is that by publishing the names of the reviewers I have no new, useful information on which to judge the work. Frontiers hasn’t fixed a problem by naming their reviewers. Imo they should either name the reviewers AND publish their reviews, or simply not bother to list the reviewers so that I don’t form bad judgments about people I know could do better. In sum, then, to me anon PPPR > published anon reviews >> signed reviews.

      Like

      1. There’s no question that the reviews should be published. We agree entirely on that. There is no justification for those reviews remaining confidential.

        Whether the reviewers’ names should also be published is where we differ. If reviewers are putting their name to their comments then there is rather more motivation for careful reviewing. You will, quite correctly, point out that signed reviews open up a lot of possibilities for the “social” side of scientific debate to become rather more bitter and heated. For example, one can easily envisage scenarios where if someone rejects your paper to Nature, and you know who they are, you may not be so predisposed to treating their next paper extremely kindly if you get it to review.

        But we all know the traditional journal system is flawed in any case. So why don’t we move towards a system where, like the astronomers and particle physicists, we exploit an arXiv-type approach? Good papers will then become “visible” to the research community not via the brand-name of the journal but through non-anonymous debate and discussion via a comment-enabled arXiv system.

        Like

      2. “like the astronomers and particle physicists, we exploit an arXiv-type approach? Good papers will then become “visible” to the research community not via the brand-name of the journal”

        Speaking as an astronomer, this sounds more like a vision than reality.

        There are several good things about arXiv:

        o It is a one-stop shop. Essentially everything is there. There is only one place to check for new stuff.

        o An arXiv reference leads quickly to the paper. A traditional reference does not, at least not usually. A DOI will, but a) not all journals have them and b) they are sometimes rather cumbersome.

        o The stuff there is freely accessible.

        o One can revise the paper. While I dislike revisions after acceptance or at the latest publication by a journal (if it is that important, publish an erratum), it is sometimes useful to update stuff like links to supplemental information.

        o One can put stuff there before acceptance, or even submission. I don’t do so myself, at least not usually, but for some people and/or in some circumstances this might make sense.

        However, I don’t think that arXiv has really changed the culture of debate that much, though. No-one can read everything. People filter. A common filter is publication in a respected journal. Many people forget that one reason for the high quality of stuff at arXiv is that most of it is intended for, and eventually appears in, a traditional journal (which these days might be online only).

        I’m sure that arXiv papers which are intended for a traditional journal and/or where readers recognize the names of the authors get much more attention.

        Also, it is not clear who can put stuff on arXiv. Apparently an endorsement is necessary (except for us old folks who were “grandfathered in” when the endorsement system was set up) but not always sufficient. There is a selection effect (something astronomers should understand) at work here: most people who have trouble putting something on arXiv are hesitant about discussing it publicly, for fear (justified or not) that it would make their situation even worse. Some of this has become more transparent: there is now a defined appeals process and so on. (To be fair, there are some otherwise respectable journals which apparently have other criteria in addition to scientific quality which are used to determine who can publish there. However, an author can go to another journal, but one can’t go to another arXiv, because there isn’t one, and competition here wouldn’t make sense, because it would destroy the one-stop-shop concept (unless everyone put all papers in all repositories, but even here it might be that there are some accepted at A and not at B as well as vice versa); sometimes a monopoly is good, and this is one of those times.)

        Like

  6. Hi, Boris.

    “Note that I also have a reputation for openly challenging published research, but I wouldn’t feel happy encouraging others to take that risk.”

    I understand entirely your qualms and concerns regarding libel cases. But anonymous commenting will never fix this — it simply means that the problem is brushed under the carpet. Where it’ll fester.

    Universities need to protect their staff against these types of legal challenge. It is beyond farcical that we have reached the point where pointing out deficiencies in published research can lead to libel cases. I also remain of the opinion, however, that the type of legal challenge you describe represents the extreme end of the spectrum. Is this type of legal issue really endemic?

    Julian (Stirling), Raphael (Levy) and I have been rather vocal about the deficiencies in a series of papers related to the morphology of nanoparticles. We have not faced any type of legal challenge.

    As you say, we’re going to have to agree to disagree on this aspect of PPPR. Once again, however, I’d like to say a big thank you for establishing PubPeer — you and your colleagues have made an exceptionally important contribution to the evolution of peer review (and, thus, to the evolution of science). I will certainly continue to sing PubPeer’s praises when I give conference presentations (although I’ll try to encourage colleagues to put their name to any comments they post!)

    All the very best,

    Philip

    Like

    1. Philip,

      I enjoy your blog, but I am interested in your thoughts on anonymous speech and political freedom.

      In many counties “to speak truth to power” can be a dangerous occupation. There was a case recently on retraction watch where academic and political freedoms began to overlap, though fortunately no one was injured in this case:

      http://retractionwatch.com/2015/09/08/whistleblower-released-after-being-held-for-4-days-in-bangkok-airport/#more-32088

      I know that free anonymous political speech is abused even more than free speech of academics, but how do you avoid abuse without also the removal of the protection that it offers? This is a question that many parts of the Web have been dealing long before scientists started to get involved!

      AC

      Like

      1. You have provided a great example of exactly the culture I have been railing against. Your comment is entirely innocuous, with nothing in there that could in any way lead to legal proceedings or even censure of any type. And yet you post under a pseudonym. With an anonymous e-mail address.

        Why? Why couldn’t you put your name to your comments? What is it that means you can’t openly stand behind what you’ve written above?

        I am well aware of the dangers of speaking truth to power under some circumstances and, indeed, I highlight this in the post. (And have discussed it with Boris and Raphael in the comments thread above). Your comment is light years away from that type of situation — you have said nothing that could endanger your freedom.

        And yet you still posted anonymously.

        Like

      2. In reply to Philip (no reply option was available on you comment).

        Leaving aside the anonymous nature of AC, it would be interesting to hear your answer to his/her question. Are you always against anonymity or do you recognise it as necessary under some circumstances? Where would you put the threshold?

        Like

        1. Boris, I said specifically in the post that there are cases at the extreme end of the spectrum where anonymity is justified. Here’s the exact quote:


          I am not a fan of internet anonymity. At all. I understand entirely the arguments regularly made by PubPeer (and many others) in favour of anonymous commenting. In particular, I am intensely aware of the major power imbalance that exists between, for example, a 1st year PhD student commenting on a paper at PubPeer and the world-leading, award-winning, scientifically decorated and oh-so-prestigious scientist whose group carried out the work that is being critiqued/attacked. Similarly, and in common with Peter Coles, I personally know bloggers who write important, challenging, and influential posts while remaining anonymous.

          I also fully realise that there are are extreme cases when it might not only be career-theatening, but life-threatening for a blogger to reveal their identity. However, those are exactly that: extreme cases.

          As to where I’d put the threshold? Well, it’s certainly rather far removed from posting an innocuous comment on this blog! AC’s comment is exactly what I’m talking about. There’s absolutely no need for anonymity. They’ve said nothing at all contentious and yet the default position is anonymity. This culture infects the internet and, as I said in the blog post, normalises the viciousness seen in so many online fora. (I’m speaking more broadly than PPPR here. See https://muircheartblog.wordpress.com/2015/06/17/sexism-murder-art-and-science/).

          I guess I’m just “old school” — I think it’s rather more polite, courteous, and helpful to be as open and honest as possible.

          Like

      3. “Why? Why couldn’t you put your name to your comments? What is it that means you can’t openly stand behind what you’ve written above?”

        Since I was asked (perhaps facetiously) the same essential question… Most likely the answer is prosaic: it is how AC has got his/her account set up for commenting on forums where there isn’t a whole lot of decorum. That’s most of the web. But what I’d like to know is why it matters whether you have AC’s full name. Do you need the affiliation, too? Would you Google AC before responding? Is it an ability to locate someone physically that is your real intent? I’m trying to understand your motivation and just how much about AC you need to know before your desire for openness and honesty is satisfied.

        Like

        1. I’m trying to understand your motivation and just how much about AC you need to know before your desire for openness and honesty is satisfied.

          My motivations are very straight-forward and are explained in the blog post. Anonymity is fundamentally a lack of honesty at some level. It’s a lack of willingness to communicate openly and transparently. As I’ve said to Boris in a comment above, we don’t turn up at scientific conferences and workshops with cardboard boxes on our heads, painted with our own particular favourite avatar.

          Anonymity means that someone can make comments with impunity and never take responsibility for them. It’s this culture that gives rise to a great deal of the viciousness on the web. Scientists, as Feynman said, are meant to subscribe to “a principle of scientific thought that corresponds to a kind of utter honesty–a kind of leaning over backwards.” We should aim to debate as openly and honestly as possible. Anonymity does not move us towards that goal.
          .

          Like

      4. “As I’ve said to Boris in a comment above, we don’t turn up at scientific conferences and workshops with cardboard boxes on our heads, painted with our own particular favourite avatar.”

        Except for NeuroSkeptic, perhaps! But this is an interesting point. I know from personal experience that I don’t ask 90% of the questions at a conference that I would like to have answered. In part it’s because putting oneself in the public eye is a tough thing to do. If it weren’t we would all love public speaking. Many conferences have moved to allowing Tweeted questions and the like, and the engagement level goes up. This is at least partially due to the reduction of the feeling of personal vulnerability, not merely convenience.

        “Anonymity means that someone can make comments with impunity and never take responsibility for them.”

        Yeah, I’m with Rafael here. I’d like to see the evidence that anonymity is mostly harmful before I shift my position. The vast majority of what I’ve seen on PubPeer has been constructive and focused on the work. So in my opinion, anonymity on PubPeer has very much moved us towards the goal of open and honest discourse. It’s not perfect but nothing is, and it’s generally a good idea not to let the perfect be the enemy of the good.

        Like

  7. @practiCal fMRI

    To expand on what I said above. It’s not even that I care particularly who AC might be. No, I didn’t Google them (or wouldn’t Google them if I could). And no, I don’t want to know what they had for breakfast!

    It’s simply the principle of standing behind one’s statements and taking responsibility for them. Anonymity means that one never has to account for/be responsible for statements that have been made. We are so inured now to the use of pseudonyms and anonymity online that we see it as a natural way of communication. It’s anything but.

    Like

  8. @ practiCal fMRI September 14th, 2015 at 4:42 pm

    I know from personal experience that I don’t ask 90% of the questions at a conference that I would like to have answered. In part it’s because putting oneself in the public eye is a tough thing to do.

    Indeed. And that’s because there’s no cover of anonymity and you realise that you have to take responsibility for your comments. So language is moderated accordingly and the exchanges do not descend into a slanging match as frequently as the flame wars that develop online.

    So in my opinion, anonymity on PubPeer has very much moved us towards the goal of open and honest discourse.

    Anonymity, for the very reason you suggest, has led to more discourse. But it promotes that discourse only because those commenting know that they ultimately don’t have to take any responsibility for what they say.

    As I’ve said before ad nauseum , I find this rather depressing. As I also said to Raphael, when I say “mostly harmful” what I mean is that anonymity promotes a basic lack of willingness to stand behind what one says. It’s got nothing to do with the question of the validity of the comments and everything to do with the culture of scientific debate.

    Guess I’ll just give in at this stage, join the crowd, and stick that cardboard box on my head…

    Like

    1. Agree that there is much to get depressed about when it comes to discourse, culture, etc. I prefer to encourage more debate and deal with the bad behavior as it arises, rather than try to ensure only civil debate and thereby curtail some involvement. And if needs be the corollary to free speech is freedom to listen, or not, as one sees fit.

      As for giving in, there’s no need! You can continue to lead by example if you feel strongly about it. I would never dream of stopping someone from signing a review or a comment, if it’s their choice. For me it won’t make a jot of difference how I interact with you, but I do understand that not everyone has the same opinions or situations. And that is why I want others to be able to post anonymously: because not everyone is like me.

      Like

      1. That’s a good way to close this, practiCal fMRI (although I still wish that I could refer to you by something other than that convoluted moniker!). You make a lot of sense, although I’d disagree with the idea that I lead by example. Anyone who follows my example in just about anything will end up in a very bad place indeed! 😉

        Been a pleasure discussing this with you.

        Like

      2. “although I still wish that I could refer to you by something other than that convoluted moniker”

        Whatever his real name is, it is no match for “Professor Moriarty”!

        BTW, has anyone ever thought that “Prof. Moriarty” was a pseudonym?

        Liked by 1 person

  9. @Philiip Helbig

    Many times it’s been assumed that Prof. Moriarty was a pseudonym. And then, when I point out that it’s not, some of them have had the temerity to argue with me and tell me I’m lying. Even after I pointed them to my university homepage!

    On the issue of the arXiv you mentioned above, I’ll agree. What I’m suggesting is a long way from what the arXiv is at the moment. But wouldn’t it be good if it could evolve that way?!

    Like

  10. Well there is nothing like wading into a debate when it has gone quiet, just to loudly shout ones opinions at an echoing empty room. Pity I didn’t get here on time. Anyway, to my loud opinions.

    In general I agree with Phil really shouldn’t be an expected and necessary part of a debate. But I think there are two separate types of discussion which happen on pub peer, and one anonymity really hurts, the other it is at worst a mild annoyance to hard-liners such as me and Phil.

    Starting in reverse. Pub peer has had great success when something obviously wrong has been found, and someone feels the need to highlight this. If this is big and obvious enough then once it is pointed out the ball is now rolling, it is clear there is a real issue. Who cares if this starts as an anonymous tip off? It really doesn’t matter.

    The second type of discussion is the in depth scientific discussion of a particular issue in a paper. Here is where anonymity really starts to hurt. In a perfect world, of course, it shouldn’t matter who said what, only the truth should matter. But anonymity brings with it the chance to be a sock puppet, the chance to hide conflicting interest and the chance to say whatever the hell you want to give the illusion of a real debate only to cut and run when you get called on it (perhaps run from the debate, or run to a new pseudonym and start again).

    With this second type of discussion, I see can’t see any good reason to be anonymous. If you actually have the expertise in a field to argue a specific point in detail what possible incentives are there to hide your identity. I see a few (there are probably more):
    * You aren’t really that confident in your criticism and want to be able to ditch it without the embarrassment of people knowing you were wrong.
    * You have a clear conflicting interest and want to hide that.
    * You are an early career researcher and you don’t want to start a spat with those at the top.

    Frankly, I have zero sympathy for any of these 3 objections:
    1. If you aren’t confident then it is either because your idea is half baked and you really owe yourself and those you are criticising the courtesy of really thinking it through until you are confident. If you turn out to be wrong, then just admit it! We have all been wrong!
    2. It is obvious what is wrong here.
    3. As a PhD student this never put me off a good debate with those far senior to me. If you plan to get to the top of your discipline without ever publicly questioning established views then you have a very strange view of science.

    For me I will always answer one off comments from those who are anonymous, but after my experiences with pub peer (http://physicsfocus.org/dark-side-post-publication-peer-review/) I will never get into another lengthy debate with anonymous commenters.

    Like

    1. Hi Julian,

      I followed stripy saga with interest. I’d just like to add a couple of comments about your “complaints” about anonymity, from the point of view of an interested outsider.

      I’m guessing that everybody interpreted the “sockpuppet” comments as a pretty desperate defence by Stellacci. So, although I can understand that the discussion was frustrating (everybody wants the last word), I didn’t consider the other comments to be at all effective, and therefore maybe not so hurtful.

      As others have commented, PubPeer has seen very, very few instances of such, err, energised commenting. Maybe hard cases do make bad law.

      Like

  11. We haven’t all flown the coop, Julian! Some of us under pseudonyms are lurking in the Internet’s darker corners 🙂

    You said:

    “The second type of discussion is the in depth scientific discussion of a particular issue in a paper. Here is where anonymity really starts to hurt.”

    Yes, I can see that. Your example – and examples are always very useful – is a good one to make the point. So let me pour a bit more concern onto yours. The use of an unmoderated* online forum, especially when there are many participants, may not be the best place to have an in-depth discussion at all. People online don’t like to be kept waiting for responses, for instance. That alone can lead to confusion. And, as the slightly bizarre placement of “reply” buttons in this comment thread attests, it is all to easy for people to be misunderstood simply because they are trying to answer comments from different people over a span of time. In such a situation I don’t think it matters whether the participants are anonymous or not, the problem is logistical. And imo, it would be time to slow the debate down and ensure that everyone can track the conversation properly. At which point it may well be suitable to attempt to remove the anonymity, depending on the circumstances.

    So perhaps what we need is a separate, moderated “VIP area” where a discussion can be limited to a few participants yet watched by all. (You can see it but you can only get there if you’re invited!) The rabid masses would then be free to shout their comments in the cheap seats without distracting from the main debate. New VIPs could be added to the VIP area as determined by a moderator.

    *I realize that other users can moderate to a certain extent, but at some point it degenerates to people shouting at each other online if nobody has the power to hit a mute button.

    Like

  12. This terrific discussion is itself a beautiful example of post-publication peer review.

    There is clearly an important role for the anonymous comments that identify critical flaws in papers. I agree with PubPeer that the option of anonymity is useful there, especially if misconduct is suspected.

    There are also many wonderful constructive conversations on PubPeer. However, it seems the reputation has been cemented as a place to uncover deep problems. I fear that while PubPeer was launched to become an online journal club, it won’t be able to really deliver on that vision.

    I am dreaming about a time when many academic journal clubs end up online after the in-person discussion. I am a huge fan of PubPeer and 1.5 years ago wrote about this goal (http://blog.pubchase.com/we-can-fix-peer-review-now/). Today, despite the success of PubPeer, I wonder if we need a separate site dedicated exclusively to constructive journal club discussions. (Possibly PubPeer itself can move towards that despite the current reputation.)

    The fraction of papers that need to be shamed and flagged as problematic on PubPeer is small. The fraction that can benefit from a public Q&A and discussion – pretty much all.

    Like

    1. That would be indeed a nice world of science to live in. Instead, one hears senior scientists make false congratulations to each other on new papers they either never read or disagree with.

      Will junior scientists ever dare to critically discuss work of other labs in the open, non-anonymously? Or will the PIs from that labs call their PI and complain about the insolence, with all the predictable consequences to the brave junior critics?

      Maybe this is why the main thing largely going on PubPeer is anonymous flagging of misconduct?

      Like

    1. Thanks for the link. I’m familiar with that frustrating case. Yet another issue with traditional, anonymous pre-publication peer review.

      Like

  13. I went the pseudonymous route. Probably partly because I was mainly commenting on a topic that was not specifically related to my professional life, partly because the topic was known to be contentious (climate science), and partly for no good reason at all. In retrospect, I may have chosen differently if I knew then what I know now, but I didn’t. I also didn’t expect to do anything other than write a blog pseudonymously. I didn’t expect many people to read it. I’ve know been “outed” which is actually quite nice, but I still choose not to directly associate myself with my blog (again, for no really good reason). I have, however, published enough under my own name that links to my blog, that the link is now pretty well known.

    Okay, this comment has rambled. I guess what I was going to say is that I broadly agree with the point you’re making “people shouldn’t use anonymity to say things that they wouldn’t say if known”. I don’t know if I specifically achieved this, but it was certainly a goal. Also, whether one should remain pseudonymous or not might depend on the impact your writing is having. Quietly writing a blog that noone notices, or the odd innocuous comment, probably doesn’t matter. Being involved in some kind of major public issue while pseudonymous probably does matter. However, maybe the thing I’ve noticed the most is those who will be highly critical of some people’s pseudonymity, while completely dismissing – or ignoring – the pseudonymity of those who say things with which they agree.

    Maybe one should also distinguish between someone who comments pseudonymously, but always uses the same handle, and those who use multiple handles.

    At the end of the day, it would be very nice if people commenting online behaved as they would if they were interacting face-to-face. It’s possible that pseudonymity/anonymity gives some the freedom to say things that they wouldn’t say otherwise. Having interacted online about a rather contentious topic for a while now, I’m not convinced that it is only this. Some seem willing to say the most appalling things online even if known. I wish it were as simple as simply people being pseudonymous/anonymous, but I don’t think it quite is. There may, however, be circumstances (commenting on papers in your field) where it would make a big difference, though.

    Like

    1. Thanks for this important comment. I’m about to go into town to see “Horrible Histories Live” with the kids now and am similarly occupied over this weekend (it was my daughter Saoirse’s 10th birthday yesterday). I’ll reply as soon as I can because your comment deserves a considered response.

      Like

      1. Thanks. Enjoy the show. Another observation that I meant to make (but forgot) is that – having been outed – if I comment online, I will either have another commenter pointing who I am, or those who respond to my comments will do so using my real name, not my pseudonym – even if they are themselves pseudonymous. It doesn’t both me, but I assume it’s meant to be some kind of intimidation tactic: “I know who you are”.

        Like

  14. It’s simple. Any anonymous commenter has to first be reviewed by named authors (no reviews of anonymous by other anonymous. The latter would only count if other named associates concurred.

    Like

  15. Hello Phil. Within academia, we are faced with a shift towards greater anxiety and precariousness: more short-time contracts, fewer permanent jobs.There are high-profile cases of people’s employment status being threatened by what they say online. Taken together, this provides powerful disincentives against sticking their head up above the parapet.

    I agree with the observation about it being uncomfortable talking to anonymous people online. I don’t automatically find it uncomfortable, but there have been occasions where I could no longer have as open an exchange as I would like without understanding more about their background and motivations.

    More broadly, there is obviously a tension here between i) scientific norms of the *ideas* being the important thing, removing personal biases in the interests of seeking truth ii) having ‘good faith’ exchanges with people (not that I’ve ever seen a good def of ‘good faith’, but it’s a concept that I’ve seen used a lot).

    Like

Comments are closed.