Tag: education

  • 15 tips on making your lab a better place for ECRs

    15 tips on making your lab a better place for ECRs

    Download as an infographic

    Academia can be an exceptional career path but it can also be tough, especially for Early Career Researchers (ECRs). Having navigated the world of academia myself, I know first-hand what a difference it makes when departments and labs consciously create a supportive environment for ECRs and the damage done when they don’t.

    Here are 15 actionable ways you can make your team a better, supportive, and more inspiring place for the next generation of researchers. Some are quick wins, others are going to require some work. All of them matter.

    1. Invite ECRs to Give Seminar Talks (And Cover the Costs, Upfront)

    ECRs need platforms to share their work and build their networks. Regularly invite them to present at seminar series. But don’t just offer them a slot; cover their travel (ideally up front!), accommodation, and, if possible, honoraria. Here’s a great example of how it can be done. Starting or changing your seminar format to one specifically for ECRs would be a huge boost and move us away from those same old boring conference talks.

    2. Prioritise Dedicated Training Over Journal Clubs

    Replace some traditional journal clubs with real training: data analysis workshops, grant writing bootcamps, public peer review sessions, presentation skill seminars, and meta-science discussions. ECRs are hungry for skills beyond technical lab work. Give them that and not only will your lab benefit from more rounded individuals but the ECRs will be better placed for alternative careers and academia.

    3. Actively Encourage Collaboration

    Encourage and facilitate collaborations within your lab, your department, and outside your institution. Networks are critical for scientific growth and career development.

    Insular labs stagnate; collaborative ones thrive

    4. Foster Public Speaking & Writing Opportunities

    Push your ECRs to give talks (both academic and public-facing) and to write (opinion pieces, reviews, blogs, preLights, etc.). This builds their CVs and confidence, and opens up alternative career avenues. Make introductions, offer edits, and celebrate their successes.

    5. Host Annual Career Development Days

    Dedicate one full day each year to reviewing CVs, prepping for job or fellowship interviews, and working on grant applications together. Share your insider knowledge of funding opportunities, especially those small pots of money for independent ECR projects.

    6. Relieve Unnecessary Pressure and Stand Up for ECRs

    You may not be able to pay everyone more, but you absolutely can use your position to shield ECRs from unreasonable demands, departmental politics, or unnecessary admin. Don’t forget what it felt like at their career stage. Challenge institutional inertia on their behalf.

    7. Treat ECRs as the Experts They Are

    Most ECRs are postdocs or just a lucky break away from a permanent post themselves. Treat them as equals and experts, not just as helping hands. If you need reminding of what the postdoc experience can be, read more.

    8. Listen—and Act—on What ECRs Say They Need

    When ECRs say they need equipment, software, or support, believe them. Prioritise these needs and advocate for necessary resources. Their ability to thrive depends on it and so does your lab’s output.

    9. Cover Upfront Costs Instead of Passing the Burden Down

    Departments (not individuals) should absorb upfront costs, or you risk excluding those who can’t afford a financial hit. As a PI, advocate fiercely for this at department level. This isn’t just about bringing academia in line with normal businesses, it’s about equity and inclusion.

    10. Make Your Space Safe and Inclusive, Always

    Speak up against any abusive behaviours or toxic politics even if (especially if) it means calling out another PI. Silence protects abusers. ECRs should never have to navigate harassment, discrimination, or power games just to do their job.

    11. Avoid Internal Competition and Value Individuality

    Don’t pit ECRs against each other on the same projects. Take time to know each person’s goals and working style. Support those who want to stay in academia and those considering other paths, equally and openly. Avoid favouritism at all costs as it will slow progress, damage your labs culture and result in people leaving.

    12. Design and Update Individual Training Plans

    Every ECR deserves a clear, personalised training plan, reviewed annually. Training is more than just learning a technique, it’s about building independence, transferable skills, and professional confidence.

    13. Take Leadership Courses Yourself

    If you run a lab, commit to learning good management and leadership! Consider formal courses in leadership and people management (this should really be mandatory for all PIs and provided by institutions). You can’t know what you were never taught; investing in yourself helps your team too. We’re developing such courses so keep your eye out.

    14. Preprint Your Lab’s Papers

    Timely outputs are critical for ECRs applying for jobs, fellowships, and grants. Preprints can make all the difference, saving 6+ months. Make preprinting your default and encourage your team to get their work out there when it’s ready.

    15. Model Integrity and Open Science

    Lead by example: be transparent in your research, foster a culture of sharing, and have regular discussions about research integrity and communication. How you behave sets the standards for those you train and the next generation of PIs.

    Bonus: Small Changes Really Do Matter

    It’s easy to think that improving life for ECRs in your lab or department can’t change the system. It absolutely can. For those ECRs in a genuinely good environment, it’s transformative and your example sets new expectations for others. Change is slow, but it always starts somewhere.

    If you’re a PI, lab head, or department leader, you might already be doing some of these things. If not, now’s the time to start. It matters more than you might realise for your ECRs, your science, and the future of academia.

    If you found these suggestions useful, please share, and let’s build a better, kinder academic world together.

  • How to get more involved with preprints and open science

    How to get more involved with preprints and open science

    Preprints are revolutionising the way we share and communicate scientific findings. They have numerous benefits and advantages for all stakeholders but particularly for ECRs. If you are an ECR you need to be posting preprints. If you train or are responsible for ECRs then you need to be making sure you facilitate preprinting of their work.

    But how can you get more involved in this fast moving world?

    Use preprints

    OK I’ve just mentioned this one but not only should you be posting preprints yourself but you should be reading and citing other preprints in your field. This will keep you 1–2 years ahead of those who only read published papers. When you do publish, choose open access journals and those that are more friendly to changing the broken system.

    Host/take part in preprint journal clubs

    Journal clubs can be useful and are often a staple in “training” for ECRs within a bioscience department. Stop picking CNS papers because they’re flashy and start using preprints to be at the true cutting edge of your field. To make them even more useful you should spend a little extra effort on writing up the discussion as a comment for the authors. This way your journal club is helping to advance preprint use and also advance science by helping authors refine and improve their work.

    Share data and methods openly

    Tied into using preprint but if you have a dataset or useful methods, upload these to repositories when you post the preprint. Sharing code openly can even lead to new collaborations and significantly improve your own work — we found this with our COVID papers where sharing openly led to a collaboration for the first paper and then posting a preprint led to our second paper being co-published, again making the conclusions much stronger.

    Educate yourself (and others)

    It’s so surprising how many academics (including “esteemed” professors) who just don’t understand the history of our publishing system or where peer review comes from. This is vital in understanding the problems within the system and why it needs to change. There’s a lot of survivorship bias in academia and looking back can help us move forwards.

    Follow open science leaders

    Some of the brilliant people who are leading the change towards open science and preprint use are very active on social media. On BlueSky, you can follow the Preprints and Metascience feeds.

    Get involved with communities

    This is perhaps one of the best ways of getting more involved in preprints and open science.

    PREreview —  community and training focussed on increasing equity in preprint peer review. Recommended platform for uploading community or journal club reviews of preprints.

    preLights — preprint highlighting that allows you to write about interesting preprints and collaborate with others in the community. An excellent initial (active) step into the world of preprints.

    Preprints in Motion — podcast focussed specifically on highlighting preprints and the ECRs behind them in addition to discussing the wider issues in academia. Contact preprintsinmotion@gmail.com

    Talk to co-workers about preprints

    Now you’re using preprints, you’re writing about them or have been involved in the fellowship programs above. Get out there and tell everyone why they should be preprinting and making science a better place for all! Spread the gospel!

    If you’ve posted an interesting preprint or read one recently you can also highlight it to Preprints in Motion for a full podcast episode focussed on the preprint and ECR.

    Attend open science events

    There are many open science events you could attend such as conferences and workshops from FORCE11 and various universities (e.g. Sheffield University’s OpenFest).

    Write about preprints

    This may be through preLights but can also be more casual or opinion pieces in science magazines and journals. I’d strongly recommend preLights because not only is it a great community but it helps establish your own name in the preprint sphere.

    Start your own initiative

    We’re always happy to discuss ideas and provide support in some exciting new initiative led by you!

  • What’s right with academia

    What’s right with academia

    The positives of an academic career – there are many!

    Work when you want…I mean flexibility!

    Don’t go to work — no one cares. In academia you don’t have to deal with customers or tight deadlines or anything external that demands you physically be at work all of the time. Yes you might have some demanding cell cultures or meetings but outside of that you’re time is yours. This means you can collect kids from school or work from home or even not actually do any work at all (think mental health day rather than lazy day)! Academic projects never really end, a paper is just a launch point to another paper. This means things move slowly with often ill-defined goals and objectives. But this also means that there (can be) less pressure, certainly with regards to delivery.

    This is a genuine benefit that is very difficult to find in other jobs, in fact the only others who probably get to do this to the same level are those who work for themselves.

    Freedom

    Very much related to “work when you want” is freedom. That’s freedom to perform research on the topics that interest you. There are some constraints such as funding, jobs etc but generally this is your choice.

    You also have the freedom to work with whomever you wish. You can choose brilliant (and lovely; never underestimate a good person) people to work with through collaborations. This gives you a global community of passionate experts. You see this community whenever there are institutional seminar series where the organisers mates are invited around the world to catch up…erm give scientific talks and network. Few other careers are quite as global (at the employee level).

    Now don’t get me wrong, this is one of the most touted benefits of an academic career but often without the caveat that it’s only at its truest when you reach the PI-level. But as a postdoc you can choose which lab you work in and therefore the topic you work on. You can direct your research down avenues that interest you. You can learn techniques that you want to learn.

    Travel

    Speaking of that global community, you also get to travel. That may be seminar invitations (again, largely PIs only) or it could be to learn a new skill or to attend a conference. There is a degree of travel in academia that isn’t common to many alternative careers. And that’s nice travel too. Not regular travelling to see clients with lots of flights. This is once or twice a year travel to (usually) beautiful parts of the world, all paid for through grants and travel awards. And as you move up you get more travel opportunities and invitations — hell there are some “big name” profs who have conference seasons where they just travel around giving the same talk for large chunks of the year; I’m sure I’ll comment on my distaste of that eventually.

    Job for life

    Now I really struggled to include this one as it is not at all as true as it may have once been. But for the very lucky few, academia is a rare career that can give you a job for life. Certainly, it is exceedingly difficult to fire an academic who is producing papers (even, apparently no matter what incredibly unacceptable things they may also be doing). This is heavily restricted to PI-level and above however and so, although a benefit, I won’t linger on this one.

    Yearly cycle

    There’s a certain amount of structure to an academic year. Much like the seasons, the academic year has clearly defined structural periods. There are teaching periods, followed by space for pure research with undergraduate students coming in at set periods each year. It can be nice to have that varied structure over the course of each year.

    The thrill of discovery

    One of the best parts! My favourite part of academia is that feeling you get when you first discover something new. Maybe you’ve spent hours on a confocal and see some amazing cell behaviour. Until you excitedly run down the corridor telling everyone, you are the first and only person across the entire existance of humanity to have that nugget of knowledge. And that is a pretty good, and addictive, feeling.

    That feeling is then followed by the fun of communicating your findings (and not just in a paper). This is perhaps my favourite of those “7+” jobs we have to do each day. Personally, I think it is also the most important role we play. Good communication is often overlooked in academic appointments but it is such a crucial skill and it is through communication that, ultimately, we further humanity and our collective knowledge.

    Teaching/training/mentoring others

    When asked what they most enjoy about academia, many people answer with “mentoring and teaching the next generation”. Second only to good communication, good training is vital to our overall goal of advancing knowledge.

    Although often a source of stress (just look at the current climate in the UK), teaching has huge rewards for academic staff and society. So much so that I don’t believe that I really need to write much here.

    Contributing to something meaningful

    Full disclosure. I was rather hesitant to include this one as so much of the literature just doesn’t fall under this. Most research has very little real world impact, if any. But our work *can* have a real and tangible impact on people, systems and humanity. A small step fowards in understanding how a biological process works is a small contribution to something that will one day result in saving lives and lessening the burden of disease. If we’re real lucky, then our work has a more immediate impact — just look at the vaccine response to COVID-19.

    Science can be a winding path but there is so much light at the end. Photo by Jorge Reyna on Unsplash

    So there we have it, some of what I think are the best bits of an academic career. What did I miss, what is your favourite part of the job?

  • What’s wrong with academia?

    What’s wrong with academia?

    Ah, a question I ask myself almost daily. Now, don’t get me wrong I really do love my job — even if this particular career stage is one that I’m not exactly amazing at — but one of my areas of research is the academic culture. This means I spend more time than most thinking about how we do research and the culture in which we do it. In this post, I will outline some the key issues of working in academia.

    Before we start, this is not designed to deter potential PhD applicants, but I do believe it is essential to be fully informed going into this career. It’s worth noting that, despite what I’m about to say, I am still currently in this career and I’m actively writing grants and fellowships to try and gain an independent position…whilst also being very open to leaving academia in the next year. I might also write a follow up about the positives of an academic career to provide some balance. So lets get into this.

    Mental health

    Underpinning (and all too often caused by) all of the below is poor mental health. This impacts just about every stage of academia but has most often been studied in PhD students, postdocs and junior faculty. There are no shortage of studies proving that this is a major issue. Personally I regularly get into arguements with PIs and (particularly sadly) people at earlier career stages who should know better over this. Too many refuse to acknowledge these issues and dismiss those who speak out as “trouble makers” or “malcontents” who must hate academia. For the record, speaking out and offering solutions is done because I love academia and what it could, and should, be.

    Bullying / abuse / power-dynamics

    “early career researcher (ECR)” — let us just sit with this one for a moment. Somebody not familiar with academia would probably be highly confused that a 35+ year old who’s been in the career for over 10 years, has a PhD and multiple years of postdoc experience would still be considered “early career”. It isn’t just absurd, it’s insulting. But this is symptomatic of a larger problem. By calling people “early career” and framing postdocs as “training positions”, it is easier to justify the low pay and overwork. From the outside, academia often looks like a liberal paradise. The reality however is that academia as a structure is incredibly conservative and reluctant to change. Hierarchy and power-dynamics are at the heart of this reticence to change.

    Then there is the abundant bullying and abuse that occurs. This abuse is everything from gaslighting (which I’ve experienced myself) and micromanagement to sexual assault. There was a very high profile case recently involving David Sabatini and this is unfortunately barely the tip of the iceberg. There was a recent example on Twitter of a female PhD student who was instructing on a course. The student recieved highly sexist and abusive feedback — and this is a widespread problem that female academics face on course feedback/evaluations.

    These things are perpetuated by those in power who consistently refuse to do anything about such behaviour because those big names bring in money and reputation…as if there aren’t hundreds of other perfectly talented people who couldn’t do the same if given the chance. Until there are consequences for such behaviour it will continue to force good people out of academia.

    It’s 7-jobs in one

    Don’t believe me?

    1. Researcher

    2. Teacher

    3. Administrator

    4. Project manager

    5. Team manager

    6. Public engagement officer / sci-commer

    7. Writer

    And you have to be good at them all — without any formal training in any aspect I should add!

    Exploitation

    Unfortunately (key word of this article!) the way that academia is structured, those who most benefit from the system are PIs. Those below that career stage are therefore frequently exploited as little more than cheap or even free labour. There’s no real professional development for PhDs or postdocs and no benefit to doing anything other than working all out to get as many “high impact” papers as possible with the 3 year contract.

    And it doesn’t end when you leave a position to move onto your next. There is an unwritten expectation that you will continue to work on your previous project to help finish it which often means writing papers long into your new position. This isn’t covered by your new salary and can conflict with your new responsibilities. This means that you must finish off your previous work for free and in your own free time.

    Exploitation is perhaps a harsh way of saying goodwill. But given the other issues I’m highlighting here it is perhaps clear as to why that goodwill is disappearing.

    Long hours

    One of the biggest benefits of academia is that you get a lot of flexibility as you’re in charge of your own time. Or that’s the myth anyway…Anybody who works with animals or (difficult) cell cultures will tell you that you run on experiment/organism time. This can mean very long days — just look on Twitter which always cycles back to competing on who works the most hours — but also working weekends or even nights. I’ve had periods where I worked 7 days a week or days where I was in the lab for 15+hours & days that turned into nights (anyone for a 3am time point?). In fact there are still certain experiments (read most) that for me mean very long days as there is no other way of doing such time points. This is easier if there’s a team who can share the workload, which thankfully I mostly have, but if you’re alone or working in a not so supportive team then say goodbye to any free time you once had.

    Got a paper or grant due? Then you’re likely going to be writing that over the Christmas/holiday/birthday break and on any vacation you may take. Now this last one is optional, but far too many people (including me) fall into this trap. Or at least justify it as a “trap”. The reality is that far too many funders release grants and fellowships over such breaks — early Jan should not be a submission date for a career defining fellowship or grant and yet it so often is. This last point of course hurts female scientists the most as they often also have full time family commitments over that period.

    Poor pay (relative to comparable expertise and education)

    3 degrees, 7.5 years of education and a further 4 years of postdoc “training”. Yet I earn less than many who left undergrad for a career and I earn relatively little more than people without any education. We’re all in the same position here as postdocs. The academy makes many excuses for this low pay but, as you’ll see from elsewhere in this post, those excuses don’t hold up to any inspection. I wont talk too much on this as there is an excellent article touching on pay and “training” here. None of this is to say that the pay is bad relative to my childhood background — it’s more than anyone in my family will ever earn — but it is low given the training, education and expertise required. It is low given how much academia takes. It is low given that scientists got us out of a global pandemic.

    For me the low pay impacts my living situation hugely. Being in London, the rental market is completely insane and out of control. Almost 50% of my take home wage each month goes on rent/bills and I do not live alone and I do not live somewhere particularly nice either. This also means that each month I must choose between socialising and putting money into my savings, you know for a future or holidays. Oh and I no longer pay into my pension either and I’m far from being the only postdoc in London without a pension. Academia is quite literally taking from my future.

    Requirement to move/relocate

    The low pay is further impacted by the short contracts. 3 years for a postdoc (max 5) is standard in the UK. This means all postdocs are under a constant pressure and huge instability. Life is effectively on hold for many during the postdoc years. Did I mention those postdoc years can easily last 8+ years (and the length is increasing)? A decade of ones life on hold, full of insecurity, low pay, limited career progression and basically no training. It’s an awful lot to ask of anyone. Not to mention that for female academics, the postdoc years are often the prime child bearing/rearing years. This makes having a family and successful academic career highly challenging. This is all made worse by funders who place an emphasis on moving between institutions. Personally, I’ve lived in 3 cities in the past 4.5 years alone. With each move comes starting your life over all again; new job that often means you need to rapidly learn a new subfield, new city, no/few existing friends in that new city.

    Lack of formal training

    If you’re in a PhD programme or have come out the other end then this will be overly familiar. There is a training fallacy in academia that exists at every stage. It is a system in which nobody is given appropriate (if any) training for their current or future role. It gets less enticing still if you wish to leave the PI-track and move into a non-Ac role as the academy still has no idea how to prepare people for this, despite it being the most common path.

    Worse still, most promotions and fellowships are given based on papers published, luck and scientific ability. Yet as one moves up in academia the things that become important are your ability to manage a project, to manage and train people and effectively run a small business. None of this is assessed or even required to demonstrated before you’re put into positions requiring these skills. This results in a completely broken system where individual careerism is the most important thing — to most but certainly not all, there are some brilliant PIs out there.

    One final point on the lack of training. This leads to huge gaps in knowledge of how we do science (metascience) and why things work the way they currently do. A good example is the publishing industry. Far too many scientists have a surprising lack of understanding as to the history of peer review and publishing. For example, most don’t know that the publishing system as it is was created post-WW2 by Robert Maxwell (father of Ghislane Maxwell, of Epstein infamy). Or that the academic publishing business sits somewhere between the music industry and movie industry in terms of the money involved — which I find mindblowing. If we don’t know why things are broken then no wonder people rely on “my experience” and ignore the evidence.

    Funding & lack of jobs

    Funding in academia is constantly fraught. No-government funds science appropriately, which in light of the COVID pandemic is particularly egregious. But there are also numerous problems with the way the funds are distributed and used.

    For example, in the UK one of the major funders, the Welcome Trust, have recently revamped their funding schemes for early careers and introduced a 3-year post PhD limit for applications to one of their major fellowships. This decision makes a huge number of early career researchers ineligible and is difficult to square with how science works these days (requiring more time and more data to publish papers). Thankfully, most other funders in the UK have removed age-limits on their fellowships. But this kind of decision also creates a huge inequality in the system with lucky postdocs being awarded a fellowship to start their own lab over much more experienced postdocs and those who can actually demonstrate they could do the job of PI, rather than just publish some papers and write a fellowship. Personally, I think we’re going to see a lot more abuse from this too due to the nature of the kind of people who will be awarded these early fellowships before they’re ready but I’d love to be proven wrong.

    Funding is also most often awarded to those coming from big name labs who have the resources to spare for preliminary data and have enough lab members that they can generate large (“high-impact”) papers. This is yet another element of “luck” that flows through the entire academic career. Those who are successful often don’t like this because they feel it takes away from their achievements and hard work. I’m not trying to do that but if so much of the academic career wasn’t down to luck then we wouldn’t be losing so many brilliant scientists.

    The funding situation also means there are few permanent positions, particularly beyond the post-doc stage. Unfortunately, this impacts minorities, women and 1st-gen groups the most.

    Lack of positivity / unappreciation

    Ironic ending this piece by commenting on a lack of positivity so this will be the penultimate section instead. Other than bullying, this is perhaps the section most dependent upon the PI in question. Some bring amazing positivity and appreciation. For example, my current PI is very clear that he appreciates me — and this makes a huge difference on a day to day basis. There’s another PI in the department who is perhaps the nicest and most positive person I’ve ever met. However, others in the department are clear that they do not appreciate me and are actively disrespectful. This also has a big impact in my daily working environment. The problem is that too many are stuck on the notion that success in academia is papers. Anything else is a distraction or waste of time.

    Now from a pure career perspective this is kind of true — we’re not really judged on much else. But from a moral, ethical and humanity view this is wrong. But it also means most efforts to actually improve a department or academia more widely are actively unappreciated. I work in a place where you’re value is largely just how many hours you work. It’s incredibly toxic and forces new hires into working long, unsustainable, hours. It also directly leads to abuse.

    But let’s move away from my experience and look at the evidence. There are no shortage of people coming forward to state that they haven’t been given the recognition they should have, often in the form of being excluded as an author from papers. But it goes much further than this, particularly (again) for women and minorities. Female academics do the majority of the tasks that keep academia running, such as sitting on committees and those things that contribute to the culture of a department. However, these tasks are almost completely unrecognised and certainly not appreciated or valued in tenure applications or promotions. For minorities, DEI work often falls to them. This work can also come at the cost of research output for these groups, making them less competitive in a system that only values one thing.

    Metrics, metrics, metrics

    The competition for positions and grants has lead academia to focus on papers and citations as the primary indicators of success. This is influencing how research is conducted and presented, and makes it hard to complement traditional publishing with new ways to share knowledge. In effect, academia is still largely stuck in a pre-internet age. As more metrics are devised, academics find themselves under even more pressure. Far too many PIs push this pressure onto their lab members, creating some of the issues above. Many more don’t do this but suffer from the pressure — in a job that really shouldn’t have a whole lot of pressure or time sensitive stress. The worst bit of it all is that evidence continually shows, overwhelming, that the types of metrics we use are not effective. And with so many being based on a publishing system (Robert Maxwell remember) that is completely out of control and wholly unfit for purpose.

    Ultimately, many of these issues boil down to egotistic narcissists, sorry I mean careerists, striving for personal gain above the advancement of knowledge or training of others. But we (or rather funders, PIs and institutions) are directly responsible for this and for just how broken the whole system is. Change is too slow and difficult, perhaps it’s time to burn it all down and build back something that actually works.

    A (rather important) note to all of this. Many people in academia will offer advice but take care what advice/who you listen to. A) Many people are indoctrinated and don’t see any real problems with academia, B) many others will “play the game” and as they benefit from how things work don’t believe in change or that the issues are not that bad and C) far too many conflate experience with evidence. If the evidence does not support an individuals experience then that person should really re-think why that is. D) Those who’ve had bad experiences will struggle to give you a balanced opinion. Of course I have my own viewpoint and biases; I highlight the issues not to complain but because I love academia and what is could and should be.

    The foundations of academia are outdated and broken. It is up to us to fix them. Photo by Mick Haupt on Unsplash
  • Preprints in training; An AI approach

    Preprints in training; An AI approach

    A recent preprint by Rong Ni & Ludo Waltman (1) surveyed authors on their experiences and attitudes to preprinting. A number of interesting points come out of this data but I wanted to focus on one in particular here; training. Figure 3 in the article looks at the sources of learning about preprints and “in a training event” comes very low (less than 10% of respondents). Now I’ve touched upon my thoughts on the current system of “training” grad students and PhD’s before; it’s outdated and frankly, not up to scratch. I’ve also tried for a few years in the places I’ve worked to replace journal clubs with a more useful and structured training program, that would include various elements, one of which is the publication system, how it works, it’s history, where preprints fit in and then a useful journal club where a preprint is reviewed publicly.

    To that end, I have been part of a team in the past where we developed a training course focussed around preprints and publishing in the life and biomedical sciences (2). In hindsight, there are a few things I think that course is missing but it’s a good set of materials that could be used, by anyone, immediately.

    However I’ve also recently been playing around with ChatGPT and thought it might be interesting to see what kind of preprint focussed course AI would design. I began by simply asking ChatGPT to “design a training course on the use of preprints in bioscience”. The initial “course” was rather simple, but not a bad starting point. I refined things with subsequent, more specific, additions and then I compiled things into a slightly more sensible order. Note that I’ve not edited any of the outputs, simply rearranged the order of the sections.

    What was eventually produced could be refined better and is by no means perfect but it’s incredibly interesting to see just what AI can do, given the right inputs — and by right inputs, I was on a train when doing this so it’s somewhat rushed & messy on the human end (I know, if only we could remove those useless sacks of water)!

    So here we go, a (mostly) ChatGPT designed training course for preprints in the biosciences:

    Course Title: “Using Preprints in Bioscience: A Practical Guide”

    Course Overview: Preprints are an increasingly popular way for researchers to share their work quickly and get feedback from the scientific community. However, many bioscience researchers are not yet familiar with preprints and may be hesitant to use them. This course aims to provide an introduction to preprints, their benefits and drawbacks, and how to use them in bioscience research.

    Course Objectives:

    • Understand what preprints are and how they differ from peer-reviewed publications
    • Learn the benefits and drawbacks of using preprints in bioscience research
    • Understand the impact of preprints on scientific communication and publishing
    • Develop the skills necessary to upload and share preprints effectively
    • Learn how to navigate preprint servers and find relevant preprints
    • Understand the ethical considerations of preprints in bioscience research

    Course Format: The course will be delivered through a combination of lectures, case studies, and hands-on exercises. Participants will have the opportunity to upload and share their own preprints, as well as evaluate preprints from other researchers. The course will be offered in-person or online, with interactive elements such as group discussions and Q&A sessions.

    Target Audience: This course is designed for bioscience researchers at all career stages who want to learn about preprints and how to use them effectively. It is also suitable for librarians, journal editors, and other professionals involved in scientific communication and publishing.

    1. History of Peer Review and Publishing in the Life and Biosciences

    • Overview of the history of scientific publishing and peer review in the life and biosciences
    • Key milestones and turning points in the development of scientific publishing and peer review
    • Understanding how the history of peer review and publishing has shaped the current scientific publishing landscape
    • Introduction to the different models of scientific publishing in the life and biosciences, including traditional peer review, open access, and preprints

    This section will provide a historical perspective on scientific publishing and peer review in the life and biosciences, starting with the early scientific societies and the advent of the scientific journal in the 17th century. Participants will learn about key milestones and turning points in the development of scientific publishing and peer review, including the establishment of the first peer-reviewed scientific journal and the introduction of the double-blind peer review process. The section will also explore how the history of peer review and publishing has shaped the current scientific publishing landscape, and how the traditional model of peer review is being challenged by new models of publishing, such as open access and preprints. Participants will gain a deeper understanding of the different models of scientific publishing in the life and biosciences and their advantages and disadvantages.

    2. Introduction to Preprints, their Benefits and Limitations, Finding and Using Preprints, and Considerations when Reading or Citing Preprints

    • Definition of preprints and how they differ from peer-reviewed publications
    • Benefits of preprints including rapid dissemination, increased visibility, and feedback from the scientific community
    • Limitations of preprints including lack of peer review and potential for misinformation
    • Overview of preprint servers and how to find relevant preprints
    • Best practices for reading and citing preprints including evaluating the quality of the preprint and identifying any limitations or uncertainties
    • Importance of acknowledging preprints in publications and understanding their impact on future publication

    This section will provide a comprehensive introduction to preprints in bioscience. Participants will learn what preprints are and how they differ from peer-reviewed publications. The section will also discuss the benefits of preprints, including their rapid dissemination, increased visibility, and feedback from the scientific community, as well as the limitations of preprints, such as their lack of peer review and potential for misinformation. Participants will be introduced to preprint servers, such as bioRxiv and medRxiv, and learn how to find relevant preprints. Additionally, the section will cover best practices for reading and citing preprints, including evaluating the quality of the preprint and identifying any limitations or uncertainties. Participants will also learn about the importance of acknowledging preprints in publications and understanding their impact on future publication. This section will emphasize the importance of critical evaluation when reading or citing preprints, and provide participants with the tools to make informed decisions about using preprints in their research.Top of FormBottom of Form

    3. Preprint Peer Review and Post-Publication Peer Review

    • Overview of preprint peer review and the different models used by preprint servers
    • Advantages and disadvantages of preprint peer review
    • Introduction to post-publication peer review and its role in scientific publishing
    • Comparison between preprint peer review and post-publication peer review
    • Discussion of the strengths and weaknesses of the post-publication peer review movement
    • Examples of successful post-publication peer review initiatives in bioscience

    This section will provide an in-depth analysis of preprint peer review and the different models used by preprint servers. Participants will learn about the advantages and disadvantages of preprint peer review, as well as the various ways that preprint servers are experimenting with new models of peer review. The section will also introduce the concept of post-publication peer review, which has gained popularity in recent years as a way to address some of the limitations of preprint peer review. Participants will learn about the different models of post-publication peer review and the role it can play in scientific publishing. The section will conclude with a discussion of the strengths and weaknesses of the post-publication peer review movement, as well as examples of successful initiatives in bioscience.

    4. Preprint Review Task and Public Sharing on a Preprint Platform

    • Introduction to preprint review and the concept of a preprint journal club
    • Overview of the task to review a preprint and share that review publicly on a preprint platform, such as bioRxiv
    • Best practices for preprint review, including evaluating the scientific soundness of the preprint and identifying any limitations or uncertainties
    • Discussion of the importance of open science and sharing research findings with the wider scientific community
    • Overview of the preprint sharing process on bioRxiv, including guidelines and recommendations for posting a preprint review
    • Tips for participating in and leading a preprint journal club

    This section will provide participants with an opportunity to engage in the preprint review process and share their review publicly on a preprint platform, such as bioRxiv, in the style of a journal club. Participants will learn about the concept of a preprint journal club and its role in promoting open science and sharing research findings with the wider scientific community. The section will provide an overview of the task to review a preprint and share that review publicly on a preprint platform, and provide best practices for preprint review, including evaluating the scientific soundness of the preprint and identifying any limitations or uncertainties. Participants will also gain an understanding of the importance of open science and sharing research findings with the wider scientific community.

    The section will cover the preprint sharing process on bioRxiv, including guidelines and recommendations for posting a preprint review, and provide tips for participating in and leading a preprint journal club. Participants will be encouraged to engage in constructive discussions and provide thoughtful feedback to their peers, fostering a sense of community and collaboration in the scientific publishing process. This section will provide participants with practical experience in the preprint review process and the opportunity to engage in open science practices that can benefit the wider scientific community.

    There you have it, a 4-week course on preprints that could be easily delivered as part of a structured and common training program to all 1st year grad/PhD students. What do you think, did the AI do a good job or are there glaring omissions?

    I might play around more with this when I have some real time to spend on it properly to expand this out to match my idea of what I think we should have for all student scientists.

    (1) https://dapp.orvium.io/deposits/6442f782b2b5580ba561406b/view

    (2) https://fischertad.github.io/Preprints_and_Publishing_in_Life_Biomedical_Sciences/

    AI is really beginning to take off. Photo by Shahadat Rahman on Unsplash
  • One for all and all for one; community building in academia

    One for all and all for one; community building in academia

    Community; a social unit with commonality such as values, identity, norms, place or aim.

    Photo by Hannah Busing on Unsplash

    In academia, a community is often a group with similar interests or goals. These could be societies, universities (or individual departments), similar career stages (NewPI slack, UK & EU pdoc slack), business-run (eLife Ambassadors) or those fighting for change (ASAPbio, preLights). Every department has a responsibility to consider the local community within the department (between labs). In fact, this has been one of the single most important contributing factors to my experience at an institution. Those with good department communities were (by far) the best places to work and did the most collaborative science. Don’t underestimate the power and importance of a good community spirit.

    Why do we need a community?

    Science can often feel community-less as we can work long, unsocial hours on our own and are often siloed off from our peers. Academia forces us to move cities or countries every 3–5 years and can make relationships challenging. Yet without a community, we are alone. Alone to figure things out (such as starting a lab or #firstgen problems). Alone when facing life’s challenges. Alone when celebrating successes. Without community, we are lone wolves. Humans, much like wolves, are pack animals. We need to socialise and we need a community.

    A good community gives members a sense of place and feeling of belonging. It empowers members; be it with knowledge, with strength, with comfort and with support. Perhaps most important of all, a community based around a goal empowers members and gives them the foundation need to generate real, positive, change.

    So where can we look for examples of good communities in academia? For those built around a career stage then NewPI slack or FuturePI slack are good examples. These groups equip members with resources and knowledge so that they can be better placed to pursue their chosen career or start running earlier. For communities focussed on bringing change to academia, ASAPbio is a prime example. Through community building initiatives, ASAPbio has helped to increase preprint adoption across the life sciences. Preprint adoption is one step towards a better scientific environment and hugely benefits early career researchers (ECRs). True change (especially in academia) requires a strong community to foster local changes which build into a larger impact.

    If you build it they will come.

    Sadly not entirely true. Building a successful community requires a lot of work. In fact, many big brands are starting to invest in community builders as they recognise the importance of community. Academia has yet to fully reach this realisation but the alt-ac (those who’ve left academia) community is thriving on Twitter. However, the effort in building a strong community is well worth it. So what do you need to do this?

    Space

    Building a successful community starts with creating a space for the community. There are many options when it comes to choosing a space with some listed here that are used for academic communities.

    Whichever space is chosen, it is not enough to simply provide a space and then do nothing more. If you do this then you’re doing nothing more than collecting people. The space you chose should be appropriate for the type of community you want to organise and engage with. You wouldn’t choose a physical space if you wanted a worldwide community.

    Purpose and aims

    An aimless community wouldn’t meet the definition of community I gave at the start. A community requires clear goals which provide its purpose. Having a clear purpose and aims requires a good understanding of your members (or idealised members).

    The purpose can be anything so instead of writing more here I’ll simply ask some questions you may wish to consider:

    Why have you started this community? Why is this community needed and who is it for? Is the community to be used for networking or fighting for change? What kind of members do you want in the community and why should they join? What will you offer members? How will you ensure the community is a safe and inclusive space?

    I find asking lots of questions like these at the start, before creating the community, can be very helpful.

    Clear rules / guidelines

    You need to define what the rules are within your community. There may be specific rules (for example on who can join) but there are general rules that should exist across all communities. These guidelines keep the community on track and are critical for the health, wellbeing and safety of the community. That said, don’t have a long list of rules, distil them down into a core, simple, list that everyone can understand and abide by.

    Remember, a community isn’t a dictatorship. Community rules should be made by the community and with the aim of inclusivity and safety at their core. The rules should guide and define the community and its members. The rules can also change as you reflect on what works and doesn’t work and you get feedback from members and begin shaping things together.

    Active members

    A community of 40 active members is stronger and better than a community of 4000 inactive members. Communities are all about people but that means you require at least a handful of members who are active. This could be sharing jobs or opportunities, sharing knowledge or resources or answering questions from others. Often, it is better to launch a new community with a small active group so that this essential element is there from the beginning. It’s not about scaling quickly

    One method that can be useful for encouraging an active membership is effective onboarding. Some communities onboard in dedicated groups — for example ASAPbio who provide structured training each year when they onboard a new cohort. Another method for ensuring activity is to regularly use the platform yourself (as the community leader), interact with new members when they join and plan activities or resources that add value.

    Investment and energy

    You, as leader of this cult community need to dedicate time to nurturing and growing it. This could be creating incentives for people to join or participate, policing the rules, recruiting more members to grow the community or all of these at once and more. A successful community requires a frequent influx of

    In essence, be a good leader, show what your community could be and embody the vision. People will always follow a good leader and always abandon a bad one. This doesn’t mean that a community must have a single leader — if you can pass the baton or share the load then the community will be stronger for it. Having active members means that you can delegate or take shared responsibility for the community.

    A good leader amplifies and promotes others while providing the space they need to be awesome. One of the best things about community building is that you get to enrich the lives of others who can then go on to bring about change or create better local environments.

    Measuring success

    Acquisition, conversion and engagement. How many people are discovering the community (acquisition) and of those, how many new members are joining the community (conversion). Though perhaps the most important metric of success is engagement. Within the community, how many active members do you have? How useful are the resources you offer? Afterall, an active group of people can change the world, a silent group couldn’t organise a piss-up in a brewery.

  • Pressure, precarity and paucity; the existence of the postdoc

    Pressure, precarity and paucity; the existence of the postdoc

    Originally posted in 2022 on a different blog, I’ve included this again here along with a recent related episode from the podcast. Sadly, things have not changed enough for me to need to update this in 2025.


    I was recently quoted in a Nature news piece about the paucity of postdocs. However, the article couldn’t dive into what being “valued” actually entails so I thought I’d discuss this more. There’s also a 2025 episode of the Preprints in Motion podcast exploring this more too.

    However, to start with, there is no one-fits-all answer. Some people should be in academia, some people should be in industry, some in medical writing, some in consulting etc… And, to avoid landing myself in any trouble I’d like to spell it out as clearly as I can; I am writing here based on *lots* of conversations with postdocs and PhDs. There will be things I feel too but this post is me writing in an expert capacity. It will also be generalised statements that some people will relate to whilst others may feel I’ve missed things — if that’s the case please add them in the comments or contact me and I can update as needed. Finally, please don’t attack me or label me as being “too negative”. I am simply stating things that are shared by many (who don’t always feel they can speak out) and that is backed by the current evidence.

    What does “being valued” entail?

    Being valued isn’t tied to a single thing and different people will need different things to feel valued by an employer or manager. Indeed, you can feel valued by one but not both of those. However, I would argue that the entire value package in academia, particularly at the postdoc level, is lacking and not at all “competitive”.

    When people think value, one of the first things they associate with that is salary. Thankfully, this is one area most people accept is lacking in academia. And it is an academia problem, not a postdoc problem as every stage (except the inept senior management) is underpaid in academia. Salary also hits differently depending on where you live. In the UK, if you live outside of the golden triangle (London-Oxford-Cambridge) then a postdoc wage may be considered pretty good, especially when compared to working class wages that largely exist in the North. However, within that golden triangle the postdoc wage doesn’t stretch; as I’ve already written about elsewhere. The other component to salary is that we’ve spent 8+ years in education during which we don’t pay into our pensions, we don’t earn a real wage and we’re delaying life. The postdoc period then comes with a poor wage (undergraduates can get entry level graduate jobs at not much lower a starting salary) and often involves lots of moving which is expensive and further complicates settling down and doing the “adult” things. The average postdoc is paid around £6,000 less than the UK average wage¹. I always find it sad that society values science so little that governments are not incentivised to spend more on STEM, especially after the vital role bioscience played in the pandemic.

    Value also comes in the form of recognition and reward for our efforts. This is one area in which academia fails spectacularly. Academics views papers (and first author papers at that) as a suitable reward. Papers are also the things used to judge us, which makes them a very twisted reward indeed. A reward would be a pay increase based on exceptional work (non-paper based) or even just a verbal recognition of a person’s efforts in departmental meetings. But the majority of tasks in academia are not recognised as they don’t lead to a paper; so how could you expect people to feel valued when you ignore the majority of their effort? The bigger problem here is that this disproportionately impacts women and minorities who are the ones who perform the bulk of community and unseen tasks. This doesn’t just come from management either, other postdocs seem remarkably keen to bring each other down — careerism is the absolute ruin of science (but more on that another time).

    And on that note, the feeling of being respected is also intricately linked with feeling valued. Postdocs get distinctly less respect than a PI even though they may be no less experienced. It’s almost as if that step of fellowship/lectureship conveys “expert” and the rest of the postdoc pool are what? Not experts? This isn’t just academia either, in the “real world” postdocs are still not held in the same regard as a freshly minted PI. I think postdocs are often seen as a temporary pair of hands to do the bidding of the PI and little more². When papers are published, if you’re lucky enough to get media attention, it is the PI who almost exclusively gets that attention and recognition. It is the PI who is invited to conferences and seminars to talk about the work. Sure postdocs can present at a conference but we’re rarely actually invited to speak as a PI is. In the long term, it is the PI who benefits more than anyone else from those publications. Most postdocs will leave academia where papers really don’t matter that much, besides showing that you can write and as evidence of completing a project. Yet it is the postdocs who run themselves into the ground working long (unpaid) hours and weekends to get the papers done in the 3 year limits of grants.

    Recognition and respect also interect with the wider support for postdocs. Traditionally, the postdoc period was supposed to be a training position although this is not only now outdated but I’m not sure there has ever been acceptable training. I’ve never worked at a place that has any tangible training plan in place and whenever I suggest such things I face remarkable resistence to the idea. To quote from a brilliant piece; “in labs throughout the country, including those on the cutting edge of research, mentorship practices still take their cue from the earliest European colleges, where a single, experienced, sage-like scholar served as mentor to a group of excited and engaged students. This literally medieval basis for mentorship in science is so entrenched that most research training programs at the graduate and postdoctoral levels take a hands-off approach to mentoring, leading to a wide variety of mentorship experiences for trainees, even within the same departments and programs”. There is a distinct lack of support for career development (something I’ve definitely experienced myself) with university level offerings varying hugely in quality and utility. PIs are sorely under-equipped for supporting people in positions outside of academia, though far too many are also incapable of supporting academic careers. The whole system needs to rethink the role of postdoc and to provide structured training from the university and PI-level for skills that will land non-academic careers. I honestly believe that the biggest barrier here is simply that if we acknowledge that a postdoc is not actually a training position then the salary would need to increase significantly.

    Work-life balance. Nobody in academia should need me to write any more than that as we’re all well aware of how non-existent that can be. If an employer values you then they want you to look after yourself and take time away from work. Hell, even if they don’t care about you, if they care about your work output they will want the same thing. Academia fails miserably here and it does so under the gaslighting guise of “passion”. My PhD PI and current PI both frequently tell me to take holidays and are both hugely supportive. However, I’ve been in positions where I’ve been horribly overworked and I still witness a toxic culture of constant working. I really struggle with understanding where the pressure in academia comes from. The research is not so essential that it can justify the pressure. Academia moves slowly so the career progression can’t really justify the pressure either. Maybe it’s the short contracts and messed up way we “value” and judge success. Having a life in conjunction with being successful at work should be the norm but within academia this is absolutely not the case.

    One of the things not directly mentioned in the Nature article is the improving and expanding biotech industry. This is discussed well across Twitter and in an excellent newsletter by Elliot Hershberg. The impact of this is that there is a genuine “competitive” environment that can pay postdocs better and give postdocs an actual careeer trajectory based on competence and ability rather than pure luck. The biotech industry has it’s own problems and is still largely concentrated in specific areas (Bay area in SF or around Cambridge in the UK as examples) which are very expensive areas to live in. But it represents a challenge to academia strong enough to be pulling away even senior academics and academia, as slow as it is, certainly won’t adapt in time to prevent the best leaving.

    It’s really not that difficult. Postdocs want 1) better environment and pay commenserate with training, experience and education required (to have an acceptable quality of life compared to those who didn’t enter academia) 2) an environment of respect for postdocs which includes support for our careers (in or out of academia) and better appreciation of our efforts and 3) recognition for our work, beyond just papers.

    There are some brilliant PIs out there who contradict everything I’ve said here and I don’t want this post to detract from them. But we need to aim at the lowest level so that nobody is lost or hurt by inadequate systems.

    Who’s responsibility is this? Well, it’s the funders, the uni’s and the PI’s. Postdocs are never going to come together enough to stand up for each other — there’s too much competition and it’s too precarious a position. Funders need to take an active role in combating the abuses and toxicty. It is ultimately their responsibility and it is a responsibility that they have negleted for too long. Universities are businesses. They will always protect their bottom line and reputation — we’ve seen that time and time again even in extreme situations. So we cannot rely on university management to police themselves. But the government/funders could create an independent body to oversee academic misconduct and complaints of abuse. There is no shortage of brilliant people with brilliant ideas — those “field leading” PIs are replacable and noone should be given a job for life. That kind of security leads to complacency and a lack of accountability — and abuse. My biggest problem is always that, although systemic change is unlikely, there are no excuses for not making local environments better. This is the direct responsibility of PI’s (at a lab level) and heads of department (at a larger local level). I utterly hate seeing failures locally.

    Ultimately, it is clear that the system is broken. People are crying out with stories illustrating just how bad things have gotten. People are also screaming out with solutions which must (& does) often feel like screaming into the void. As long as we continue to allow luck to be the main driver in academic careers and success we will not break free of the current system. If people are willing to accept that change is needed and willing to listen, instead of attack, then we can all work together to create a better future in academia.


    1. The average UK wage is ~£38,131. The average postdoc wage is ~£32,000 (this isn’t as easy to accurately calculate as I thought it’d be but is based on “averages” from a variety of sources).
    2. This is a great quote from an eLife piece; “If you don’t have your own lab, if you’re not producing last-author papers and getting grants, you are taken less seriously as a scientist. When people discuss their posters with me at conferences, they may look around a bit more, anxious not to miss the opportunity to talk to a ‘big fish’.”

    Photo by Jackson Simmer on Unsplash

  • In defence of preprints

    In defence of preprints

    You may have come across this remarkably ill-informed blog post. Given that this podcast advocates for preprints in the life sciences, we are compelled to issue a direct response to the blog post. Leadership does not come from remaining silent.

    You can listen to the episode discussing the blog post here, below is a clearer, evidence backed, rebuttal.

    Preprints serve scientists and the pro-science movements

    The click-bait title of the blog post is designed to be provocative. However, it also immediately highlights the authors’ lack of understanding of preprints. Preprints are most beneficial to early career researchers who benefit from being able to document and evidence their work when they are ready for it to be shared rather than waiting 6-12 months in an opaque peer review process. During public health emergencies, preprints quite literally save lives, with around 40% of the early research being shared first as a preprint; this occurred during the COVID-19 pandemic and recent mpox outbreak. This is far from serving an “anti-science agenda” as claimed by the blog post author.

    The benefits of preprints are well known by now so I won’t retread that ground here. Instead, I wish to focus on the blog post and it’s multitude of problems. Before I do however, it is worth highlighting that the author left academia to create a new journal (stacks journal) and has a huge bias in favour of traditional peer-review. Surprisingly for someone involved in the science communication space, the author comes across highly uninformed on open science and the history of publishing/peer review. This is, perhaps, due to the manner in which the blog post was written, rather than a genuine lack of understanding but is concerning regardless. The blog post fails to provide any argument for that preprints are “serv[ing] the anti-science agenda” and instead is a few misguided and incorrect opinions. There are accusations and inferences that are made without any evidence or support.

    The blog post presents a highly out-dated view that looks backwards rather than to the current moment, never mind the future.

    Anyone can post anything as a preprint

    The author claims that “nearly anyone can post an official-looking preprint” and that “preprints are like blog posts”. Most reputable preprint servers have basic screening processes in place. This reduces the sharing of pseudoscience, poor quality studies and other non-science outputs. This also identifies controversial work that may require a closer examination from a process such as peer review. Preprints are most often (although not always, depending on the server) full research articles with materials, methods and results. One point worth highlighting is that authors are often still overly cautious and regularly fail to include datasets or supplementary files, something I hope will change sooner rather than later. However, these screening processes act to prevent “anyone posting anything” onto preprint servers.

    This may be a good point to define “preprint” as the blog post author does not seem to understand what preprints are. A preprint is a manuscript shared online by the authors prior to journal-organised peer review. They are citable, have a DOI, are permanent records, may be peer reviewed and are shared when authors believe that the work is ready for public consumption. They are very, very, rarely preliminary work with the majority of preprints being complete scientific articles.

    The editorial standards of the blog post that this author has written fail even the most basic standards for ethics and integrity in scientific writing. If this is the standard for a blog post then the author makes his own point; preprints really are nothing like blog posts (they’re vastly superior).

    Preprints are comparable to the published literature

    A central tenant to the authors argument is that preprints are poor quality. This is a persistent myth and a common argument from people who do not understand preprints. However, this is just a myth and there is a growing body of evidence addressing this point. Indeed, I have authored such work myself – work that other Scholarly Kitchen (SK) associates found acceptable to publicly attack and even go so far as to email the editor and journalists telling them it was fatally flawed rubbish (it actually stood up to intense scrutiny but this highlights some personal bias I have against SK and reveals how some of those involved behave in relation to evidence they don’t like)!

    This growing body of evidence, from multiple, independent groups using a variety of different methods and approaches consistently draws the same conclusion; that preprints are comparable to the peer-review literature. When preprints do undergo changes, these are most often limited. In simple terms, peer review does indeed help to make an article better, but only in a very limited manner. Additionally, that improvement is around 5% “better”. For a system that can cost $4 billion globally and delay scientific progress by an average of 6 months, 5% may not be worthwhile.

    preprints are comparable to the peer-reviewed literature

    Given this, it would be egregiously anti-scientific to suggest that all, or even most, preprints are low quality or nothing more than glorified “blog posts…[on] social media platforms”. One valid question does remain; what about the 30% of preprints that are not eventually published? I have an active project on-going to address this question. However, any preprint expert will likely answer this in the same manner for now. There are many reasons why a preprint may not be published. This could be that the preprint is negative data or small, one-off datasets that are difficult to publish and often not worthwhile for authors, despite their scientific value. Preprints may be the final intended destination of the work (we interviewed one such author previously) or the authors may not be able to afford APCs at journals. None of those reasons are that the preprints are low quality or unreliable anti-science. I will agree that there are low-quality, unreliable preprints, such as this one (which was withdrawn within 48 hours – quicker than any journal would act). However, this equally applies to peer-reviewed literature, which is in many ways considerably more dangerous.

    The author also suggests that reporting on preprints is “alarming”. However, data shows that journalists have adapted well to preprints and generally report on preprints in an ethical, appropriate, manner and with greater standards than the Scholarly Kitchen possesses.

    Peer review does not protect the scientific literature nor is it designed to

    “For decades, scientists have relied on peer review to ensure scientific knowledge is built on a foundation of rigor and credibility. However, preprints are adding to the crumbling of that foundation”. The very first line of the blog posts asserts that peer review, and peer review alone, is providing the foundation of rigour and credibility to science. I’m not too sure what the author thinks of all those scientific breakthroughs that occurred prior to the 1970’s, before peer review was standard, but one can infer from this blog post that we should maybe not trust papers such as the one describing the DNA double helix from Watson & Crick or much of Einsteins work (not peer-reviewed, much to the surprise of many who haven’t studied the history of peer review). Peer review has only been commonplace since the 1970s. However, more fundamentally, this highlights a common misunderstanding of peer review; that it is meant to protect the literature in some way by acting as a quality control (QC) step.

    One common problem that is fuelling the anti-science agenda is the veneer of trust that peer-review (falsely) offers.

    Peer review is not designed to detect fraud, a fundamental element for protecting the literature as part of a QC. Indeed, there have been some very high profile examples of fraud in some of the biggest and most “impactful” journals, including the Surgisphere scandal. This work was published, not because peer review failed, but because peer review was never designed to detect fraud to begin with. This is further illustrated through the work of expert sleuths (forensic scientometrics) who regularly detect manipulated data, spliced gels and other abnormalities that indicate a particular result is less than trustworthy.

    Peer review is also consistently poor at identifying gross defects in papers, as evidenced by numerous studies. This is probably not surprising given the almost complete lack of training in assessing other people’s work or in identifying defects. Add the fact that reviewers are overburdened, with less time to dedicate to review activities and the likelihood of even a good, detailed reviewer missing gross defects increases. Peer reviewers are not forensically examining manuscripts, paid or not. Some evidence suggests that paying peer reviewers slightly increases turnaround times but there is no evidence that paid reviewers spend longer assessing the manuscript or put more effort into the process. Indeed, the evidence that does exist is that the paid-for reviews are not significantly different to the non-paid for reviews.

    Together, these effectively negate the protective aspect of quality control that many people falsely believe that peer review is designed to achieve. This does not render peer review pointless or something to avoid entirely. Peer review does help authors improve their work (as discussed), but collectively we need to better explain the limitations of peer review and rely less on this as the lone bastion of rigour in science.

    One common problem that is fuelling the anti-science agenda is the veneer of trust that peer-review (falsely) offers. The author makes no real effort to illuminate this point. It is due to this mis-use and mis-understanding of peer review that there does need to be a much greatly improved system of trust for research.

    A constellation of trust signals

    The authors’ suggestion that trust should come from a deeply flawed peer review process is short-sighted, failing to account for the already changing landscape. What is needed are a range of trust signals that, when taken together can provide a sense of trust in an individual research output. Peer review may very well be one of those signals but it should not be the only one. Ideally, these trust signals would be both static (reflecting fundamental qualities of the work) and dynamic (updating as new evaluations emerge). They would be applicable to a broad range of outputs and extendable to researcher assessment. A system of diverse and dynamic trust signals that accompany preprints, would offer a multifaceted evaluation framework capable of evolving over time. In the 21st century, science publishing can no longer communicate findings solely to other scientists. Accordingly, the metrics associated with publishing must reflect reality, whilst encouraging best practices and rigorous science.

    Preprint feedback occurs privately

    One piece of evidence the blog post does cite is a study that suggests only 7% of bioRxiv and medRxiv preprints receive comments. As the author is cherry-picking their citations and evidence, what they fail to communicate in the blog post is that most feedback occurs privately. More recent data suggests that 60% of surveyed authors received feedback on their preprint, rising to 70% of authors who actively sought feedback. I would much prefer public feedback, but that’s not the issue in the blog post. The blog also fails to account for any form of non-traditional feedback, such as would occur through social media channels or preprint review services such as PREreview or preLights. It’s a confusing decision to focus on preprint commenting, a feature well known to be underutilised. On bioRxiv/medRxiv, there is a panel that brings together reviews and context for each preprint (shown below). This panel is also a great example of how trust signals could be surfaced and presented to readers and also brings together wider, non-traditional, mechanisms of feedback.

    The bioRxiv/medRxiv Reviews and Context panel present alongside all posted preprints

    Scientific publishing is not sustainable in its current form. The system requires significant changes that must occur alongside changes to research(er) assessment, hiring and promotion practices and broader science communication. Academia itself also needs to change, moving away from narrow training of new scientists whilst offering no academic positions. The entire system is breaking and what is very much not needed are more poorly informed efforts that distract from the immense amount of work needed to create a better future for academia and for science.

    Disclaimer

    I have chosen to match the tone of the original blog post to provide the most appropriate response I can rather than a dry, evidence-only, article. I am not attacking the original author but when somebody creates something they claim to be built on ethics and integrity, it is reasonable to highlight the hypocrisy on display. Otherwise, I have simply used the authors own words to frame my rebuttal. It is difficult to combat a blog post with such little scientific standards without my response positively dripping in sarcasm. Whilst I would hope the original author would reflect, I have not seen any evidence to this effect and it is most likely a lost cause. Indeed, this blog post has raised significant concerns around the authors Stacks Journal and the potential lack of editorial and scientific standards employed. Hopefully, the blog post has not caused any real damage and, based on some of the social media responses, it seems clear that people see the author as biased and poorly-informed. A similar article that attempts to initiate a similar discussion around peer review can be found here – this is a much more informed article for those interested.

    Photo by J A Coates, “Sintra”