Over the last year, conversations about trust signals have increased. However, there remain many outstanding questions, various levels of implementation or experiments, and a lack of clarity on the purpose of such signals.
Historically, the credibility of a research article is judged through the presence of peer review and weak proxies linked to publication venue (such as journal name and impact factors). These proxies fail to capture the nuances of research quality and trustworthiness whilst peer review is straining under the pressure of too many submissions and too few reviewers. Worse still, LLM use is exposing the limitations of peer review as a quality control mechanism.
Peer review is a 20th century solution that is no longer fit for purpose, if it every truly was. In the 21st century, trust is a much broader question with a more tools available than ever before to address this.
What is a trust signal?
There is not yet a standard definition of trust signal. However, I believe that a trust signal must accomplish all of the following:
- Not be used as the sole marker or prioritised above others
- Not be controlled by publishers or authors (i.e. free from manipulation)
- Must convey a discrete element of trustworthiness
- Directly related to the research output
- Specific to the associated research output, non-generalisable
- Not designed to punish authors
- Account for post-publication reception and usage
- Are context dependent where necessary
- Designed to help readers in determining how trustworthy an output is
Taking these points together, a potential definition of a trust signal would thus be: one of multiple, unique signals that are specific and directly related to a given research output that are not controlled by publishers or authors and account for post-publication usage and reception. These are designed specifically for readers of a given research output to help them in determining how trustworthy the output is.
In addition to this, it is vital that, where necessary, trust signals are context dependent. For example, peer review could be a trust signal but its presence or absence is of little value. Instead, the context of the review (positive or negative) is much more valuable to the reader. Surfacing this context is also more valuable than simply only providing full peer review reports – which few will actually read. Similarly, the number of citations is not informative of how an article is actually being used. Scite.ai provide context to these citations (positive, negative, neutral), the counts of which are more informative as a trust signal.
Some current efforts
VeriXiv, a preprint server primarily/solely used the Gates Foundation utilises a badging system. These badges summarise a list of checks that the platform performs on each posted preprint. These checks encompass; authorship checks, ethics checks, integrity checks, open research checks and peer review status.
Signals is a company that utilises the metadata and other checks to evaluate an article. Signals provide badges that label an article as “looks good”, “caution” or “alert”. They also perform portfolio analyses and article analysis. Signals is strongly focussed on publishers and integrating with publisher workflows, rather than readers.
TrustMarc is an initiative developing a standards framework to issue certificates. These focus on endorsements (peer review) and provenance. This effort is still in the early, proof of concept, stages.
Trust Seal is a community led effort initiated by a coalition of scientific societies. This is designed to align, recognise, and support best practices across publishing.
Are current efforts good enough?
This might seem like a direct, even harsh, question but it’s the one we should be asking. Most of the current efforts are still in their preliminary stages, and so cannot be judged too heavily. Despite this, a few common criticisms can already be levelled.
Firstly, most of these efforts are focussed on publishers and not on helping readers to identify how trustworthy an output is. This fundamentally fails the definition of a trust signal above. By not focussing on helping readers, these efforts are simply new tools for the publishing industry. Many of the efforts are also being run by, or involve, publishers which leaves them open to manipulation. This raises questions about the use of the term “trust signals” and whether or not these are genuine efforts to help improve research quality and help readers or a to give the marketing departments at publishers something to shout about.
Secondly, there appears to have been little thought into how trust signals might be implemented and the user experience. Badges are of limited value when a large list of signals are required. This is most obviously seen with VeriXiv which perform over 20 individual checks but then condenses the results into 5 categories. Readers only see if a preprint has passed or failed all of the checks within a category. Whilst this is a start, too much information remains hidden from readers unnecessarily. In addition, the signals are all pre-publication, with the exception of peer review. This means that readers are denied important post-publication context and discussion which is often where significant issues, if they exist, are raised.
Thirdly, peer review still dominates these efforts where there has been an implementation. If peer review is still going to be highlighted as the dominant trust signal, then ultimately these efforts will be futile. This also fails to meet the definition of a trust signal outlined above. A key issue arises anytime any individual signal is placed above the others. By prioritising peer review, these efforts are maintaining the status quo. In the case of VeriXiv, the Gates Foundation is a supporter of the PRC model and so it is unsurprising that they place such an emphasis on peer review1.
But there are also positives too. VeriXiv does go beyond peer review and, despite the implementation, these grouped badges are still better than nothing and the interface is clean. This approach may be enough to reduce some concerns around preprints. The efforts focussed on creating shared frameworks and standards are also tackling an essential element to trust signals. It is logical that publishers would largely be taking the lead on those given the role of the journal as a gatekeeper of “quality” – or at least the perception that they are this2.
Barriers to trust signal adoption
It would be naive to assume that trust signals will be readily adopted by the research community. Indeed, it is much easier for the publishers to adopt some of these signals as they already have the infrastructure and could benefit from doing so. So what are some key barriers to potential trust signals?
Researchers historically resist public criticism of their work. Trust signals that are not controllable by researchers will face an uphill battle with researchers themselves. Researchers are likely to push back against anything which could label their work as untrustworthy – even when their work is poor quality and deserving of such a label. This is why it is key to avoid signals that punish researchers. This is also where careful wording and communication may help alleviate some of the potential concerns. Indeed, this is is why framing things as trustworthiness that is on a scale, rather than binary, is potentially quite important. This might help shift the conversation away from absolutes whilst encouraging better practices and behaviours.
Peer review is deeply embedded in the psyche of researchers. A significant barrier to the widening of trust signals is the current incumbent, peer review. Despite only being commonplace since the 1970s, peer review as a form of quality control is deeply embedded in research culture. Indeed, even new model which claim to be innovative still place peer review at the centre of trust. PRC is a clear example of a co-ordinated effort to damage preprinting and limit innovation in trust. The establishment has a history of stifling innovation and hijacking the open movement and this is arguably already happening in the case of trust signals. Both the PRC model and some of the efforts above fall into this category.
Too much competition, not enough cooperation. A key feature of much of the open science space are too many competing efforts. If those advocating for change can’t agree on a path forwards, then why should any researcher buy in to a given route? The competition and lack of focus is leading to increased confusion in researchers and this further limits change. Without cooperation, standards are difficult to develop and adoption is slow. There are already competing efforts to develop standards and frameworks; TrustMarc, Trust Seal and NISO are all concurrently perusing this.
Noise v signal. A key problem with introducing more signals is that this increases noise, making it more difficult to determine valid signals. Ultimately, this may just be a UX issue that is solvable with a sensible implementation. My belief is that the data underlying each trust signal should be provided and available for those who wish to look deeper. For example, peer review reports should be openly available and linked, or data on the context of citations should be available.
Genuine or misguided?
On the whole, I believe that these efforts come from a good place and are genuine attempts at improving the quality of research. However, questions remain as to whether these are designed to actually help readers or publishers with most efforts currently falling to the latter.
The focus on peer review is definitely misguided and a huge wasted opportunity. Unfortunately, there is a concerted effort to maintain the status quo that appears to be infecting every part of open science. This, along with the strong involvement of publishers, raises serious concerns; I’d like to see much greater involvement of researchers and less of the same old faces. If current trust signals are not going to help readers then I do feel much of these efforts are misguided. In a world where academia has a more diverse readership than ever before, it is our duty to communicate effectively, and trustworthiness is a vital component to that.
1 Or perhaps this is an overly cynical viewpoint and the Gates Foundation is simply providing the current primary signal (or this is a tech/implementation decision by F1000, which is owned by Taylor & Francis). If it is an implementation decision, then this feels like a missed opportunity to do more. Alternatively, it could be indications of a publisher ensuring that peer review remains the dominant signal of trust.
2 I say perception as there is plenty of evidence of publishers ignoring their own policies to publish headline grabbing articles and the evidence on preprint quality also raises questions about the role of publishers v community in setting standards
8th July 2026 update: Amended some aspects to make it clearer that F1000 own and run VeriXiv, and that Gates are the primarily/only users.
Leave a Reply