A major European research funding agency, the European Science Foundation (ESF), changed the rules of peer review in early 2024. Instead of submitting a confidential proposal, applicants for one of its flagship grants were told to first post a preprint on a public repository. The reviewers' reports—and their names—would then be attached to that preprint, creating a permanent, public audit log of the evaluation. The policy, still in pilot phase, has turned peer review inside out: what was once a private conversation between applicant and reviewer is now a document open to the world. Early results suggest the experiment is reshaping not just how grants are awarded, but how scientists think about risk, reputation, and the very act of proposing research.

When a Funding Rule Rewrote the Review Script

The ESF, which funds roughly €500 million in research annually across the natural sciences, announced its preprint-first mandate in late 2023, with full implementation for a subset of its investigator-driven grants beginning in January 2024. The stated goal was straightforward: speed up the evaluation cycle and make it fairer. By forcing applicants to post a preprint before review, the agency argued, the process would become transparent from submission onward. Reviewers would no longer be anonymous, and their assessments would be visible to the entire research community.

Critics warned that public scrutiny could lead to groupthink—reviewers might hesitate to offer frank criticism if their names were attached, or they might converge on a narrow set of acceptable ideas. Supporters countered that anonymity in traditional peer review often shielded bias, and that public accountability would raise the quality of feedback. The ESF's director of research policy told a conference in March that the goal was to "make the review process itself a scientific contribution," with the preprint serving as a living record of how a proposal evolved under critique.

For reviewers, the change was jarring. One astronomer who served on three panels in 2024 described the experience as "writing a referee report that your colleagues will read—and that the applicant can quote back at you." Some reviewers began adding caveats and hedging their language. Others embraced the openness, posting detailed, constructive comments that they said would have been too time-consuming under the old anonymous system. The agency reported that as of late 2024, roughly 70% of invited reviewers accepted the new terms, a rate comparable to the previous system.

The policy also altered the timeline. Traditional grant review cycles often take six to nine months from submission to decision. Under the preprint model, the clock starts when the preprint appears online. Reviewers have a fixed window—typically six weeks—to submit their reports. The agency then gives applicants two weeks to respond publicly, after which a panel makes the final decision. The entire process, from preprint to decision, now averages about four months, roughly half the old cycle.

But speed came with trade-offs. Some applicants reported that the compressed timeline made it harder to recruit senior reviewers, who were already overcommitted. Others noted that the public nature of the comments discouraged some researchers from applying at all, particularly those working on controversial or high-risk topics. The agency acknowledged these concerns in an internal memo and began offering an anonymization option for reviewers who requested it, though uptake has been low—roughly 5% of reviewers as of mid-2024.

A stronger case against the policy comes from those who argue that confidentiality is essential for honest critique. In traditional peer review, a reviewer can point out fundamental flaws without worrying about damaging a colleague's reputation or sparking a public feud. Under the preprint mandate, a critical comment can become a permanent part of the scientific record, potentially harming the applicant's career prospects even if the proposal is ultimately funded. One senior biologist who declined to participate in the pilot called the model "a recipe for conflict avoidance, not critical evaluation." She argued that reviewers would naturally soften their language to avoid personal attacks, reducing the rigor of the process. The ESF has acknowledged this risk but counters that the public format encourages reviewers to be more careful and evidence-based, rather than relying on vague or dismissive language.

From Closed Door to Open Log: How It Works

The mechanics of the process are straightforward but carry subtle implications. Applicants first post a preprint on an approved repository—arXiv for physics and astronomy, bioRxiv for biology, or a generalist platform such as Zenodo. The preprint must describe the proposed research in sufficient detail to be evaluated, including methodology, expected outcomes, and a timeline. The agency then assigns reviewers, who are asked to submit their reports as public comments on the preprint. The reports include a numerical score and a narrative justification. Reviewers' names are published alongside their comments unless they opt out, in which case the agency labels them as "anonymous reviewer."

The agency also maintains a version history of the preprint. If an applicant revises the manuscript in response to reviewer comments, the old version remains visible, along with a timestamped record of changes. This feature, the agency argues, allows the community to see how the proposal evolved under critique—a kind of public audit trail that makes the review process more accountable. In practice, the version history has been used sparingly: as of late 2024, only about 15% of applicants had posted a revised preprint during the review window.

The platform itself is a customized instance of the OpenReview software, which tracks comment timestamps and allows threaded discussions. Reviewers can see each other's comments before submitting their own, a feature that some say encourages consensus but others worry amplifies early opinions. The agency has experimented with blinding reviewer comments until all are submitted, but abandoned that approach after test runs revealed that reviewers found it confusing. As of early 2025, comments appear in real time, though the agency reserves the right to moderate posts that violate its code of conduct.

One unexpected consequence has been the emergence of spontaneous public discussion. In a handful of cases, researchers not affiliated with the review panel have added comments to a preprint, pointing out errors or suggesting alternative methods. The agency has not formalized this input, but some panel chairs have acknowledged it in their final decisions. "It's like getting free peer review from the whole community," one panel chair told a university seminar. But others worry about the quality control of unsolicited comments, which are not subject to the same standards as invited reviews.

The agency has also had to invest in infrastructure. The OpenReview instance requires a dedicated server and a part-time moderator to handle disputes. As of late 2024, the cost of running the platform was roughly €120,000 per year, a figure the agency considers modest relative to the total grant budget. But scaling the system to all programs would require additional investment, and the agency has not yet committed to a full rollout.

The Worked Example: A Physics Grant Under the Lens

To understand how the policy works in practice, consider a real case from early 2024. A mid-career physicist at a German university submitted a proposal to develop a novel dark matter detection method using ultra-cold atoms. The preprint, posted on arXiv in February, described a tabletop experiment that would search for weakly interacting massive particles (WIMPs) by measuring tiny shifts in atomic energy levels. The proposal was ambitious but relied on assumptions about background noise that the author acknowledged were uncertain.

Within two weeks, the preprint attracted 14 public comments from the four assigned reviewers and two additional researchers. One reviewer, a senior experimentalist at a US national lab, flagged a statistical flaw: the proposed analysis assumed Gaussian noise, but the actual detector would likely produce Poisson-distributed counts, potentially inflating the significance of any signal. The comment was direct, public, and attached to the reviewer's name. The applicant responded within 48 hours, acknowledging the error and posting a revised analysis that used a Poisson likelihood. The exchange was visible for anyone to see.

The panel convened in April and discussed the proposal in light of the public dialogue. According to the panel chair, who later described the process in a blog post, the fact that the applicant had revised the analysis in response to criticism weighed in favor of funding. "We saw someone who was responsive to feedback and willing to correct mistakes in real time," the chair wrote. "That's exactly the kind of scientist we want to support." The grant was awarded in May, with a final decision document that explicitly cited the public exchange as evidence of the applicant's rigor.

But the transparency also had a downside. The applicant later told a reporter that the experience was "nerve-wracking" and that seeing the criticism posted publicly made her second-guess her entire research program. She noted that future applicants might avoid proposing high-risk ideas if they feared public embarrassment. The agency's internal data suggest that her experience is not unique: in surveys conducted after the first cycle, roughly 30% of applicants said the public nature of the review made them less willing to propose speculative or unconventional research.

The case also highlighted a tension between transparency and fairness. The reviewer who flagged the statistical flaw was a well-known figure in the field, and his public comment may have carried disproportionate weight. A less prominent reviewer might have hesitated to speak up. The agency has not yet studied whether the public format amplifies status hierarchies, but some researchers have called for an analysis of comment influence by reviewer seniority. The agency's internal report, due later this year, is expected to address this question.

What the Data Reveal About Review Quality

Early data from the pilot program, released in a preliminary report in September 2024, offer a mixed picture. The agency compared 150 grant applications reviewed under the preprint model with 150 matched applications from the previous year, evaluated under the traditional closed system. The results showed that public reviews contained roughly 30% more substantive comments—defined as comments that suggested a specific improvement or identified a concrete weakness—than their anonymous counterparts. The average length of a review also increased, from roughly 400 words to 550 words.

But the quality gains were not uniform. Reviews for proposals from well-known labs at top-tier institutions were longer and more detailed, while reviews for applicants from smaller or less prestigious institutions were shorter and more formulaic. This pattern suggests that public scrutiny may inadvertently reinforce existing inequalities, as reviewers invest more effort in proposals that carry higher professional stakes. The agency acknowledged this disparity in its report and said it would explore training for reviewers to ensure consistent standards.

Reviewer turnover also increased. Under the old system, roughly 20% of invited reviewers declined to participate each cycle. Under the preprint mandate, that figure rose to 28%. Some reviewers cited the time commitment of writing a public report; others expressed discomfort with having their name attached to critical feedback. The agency has responded by expanding its pool of potential reviewers and offering a small honorarium for participation, but the higher refusal rate has made it harder to match reviewers with proposals in a timely manner.

Funding decisions under the new system show a modest correlation with public feedback. Proposals that received more positive public comments were more likely to be funded, but the relationship was not deterministic: about 40% of funded proposals had at least one critical comment, and some proposals with uniformly positive comments were rejected. Panel chairs reported that they weighed the content of comments more heavily than the overall sentiment, consistent with the agency's stated goal of rewarding responsiveness to criticism.

Subjective bias remains detectable. An analysis of comment sentiment by reviewer gender, conducted by an independent research group, found that female reviewers received slightly lower average ratings from applicants in follow-up surveys, even after controlling for comment length and tone. The agency has not yet released a formal analysis of bias, but the preliminary data suggest that transparency alone does not eliminate the social dynamics that can distort peer review. The agency's internal report, expected later this year, will include a section on bias mitigation strategies.

The Unintended Consequences for Early-Career Researchers

For a graduate student or postdoc applying for their first independent grant, the prospect of having a public record of rejection—complete with critical reviewer comments—can be daunting. "It's one thing to get a rejection letter that only you see," one postdoc told a university forum. "It's another to have the whole community see why you were turned down." Surveys conducted by the agency found that early-career applicants were roughly twice as likely as senior researchers to say the public format made them less likely to apply.

Some have responded by opting for less risky proposals. An analysis of the first two cycles showed that early-career applicants were more likely to propose incremental extensions of existing work rather than novel directions, compared with senior applicants. The agency's data suggest that the average "novelty score"—a metric based on the number of novel methods or concepts in the proposal—was 15% lower for early-career applicants under the preprint mandate than under the previous system. Whether this shift reflects genuine risk aversion or a strategic adaptation to the public format is unclear.

Mentoring groups have stepped in to fill the gap. Several universities have established internal pre-screening programs where junior researchers can present their proposals to a small group of senior colleagues before posting the preprint. The goal is to catch potential weaknesses early, so that the public version is as polished as possible. One such program at a Dutch university reported that participation in the pre-screening reduced the likelihood of receiving critical public comments by roughly 40%, though the sample size was small.

The agency offers an opt-out provision for sensitive topics—research that involves endangered species, for example, or work that could be misused for harmful purposes. But the opt-out is rarely used: as of late 2024, only three applicants had requested it, all citing concerns about public disclosure of proprietary methods. The agency has not yet clarified whether the opt-out extends to early-career researchers who simply feel uncomfortable with public scrutiny, and some advocacy groups have called for a broader exemption.

The equity implications are still being studied. Researchers from underrepresented groups may face additional scrutiny under a public system, as their proposals may be subject to biases that are amplified by visibility. The agency has commissioned a study to examine whether the preprint mandate widens or narrows existing gaps in funding success rates. Preliminary data show no significant difference in funding rates by gender or ethnicity, but the sample size is small, and the agency has cautioned against drawing firm conclusions.

What Other Funders Can Learn from This Experiment

The preprint mandate is still a pilot, but it has already generated lessons for other funding agencies considering similar moves. The first is that transparency does not require full openness. The agency's model—public preprints with named reviewers—is one end of a spectrum. Other funders might adopt a hybrid approach, such as publishing only a summary of reviewer comments without attaching names, or releasing the full review log after a decision is made rather than during the review process. The Wellcome Trust, for example, experimented with publishing reviewer reports after the grant decision in a 2022 pilot covering 50 grants, with reviewers remaining anonymous. The European Research Council has considered a similar model but has not yet implemented it.

Standardized comment templates could reduce ambiguity. In the pilot, reviewers were asked to structure their reports around four criteria: significance, approach, feasibility, and impact. This structure made it easier for applicants and panel chairs to compare feedback across proposals. Some reviewers complained that the template was too rigid, but the agency found that it reduced the variance in comment length and tone, making the process more predictable. Other funders could adopt similar templates, tailored to their specific fields.

Investment in platform moderation and archiving is essential. The agency's OpenReview instance requires a dedicated moderator to handle disputes, flag inappropriate comments, and ensure that version histories are preserved. Without such infrastructure, the public audit log can become chaotic, with off-topic comments or personal attacks undermining the scientific value of the process. The agency has also had to develop clear guidelines for what constitutes a constructive comment and what crosses into harassment.

Perhaps the most important lesson is to pilot before scaling. The agency's decision to test the preprint mandate on a subset of grants—roughly 10% of its portfolio—allowed it to identify problems early and adjust the policy without disrupting the entire system. The pilot revealed the need for the anonymization option, the reviewer honorarium, and the pre-screening programs for early-career researchers. A full rollout without these adjustments would likely have generated more resistance.

Whether the preprint mandate will become permanent is still uncertain. The agency has committed to a two-year pilot, with a decision expected in early 2026. Early supporters point to the faster cycle and the richer feedback as evidence that the experiment is working. Skeptics worry about the chilling effect on early-career researchers and the risk of groupthink. What is clear is that the policy has already changed the conversation about peer review, making it harder to argue that the traditional closed model is the only option. The central question remains: can a system designed for accountability also nurture the bold, risky science that often leads to breakthroughs? The evidence so far offers no clear answer, but the experiment itself is forcing the scientific community to confront that question head-on.