Aim of the pilot

With this pilot the Commission wants to tackle concerns that the evaluation process could be perceived as biased towards well-known organisations in countries with better performing Research and Innovation systems. The Commission indicates that a recent independent study has not revealed such a bias.

The pilot aims to identify whether the implementation of blind evaluation within the legal requirements and operational context creates any difficulties. If this is not the case, it might lead to a modified approach, with a greater use of blind evaluations, which could effectively mitigate the risk of real, potential or perceived reputational bias.

The recording of a Commission information webinar on the topic is available here. The slides used during this webinar can be found in attachment.

Calls concerned by the pilot

In the Horizon Europe Work Programme 2023-2024, all first-stage proposals of two-stage call topics will be evaluated blindly (except one call for Widening).

Consequences for applicants

The concept of blind evaluation requires that experts evaluators do not know the consortium structure and the applicant(s) involved. To accomplish this, the following safeguards are in place:

• Part A of the proposal will not be transmitted to evaluators.  
• If Part B of the proposal includes any identification of the applicant, the proposal will be declared inadmissible and will be rejected.

Thus, applicants submitting a proposal under the blind evaluation pilot must take into account a new admissibility criterion: they must not disclose their organisation names, acronyms, logos nor names of personnel in Part B of their first-stage application (see General Annexes A &E). If identification as listed here above is mentioned by applicants in the proposal’s Part B, the proposal will be inadmissible.

In addition, names of proposers or their organisations should not be potentially identifiable indirectly, via, for instance, links to web pages or through references to their role and previous experience. The proposal can include references to their own publications if there is no emphasis that the publication is authored by one or more of the proposers. The proposal will be declared inadmissible if its anonymity is not respected in the part B of the application. Information on the identity of participants should be included only in proposal Part A.

Updated Application Form

The template of the standard application form for stage 1 applications has been updated accordingly and now includes a dedicated page with guidance for blind evaluations and annotations in the template text to guide applicants. Note that this template serves only as an example. The actual forms, provided in the online submission system under the Funding and Tenders Portal, might differ from this example.

The following extract from the dedicated page with guidance for blind evaluations (taken from the template of the standard application form for stage 1 applications) provides concrete examples:

• How to insert references to publications? The proposal can include references to participants’ own publications if there is no emphasis that the publication is authored by one or more of the proposers.
  For example, the following statement will not be admissible: ‘For climate impact, we will use greenhouse gas emission intensities, following a methodology developed previously by a project partner (Dalin et al.)’
  but the following would be ok: ‘For climate impact, we will use greenhouse gas emission intensities, following the methodology described in Dalin et al.’
  
  
• Examples of statements resulting in inadmissible proposals:
  - ‘Most of project’s participants have been involved in the previous H2020 project, NANOCOM...’
  - ‘Partner 3 is the leading company in Spain for wind turbine installation’
  - ‘The consortium consists of leaders in the high tech industry, including the biggest in terms of capital
  constructor of micro chips ‘
  - ‘The coordinator organisation was the one who first introduced the concept of m-RNA in vaccines’
  - ‘The consortium consists of 2 research centres (including an international one based in Geneva) and the
  oldest university in Belgium’

In the different sections of the standard application form template, annotations have been added to remind applicants to pay special attention not to expose their identity. Herewith two examples (Excellence part):

• 'Describe how your project goes beyond the state-of-the-art, and the extent the proposed work is
  ambitious. Indicate any exceptional ground-breaking R&I, novel concepts and approaches, new products,
  services or business and organisational models. Where relevant, illustrate the advance by referring to
  products and services already available on the market. Refer to any patent or publication search carried
  out.' => Commission annotation in the standard application form stage 1 template: The description of the above should be provided in general terms, to preserve the anonymity of the proposal.
  
  
• 'Any national or international research and innovation activities whose results will feed into the project, and how that link will be established.' => Commission annotation in the standard application form stage 1 template: The point above refers to activities in general and not to those carried out by the participants. It should be nevertheless kept in mind that the description has to be in line with the anonymity of the proposal.

It is important to keep in mind that stage 1 will only assess the excellence and impact criteria, whereas the quality of the partners and of the consortium composition (including relevant previous publications, achievements & projects of the consortium partners) will be assessed only in stage 2. More details about the two-stage submission procedure are available in this dedicated infosheet.

Background - Legal basis

The legal basis for this pilot can be found in Article 28 of the Horizon Europe Regulation:

Article 28 Award criteria and selection

[…] 4. The Commission and other funding bodies shall take into account the possibility of a two-stage submission and evaluation procedure and where appropriate, anonymised proposals may be evaluated during the first stage of evaluation based on one or more of the award criteria referred to in paragraph 1.