Evaluation Criteria and Procedures

Evaluation of applications will be performed by the PRACE Summer of HPC Student Selection Coordinator with the assistance of independent experts.

Expert selection

The PRACE Summer of HPC Coordinator will nominate expert evaluators from within PRACE. These experts are expected to evaluate applicants on an independent, impartial and objective basis. They receive and accept an appointment letter, a confidentiality agreement and a conflict of interest declaration. Confidentiality rules must be adhered to at all times, before, during and after the evaluation.

Conflicts of interest declaration: Under the terms of the appointment letter, experts must declare beforehand any known conflicts of interest and must immediately inform the Programme Coordinator if one becomes apparent to them during the course of the evaluation. In such a case, the Programme Coordinator will take whatever action is necessary to remove any possible conflict.

Confidentiality agreement: The appointment letter also requires experts to maintain strict confidentiality with respect to the whole evaluation process. Under no circumstance may an expert attempt to contact an applicant on his own account, either during the evaluation or afterwards. At the beginning of the evaluation, the experts will be briefed by the Programme Coordinator on the evaluation procedure, their responsibilities, the issues involved in the particular area/objective and any other relevant information.

Scoring

Each candidate will be evaluated against pre-determined and equally weighted evaluation criteria. For each criteria, up to 5 points can be given with the minimum threshold for each of these being 3. The evaluated criteria will be::

1.  Technical ability (based on code test and CV)
   • Code test evaluation
   • Evidence of programming ability
2. Motivation & Enthusiasm (based on application form)
   • Motivation
   • Understanding of HPC
   • Have pursued HPC classes/projects
3. Ambassador Potential (willingness to promote programme)
   • Writing skills
   • Interest in social media
   • Experience in societies/clubs

 Ranking

The evaluation of the applications by the experts in view of the establishment of a short list will be carried out in three stages:

In the first stage Student Selection Coordinator will perform eligibility check of all applications. Only eligible applicants will be considered for evaluation in the next stage.

In the second stage, each proposal will be assessed by at least two independent experts, with experts acting individually without discussing their assigned applications with each other, nor with any third party. The experts will record their individual opinion in an Individual Assessment Report (IAR), giving scores and commenting against the evaluation criteria. Each proposal will be assessed by at least two experts.

In the third evaluation stage, all experts will hold a consensus meeting to be chaired by the PRACE Summer of HPC Coordinator or his representative to discuss the complete set of applications. The experts who assessed the same proposal will discuss among them to reach a consensus on the criteria scoring. All criteria where a significant difference appears (more than 2 points at any criteria) will be addressed. After the discussion, criteria for which differences in scoring does not exceed one point, the average marks will be used for this (these) criterion (criteria). The outcome of this meeting will be a summary table displaying the scores of all candidates agreed between the relevant experts. In case it is impossible to reach an agreement between the expert evaluators, the Coordinator will designate an additional expert evaluator to act as an adjudicator. The Coordinator will take the necessary steps to assure the quality of the IARs, with particular attention given to clarity, consistency and attainment of the appropriate level of detail. If important changes are necessary, the reports will be referred back to the experts concerned. Based on the definitive marks which are agreed between the experts at the end of this consensus meeting, a ranking will be established for each project. Project preference on applications will be taken into account and best students will be assigned to the projects of their preference if possible. However, project preference does not influence ranking and there is no guarantee that accepted students will receive invitation for their preferred projects.

Consensus meeting, final scores and ranking

At the consensus meeting the board will hold a final meeting in order to:

1. Agree on the final score attributed to each listed candidate.
2. Produce a final ranking of candidates and a proposal for the projects allocation.

Invitation to selected students

Selected students will receive invitation that will require confirmation and agreement for participating PRACE Summer of HPC programme.