Providing applicant feedback for peer review video

This video was recorded in September 2024.

Video transcript

Speaker 

• Daniel Brennan – Assistant Director Investigator Grants, NHMRC 

Welcome – (0:06)
Hello and welcome to a short video designed to assist you with providing feedback to applicants.

I'm Daniel Brennan, Assistant Director in the Investigator Grant scheme.

I'd like to begin by acknowledging the Ngunnawal people, the traditional owners of the lands on which I'm joining you here in Canberra, and to acknowledge all the other traditional owners on the lands that you are viewing this video from.

I pay my respects to Elders past, present and emerging and acknowledge any Aboriginal and Torres Strait Island people who may be viewing this video.

Thank you for taking part in this year's peer review and the important contribution that you're making to NHMRC’s peer review process.

Applicant feedback fundamentals – (0:39)
As part of your assessments, you're required to provide the applicant with constructive written feedback.

This feedback is important to the applicant to understand the strengths and weaknesses of their application and the key elements that influenced your scoring.

It was introduced for the 2020 Investigator Grants round to improve the transparency of peer review.

Please remember to always frame your feedback against the assessment criteria and score descriptors.

Following peer reviewer feedback, the Sapphire response field for applicant feedback is now between 100 and 1500 characters.

In addition to the key elements of the application that influenced your scoring, your feedback should focus on the elements that the applicant should focus on addressing with a future application if they are unsuccessful.

Also, where in your opinion the applicant has applied at the wrong level or the level applied for was not well justified, please provide details on how it did not align with the Statements of Expectations.

It may assist you in writing your feedback to ask yourself, what did this application need for me to score it higher? Or, what about this application made me score it as well as I did? Or, consider what would be the one or 2 most significant changes that the applicant could make next time to make it a stronger application and use this when framing your response.

Your feedback should always reflect the scores that you have given the applicant. By this I mean don't provide exclusively positive written feedback to an applicant you gave low scores to or vice versa.

What does real feedback look like? – (2:10)
So, what does good feedback look like?

Good feedback is concise, constructive, kind and professional.

It links the strengths and weaknesses of the application to the score descriptors and provides helpful information to the applicant on areas of their application or career that they can address for a future application.

Consider highlighting what the applicant did well, along with the most significant changes that the applicant could make for future applications.

Where possible, providing feedback across multiple criteria is helpful rather than focusing only on one assessment criterion.

Further detailed information is provided on Appendix A of the Peer Review Support Pack.

Appendix A also provides example phrases for stronger and weaker applications.

For example, when providing feedback on publications for a strong application, you might write:

‘The applicant’s nominated publications demonstrate a strong publication track record relative to opportunity. There was clear evidence applied in the explanation fields of each publication’s quality and contribution to science, along with clear explanations of the applicant’s contribution to each’.

Whereas, for a weaker response to knowledge gain in a research proposal you might write:

‘Although ambitious in its aims, the research proposal would have benefited from a more cohesive program of work rather than what appeared to be a set of disparate projects. Some information was difficult to find as it appeared in the wrong section of the application’.

In general, a good rule when providing written feedback is to imagine what type of feedback would be most helpful for you if you were the applicant when submitting your next application.

Practices to avoid – (3:50)
Secretariats spend a substantial amount of time reviewing the applicant feedback provided by peer reviewers and commonly identify practices to avoid.

This includes using variable character limit to recount to the applicant what their application involves, thereby limiting the value of the feedback that's being provided.

Making comparisons to other applications or applicants. Every application needs to be assessed against the score descriptors. Using unprofessional or derogatory language or rhetorical questions.

Most applicants are unsuccessful, so please be kind but constructive. Please do not include comments that suggest that you've considered information outside of the application or criteria that are not part of the Investigator Grants assessment.

And don't mention possible eligibility, duplicate funding, or integrity concerns.

Please raise these concerns directly with your secretariat and not include them in the feedback.

Applicant feedback sharing – (4:46)

Once assessments are complete. You will be notified when you can review the de-identified applicant feedback of reviewers for applications you have assessed.

In addition to assisting, you gauge how your assessments compared with those of the other peer reviewers for the same application, this also increases peer review accountability and transparency in the application-centric peer review process.

The purpose of feedback sharing is not to edit or change your own independent assessments or to calibrate assessments between reviewers.

Rather, it is to help identify any errors that may have occurred and to provide a learning opportunity for peer reviewers.

When reviewing, we ask you check that (1) all feedback relates to the application. Peer reviewers may make an error transcribing offline assessments into Sapphire. Please consider whether any of the comments may be about a different application or suggest a serious misinterpretation has occurred.

(2) That all comments are appropriate and professional. Where you have any concerns, please contact your secretariat.

NHMRC may contact reviewers for clarification of feedback based on your input. NHMRC staff also review a substantial proportion of applicant feedback and may contact you for clarification on the feedback or scores you have provided.

If you are contacted, please provide your response as soon as possible to prevent delaying outcomes being advised to applicants. NHMRC provides feedback verbatim, which means we require your response to these requests as soon as possible.

Thank you – (6:15)
Thank you for watching. Please remember to contact your secretariat using the GRP inboxes with any questions or concerns.

Thank you again for your assistance for Investigating Grants peer review.

We look forward to working with you.

End of transcript.