View from both sides of the fence: a remote reviewers experience

To discover what it is like to be a remote reviewer or a member of an evaluation panel for the ERC we interviewed biochemist Kresten Lindorff-Larsen. Prof Lindorff-Larsen was a panel member of the ERC’s PE4 panel for Physical and Analytical Chemical Sciences, in the Advanced Grant calls of 2018 and 2020. He has also served as a remote reviewer for the ERC. In 2023 he was part of a group who were awarded an ERC Synergy grant.

How did you find being a panellist on an ERC evaluation panel?

 

I enjoyed it very much, both on the social level and also to gain a new understanding of the dynamics and how different people engage with this process. It's always good to see the kind of diversity that's going on in terms of how people make decisions, make arguments, what they focus on, what they don't focus on. There are always differences in opinion, but my experience was that people tended generally to be good at listening to each other and understanding that different people focus on different aspects of a proposal. 

Someone might say this is a great proposal because they've demonstrated they can do all this stuff that is going to be difficult. Another person will say, well, they've already done all this stuff. They’re really just continuing with the same line of research. Both of those statements can be true at the same time. That’s when different views come in to play.  It’s very useful to engage in those kind of discussions that help you understand the process, but also help put forward one's own view at this slightly higher level, on individual proposals. 

 

Have you found anything different between being a panellist for a national grant and being a panellist for the ERC?  

 

I can only speak as a panel member for the Independent Research Fund Denmark (IRFD) and there it's extremely different in the sense that the panel that I sit on, we cover all of natural sciences. Which would probably correspond to 10 or 15 different ERC panels. In the IRFD we have several different funding mechanisms, but with the ERC there's the Synergy Grant and then all the other main grants are basically the same, just depending on your career stage. Also, the individual projects in IRFD that I review are all smaller than the ERC grants. 

The IRFD panels cover a much broader area, which means that I'm less of an expert on many of the topics. At the ERC there are experts on the different topics. In my experience with the ERC there are nearly always panel members with expertise that is close to the proposal, and several remote reviewers that are experts almost right on the field. I've never really felt like we've been trying to evaluate research proposals where there isn't some kind of domain expertise from multiple people.  

with the ERC there are nearly always panel members with expertise that is close to the proposal, and several remote reviewers that are experts almost right on the field

There will of course be some applicants that will feel that nobody understood their proposal in depth because the evaluators were not experts, but my experience as a panel member is that this is very rare. As an applicant to the ERC I've always felt that I've been in pretty good hands (even in cases when I didn’t get the grant). As a panel member, I've always felt that for most proposals, we gave the applicants a pretty substantial evaluation based on scientific knowledge both on and off the panel.

 

What is useful about being a panellist?  

 

Being a panel member, both earlier for the ERC and now in Denmark, I get to see a lot of grants, and see how people present them in different ways. That is a strong motivation for me, in addition to the social aspect, to sit on the panels. Given the importance of external funding in science, I think it’s an important task to evaluate grants as well as possible. I also learn more about what a good grant can look like. I think this has also made me write better proposals myself. One also gets some of that as a remote reviewer, but in my experience it’s in particular when you get to see many proposals at the same time.

I also learn more about what a good grant can look like. I think this has also made me write better proposals myself.

Why would a researcher act as a remote reviewer for the ERC?

 

I have a lot of grants to review for the Independent Research Fund Denmark — I review about 60 IRFD grants a year. I feel like I do my fair share of grant reviewing, so I often say no to review grants. When I say yes to something outside the IRFD, the most important factor for me is that I review things in topics that I feel pretty specialist on. It's more useful for me and it's also easier because I know the topic and I can evaluate it better. Those often tend to be with the ERC, and it’s an extra bonus that I think the ERC is overall doing a tremendously good job for European science. I'm happy to support that.

 

Could being a remote reviewer help researchers’ careers?

 

Being asked to be a reviewer also to some extent reflects on a person’s status within the field. If someone thinks that a certain person would be a great reviewer, it likely means that someone knows them for whatever area they're in. For junior scientists it helps a little bit, but I don't think it makes a difference whether you've done one or you've done ten. There's a lot of these kind of review jobs in science, e.g. journal paper reviews. It's something we do because we think that it helps the overall system. If no one does it, the system as we have it now would break down.

 

What about your own applications for ERC grants, did you find the evaluators’ feedback useful?

 

Editor’s note: The members of the ERC panels alternate to allow panel members to apply to the ERC calls in alternate years.  

I have applied for ERC grants multiple times including the ERC Starting Grant, Consolidator Grant and the Synergy Grant. We applied for a Synergy Grant in 2021, were placed on the reserve list, and then reapplied for the following year. And then in the middle of that process, we were pulled out of the reserve list from the first competition. When writing the second application we had evaluator comments from our initial application, but we only got them a few weeks in advance of the deadline. We updated the proposal a little bit based on this. However, most of the comments were extremely positive, so there was little to change. 

I realize that I am very fortunate and lucky to have received this ERC grant, so wanted to give a bit of background for the 10-year process it has been for me to get funding from the ERC. This funding came in my fourth attempt. 1/n https://t.co/zzvy7VvWnN

— Kresten Lindorff-Larsen (@LindorffLarsen) February 5, 2023

 

The ERC has slightly changed its evaluation process and how its application forms are laid out. Now, it’s more explicit that you can tell your story of your career.

 

One thing that I really liked about my experience on the ERC panel is the way that we looked at the CVs. These were short and often focused on presenting the importance of your work in the way you want.  It strikes an interesting balance between the very quantitative “I published 200 papers, in these and these journals” versus “Here are five papers that I'm really proud of because of these and these reasons, and they contributed to science in this way”. That's something that the ERC is doing pretty well.  

However, not all applicants are using it to their best ability. People still don't utilise the fact that they can explain in words why their previous contributions have been important. I think the (negative) side effect of that is that people really have to write the CV as something that is very targeted to the ERC application. You can’t just use your normal CV. My experience is that evaluators look at this relatively fairly. People actually look at the text and use that to judge the candidates.  

Similarly, a consideration for the ERC is also how you've helped mentor junior scientists and you have space to write about that. Unfortunately not all applicants do this, which is a pity because it’s in my opinion an important part of our jobs. I also tend to look at people's contributions to the community in various ways. These kind of things, not just for ERC applications, but in general, show something about people's willingness to contribute to the community. 

 

Remember: the project is still the prime determinant of funding.

 

The other thing that I think is nice, and this was somewhat of a surprise to me when I sat on an ERC Advanced Grant panel, is really how much focus there is on the project proposal, relative to the candidate. Everybody has their own expectations and experiences with this, but I had expected there would be much more focus on how strong the candidate is. The most important thing is the project, the proposal. The candidate has to be good but the proposal weighs a lot more. I think that that's really strict with the ERC in general. It's not about where you are, it's not about who you are, it's much more your idea and how you will carry it out. I think the ERC strikes that balance very well.

It's not about where you are, it's not about who you are, it's much more your idea and how you will carry it out. 

Kresten Lindorff-Larsen trained as a biochemist at the University of Copenhagen and Carlsberg Laboratory, completing his Ph.D. at the University of Cambridge in 2004. He then moved on to become an assistant professor in Copenhagen before joining D. E. Shaw Research in New York in 2007. He returned to the University of Copenhagen in 2011, where he now serves as a Professor of Computational Protein Biophysics at the Lindorff-Larsen Group within the Linderstrøm-Lang Centre for Protein Science. Current research interests include developing and applying computational methods to study the structure, dynamics and interactions of proteins, and the integration of biophysics and genomics research. He is also a Principal Investigator in the 2022 ERC Synergy grant DynaPLIX.