Being an evaluation panel chair – a worthwhile experience?

Margaret Hunt is a historian working at Uppsala University. She has been involved in the evaluation process for ERC grants both as a normal member and as a chair of a panel.  Prof Hunt answered our questions on the value, difficulties and processes of this work, as well as giving some advice to applicants.  

How did you start your work as an ERC panel chair?   

I initially worked as a panellist, and then I came in as a panel chair to fill in for somebody who had to leave unexpectedly.  It was a little bit of an emergency situation. I have a lot of experience running organizations, departments, committees, and so on, and the ERC knew that I could chair a meeting.  That's probably why I ended up being asked. But I agreed because every organization is different and I thought it would be an interesting challenge.  

I think distributing money on a competitive basis to younger and mid-career scholars is a positive intervention in general. Especially these days when there are relatively few permanent academic positions and a good number of people apply for each opening.  A certain number who get an ERC grant do not currently have permanent positions.  These people often then get a permanent job because of having been awarded an ERC grant. The work we do can have a real impact on the lives of some very talented younger and mid-career scholars. 

The work we do can have a real impact on the lives of some very talented younger and mid-career scholars. 

I also think it's important that this kind of work be done in a careful, ethical, and equitable way.  My experience prior to, and while acting as an ERC chair, is that the panel team does a very good job of setting up the parameters for doing this. However, the panel chair must operationalise that in the panel meetings when difficult decisions have to be made. 

 

You were originally based in the U.S. Do you find differences between countries in how evaluation panels are run?

 

I'd already been in Sweden for seven, eight years when I came on as a panellist and then panel chair.  I was also involved in several large grants, from the UK and Sweden, so I knew quite a bit about the research environment in those countries. I knew a lot less about other European countries or other countries that are within the ERC catchment area, so I was interested to see how that worked.  

I have not actually seen all that much difference in terms of the process of defining or deciding what quality is, which is always a difficult problem.  Very smart people routinely disagree about what merit or quality is whether it's in the United States or the UK or Sweden or Brussels.  It's challenging to try to reach agreement on the merits of proposals that are almost all quite good, but only some of which can be funded.  We're looking for the excellent ones.  

Very smart people routinely disagree about what merit or quality is whether it's in the United States or the UK or Sweden or Brussels.

You can't expect agreement to be there, but somehow you have to reach enough agreement that final decisions can be made and that's both the challenging and the interesting aspect.  It's also sometimes heart-breaking because you often lose the proposals you really love, and that is just as true for the chair as it is for regular panellists.  Nobody who's been on these panels hasn't at some point failed to get a proposal they really loved over the finish line.

 

The ERC only funds 10 to 15 percent of applications, depending on the Call.  When are the eliminations made?

 

Usually, we only have the space to interview 35 to 40 people.  It depends on how many people have applied. A larger pool of applicants usually means a higher number of interviews but in the end the success rate from initial submission numbers is the same as for smaller panels, because the total funding budget is divided proportionally to the number of submissions in each panel.    

But that also meant that the majority of applicants get knocked out of the competition in the first evaluation panel meeting. We have to reduce the pool so that only approximately 30% of proposals go on to step 2. At the same time, it is also important to spend enough time on each candidate’s proposal that most panellists understand the reasoning and can live with rejecting them (or, as the case might be, moving them to step 2). Lots of good proposals get rejected at that early stage, and some more proposals get knocked out at the end. As I say, it can be heart-breaking.

 

Is there a part of this process that you particularly enjoy or appreciate?

 

It's fun to be in a room with a bunch of smart people talking about smart projects.  Moreover, that interaction is concentrated in one week in the fall and one week in the spring, which makes it quite intense.  Somebody on my panel was comparing it to a summer camp.  You get to eat lunch with these people, and general conversations go on at dinner that are usually really interesting.  You also make new friends and find out about the things that are important to them intellectually. 

I also greatly enjoy seeing the new projects, proposals and ideas that are coming up the pipeline. There are always things that are unpredictable and unusual that turn up. It makes you think about the nature of the profession and about the nature of adjacent professions.  Panel SH6 is the study of the human past, and our panel includes archaeology as well as history. It's been both eye-opening and just plain intellectual fun to find out what the archaeologists are doing and to observe the very different ways that they think about the human past. The much longer space of time that they're interested in, the contrasts and similarities in terms of methodologies and the questions they ask, their unusual approach to sources - all of that is very intellectually stimulating.

I also greatly enjoy seeing the new projects, proposals and ideas that are coming up the pipeline.

Being a chair as opposed to being a panel member – what are the main differences?

 

You know how academics are - they don't respect authority which is one of our best features in in my opinion - so it's not like you get immediate prestige for being panel chair. It's barely even “first among equals” because that's just not the way academics work.  It's more that you have to enlist people in the project of getting the ship into port. Usually, most people who have any sense, certainly all the people on my panel (who are excellent), will cooperate with you in that project, even though it's difficult to get from the proposal to a decision.  

It's hard to make these kinds of decisions because we don't always agree. You must bring in multiple facets of any kind of proposal. Is it creative, is it new, is it well conceived, is it doable? The last of these is especially challenging because it is always hard to make predictions about the future. 

Moreover, no proposal is perfect, so there's also always the question of whether the faults in the proposal - real or imputed - rise to the level of making members of the panel say “No, we can't fund this proposal.”  Those faults are always there; if you really want to nit-pick you can find something wrong with anything. Academics are experts at that. But you must work out a way to decide that’s both ethical and informed.  It's a lot of work achieving consensus or at least convincing people to live with a decision, even if they don't always like it, and that is basically what the chair does.

 

Does your work help researchers in your field or in your university?

 

In all the fields I know of (and certainly in history) it's important for senior people who've gotten a lot of grants in the past, or who have sat on a lot of boards, to convey valuable knowledge about grants-writing, project-planning etc. to people at other stages of their career.  That is part of what senior faculty do, or should do, almost everywhere.  But it has to be done in ways that do not conflict with the goals of the ERC.  

What that means is that, while you're working as a panel chair, you can't advise people on proposals or projects that might end up as an ERC application.  You can't if you're a panellist either.  It's sometimes a problem because it often happens that people that I know –especially junior colleagues-- ask me for advice. When you are a panellist --which means that your name is not listed online until that competition is closed-- and people look for your advice having no idea that you're on the panel, you just have to say “I can't do that”. That's pretty much true of every funding agency though: if you're sitting making decisions it isn’t ethical to be advising prospective applicants.  Once you're a chair at least your name is on the ERC’s website so you can say “I really can't do it because I'm running this panel.”

if you're sitting making decisions it isn’t ethical to be advising prospective applicants

If you were talking to an applicant for an ERC grant what would tell them that would be different to advice that you would give when they're applying for a national grant?

 

ERC proposals at present consist almost entirely of applications for large projects that are close to the allowable maximum funding and call for postdocs, PhD students, research assistants, a lot of travel funds, and the like. Unfortunately, there are almost no proposals anymore that ask for money for one person with a graduate student or one person with a research assistant.   

I think that the pressure from host institutions, coupled with the decline in funding for graduate students, means that basically almost everybody who applies for ERC is pressured by their institutions to ask for the maximum and make the project as big as they can, within the financial constraints. That's unfortunate, particularly in the humanities, because not all projects are best served by that kind of model. There are plenty of people applying with a project plan that would actually be better and probably more likely to be funded if they weren't also applying for three postdocs and four PhD students. Sometimes it really should just be that individual sitting in a room for however many years and writing a book. 

And yet, there's nothing in the ERC guidelines that say you must have a vast project with multiple postdocs and PhDs and so forth. There's absolutely nothing, anywhere that says that. It's an urban myth.  The single researcher, maybe with a travel budget and a research assistant is in no way barred from applying for a Consolidator Grant.  Urban myths are very difficult to overcome though. (Obviously this is much less of an issue for the hard sciences where these kinds of big, big projects are the norm).  

there's nothing in the ERC guidelines that say you must have a vast project with multiple postdocs and PhDs and so forth

I have also come to believe that there's nothing that the ERC Scientific Council can do to change this by simply going to institutions and telling them “no, you don't have to apply for the maximum amount.” What might be necessary is a different category of grant or else some subset within the ERC grants that's deliberately either a lower sum of money or deliberately focused on research basically by one or two persons. The advantages of that would be that more people could be funded because every project wouldn't have to be two million a throw.  I think it would also expand the number of people and institutions who could apply for these grants, because presently many institutions cannot administer huge projects.

 

Under what circumstances would you recommend being a panel chair for the ERC?

 

I'm on phased retirement so I had a little bit of extra time to put into it.  The first time I chaired the panel I was still teaching full-time, and that was quite a lot of work.  You have to read all the proposals in advance (at least the B1s, which is the first stage of the proposal) and it's not at a great time in the academic year. That's something people will need to weigh if they're thinking about doing this.  

I would say that a prerequisite is having a good deal of experience in organising various groups of people, particularly in academia where people are prepared to be their own boss to a large extent.  You must have experience getting academics to move from A to B. Plenty of academics do. I think it's not only a useful job but a necessary job and somebody has to do it. 

a prerequisite is having a good deal of experience in organising various groups of people, particularly in academia where people are prepared to be their own boss

Margaret Hunt’s educational trajectory began with an undergraduate degree in music, and progressed through a master’s degree in theology, a PhD in history and five years as a feminist and community activist in Boston, Massachusetts; Juneau, Alaska; and the South Bronx, New York City. Today she is a social, cultural and gender historian of early modern Europe and global history at Uppsala University. Prof Hunt’s primary focus is on European/South Asian encounters, maritime and military history and women and the law. Her most recent project, with Leos Müller of Stockholm University, is funded by the Swedish Research Council and it focuses on the Scandinavian Prize Papers (papers of seized ships from the 1600s to the early 1800s).