ERC – AdG and StG Calls 2010
Overview
Evaluation Process
Alejandro MARTIN HOBDEY
ERC
Israel, 16 December 2009
European Research Council
Overview of presentation
• Submission of proposals
• Proposal structure
• Evaluation process
• Evaluation criteria
• Inter-disciplinarity
• Financial / Budget issues
│2
European Research Council
Submission of proposals
• Single submission
“1 stage, 2 step” evaluation
• Electronic submission via EPSS only
• Deadlines strictly enforced
• Proposals have two parts:
Part A: Administrative forms
• Structured information
Part B: Scientific proposal itself
• Free form pdf file
│3
European Research Council
Submission is to Panels
• Applicant submits to a Targeted Panel (of PI choice)
Can flag one “Alternative Review Panel”
• Applicant chooses his/her panel, that panel is “responsible”
and takes ownership for the evaluation of that proposals
• Switching proposals between panels not possible unless clear
mistake on part of applicant
• Switching proposals between domains (deadlines) not
possible
• But: In case cross-panel or cross-domain proposals,
evaluation by members of other panels possible
│4
European Research Council
Call budget breakdown
• Total budget for Call: ~€500M
• Indicative call budget
• Physical Sciences – 39%
• Social Sciences – 14%
• Life Sciences – 34%
+ 13% for Interdisciplinary – Cross Panel / Cross
domain
Within each domain, budget breakdown is broken down by
demand (equal chance in each panel!)
│5
European Research Council
Panel structure
• 3 domains – 25 panels
10 PE panels
9 LS Panels
6 SH Panels
• AdG and StG Panels structure the same
• Two sets of AdG panels, meeting on alternative years
• Members of “alternative panels” for given year may be
used for remote evaluation in particular cases
│6
European Research Council
Panel Structure
Social Sciences and Humanities
• SH1 - Individuals, institutions and markets: economics, finance and
management
• SH2 - Institutions, values and beliefs and behaviour: sociology,
social anthropology, political science, law, communication, social
studies of science and technology
• SH3 - Environment and society: environmental studies, demography,
social geography, urban and regional studies
• SH4 - The Human Mind and its complexity: cognition, psychology,
linguistics, philosophy and education
• SH5 - Cultures and cultural production: literature, visual and
performing arts, music, cultural and comparative studies
• SH6 - The study of the human past: archaeology, history and
memory
│7
European Research Council
Panel Structure
Life Sciences
• LS1 - Molecular and Structural Biology and Biochemistry
• LS2 - Genetics, Genomics, Bioinformatics and Systems Biology
• LS3 - Cellular and Developmental Biology
• LS4 - Physiology, Pathophysiology and Endocrinology
• LS5 - Neurosciences and neural disorders
• LS6 - Immunity and infection
• LS7 - Diagnostic tools, therapies and public health
• LS8 - Evolutionary, population and environmental biology
• LS9 - Applied life sciences and biotechnology
│8
European Research Council
AdG Panels Structure
Physical Sciences and Engineering (deadline 28
February 2008)
• PE1 - Mathematical foundations
• PE2 - Fundamental constituents of matter
• PE3 - Condensed matter physics
• PE4 - Physical and analytical chemical sciences
• PE5 - Materials and synthesis
• PE6 - Computer science
• PE7 - Engineering sciences: systems, electronics,
communications, optics
• PE8 - Engineering sciences: products and processes
• PE9 - Universe science
• PE10 - Earth system science
│9
European Research Council
Proposal structure: Part A
• Part A: “Administrative forms” containing
A1 Information on PI
A2 Information on Host Institution
A3 Budget breakdown by year and partner
│ 10
European Research Council
Proposal Structure: Part B
• Section 1 = “The PI”
• Scientific Leadership profile (2 page)
• CV (including “funding ID”) (2 page)
• 10-year track record (2 page)
• Extended synopsis (5 page)
• Section 2 = Full Scientific proposal (15 page)
• Section 3 = Research Environment description (2 page)
• Ethical Review information (where appropriate)
• Separate additional document:
Host Institution Support Letter (compumsory)
Copy of Ph.D. (compulsory StG)
Documents to prove extension to 10 year eligibility window (StG)
│ 11
European Research Council
Two step evaluation
• Step 1:
Section 1 of Part B evaluated against Criterion 1 (PI)
and 2 (Research Project)
Proposal needs to pass threshold for both criteria to
pass to second step
Evaluated by Panel Members + possibly “alternate”
panel members where necessary
• Step 2:
All three sections evaluated against all three
evaluation criteria
Evaluated by Panel Members + Remote Referees
StG: Step 2 includes interviews in all cases
│ 12
European Research Council
Evaluation process
Submission Step 1 Step 2
(panel) (Panel + remote)
Proposal Proposal Proposal
Section 1 Section 1 Section 1
Section 2 Section 2 Section 2
Section 3 Section 3 Section 3
+ HI support letter
+ copy Ph.D (StG)
Indiv assessments
Indiv Assessments Individual marks
Individual Marks Interdisciplinary flag
Eligibility Check PANEL MEETINGS PANEL MEETINGS
Ranking Ranking
│ 13
European Research Council
Marking scheme
• Criteria 1 and 2 will be marked according to the following scheme:
• 4: Outstanding
• 3 Excellent
• 2 Very good
• 1 Non-fundable / fail
• Criteria 3 is pass fail
• Quality threshold of: >=2; ½ marks allowed
• Proposals below the quality threshold for either of the two criteria are
eliminated (in Step 1) / “not fundable” (in Step 2)
• Proposals passing from Step 1 to Step 2 have to pass all thresholds,
but also will be limited according to a given multiple of the funding
available for that panel (~x3)
│ 14
European Research Council
Remote part of evaluation
• Remote part of evaluation will take part completely
electronically
• Step 1: proposals assigned to ~4 PMs
• Step 2: proposals also sent also to specialized remote
experts (to be determined at Step 1 panel meeting)
• Reviews will be gathered on-line via a dedicated, secure
web system
• Note different deadlines for different panel (clearly
indicated on user interface)
• Panel Coordinators will assist/remind PMs in keeping track
of what reviews are due for which deadlines
│ 15
European Research Council
Panel Meetings
• Step 1 and Step 2 Panel meetings similar
• Objective is to take decisions on which are the successful
proposals, document these decisions, and to finalise marks
and feedback to applicants (very important)
• Goal: to have done as much as possible of this work
remotely ahead of time: basis of feedback is the (4)
Individual Assessments
• May have a lead reviewer, who presents the proposal and
reviews opinions to panel, and is primarily responsible for
drafting panel comment
│ 16
European Research Council
Interdisciplinary Proposals
• Primary approach for Interdisciplinary proposals: “mainstreaming”
• Proposal submitted to a target panel primarily responsible for its
evaluation
• Both Step 1 and Step 2:
Assigned for reviews from PMs outside primary panel, if necessary
• Step 2:
Remote reviewers: any reviewer in the world (hence any domain)
• If panel not comfortable in Step 2 meeting to take final decision, this is
left to a final Panel Chair Meeting
• Interdisciplinary Research domain (cross-domain & cross-panel)
indicative budget of 13% total budget
• Decision taken by combined panel of all Panel Chairs
│ 17
European Research Council
Budget considerations of proposals
• Budget considerations/analysis arises in Step 2 evaluation
(meeting)
• Panels have responsibility to ensure that resources
requested are reasonable and well justified
• Budget analysis needs to be justified on a proposal by
proposal basis (no cross-the-board cuts)
• Panels to recommend a final maximum EC budget based
on the resources allocated/ removed
• We do not get into “micro-management” of project finances
• Awards made on a “take-it-or-leave-it” basis: no
“negotiations”
│ 18
European Research Council
Conflict of Interest : taken very seriously
• Conflict of Interest taken very seriously
• ERC runs certain computer checks/ matching which
eliminates ~80% of the cases
• But: Primary responsibility always lies on the
evaluator to declare any conflicts of interest
• Rules are laid out clearly in the Panel Members
Appointment Letter
• Panel members requested leave the meeting room
during discussion of proposals for which they have a
CoI
│ 19
European Research Council
Approach taken for “streaming” (StG only)
• “By default”: people fall into the stream that corresponds by
the number of years past Ph.D. (covers 90-95% of cases)
• Request of change of stream: for this to be manageable, take
rather strict / conservative approach
• Applicants need to argue their case for different consideration.
• All “objective” (parental leave, military service, long term
illness, etc) reasons that apply for extending the eligibility
window apply by analogy to extending the “starter” window.
• Subjective reasons: “Career break” outside science/research (e.g.
volunteer work, professional activity completely unrelated to research
career, unemployment, etc)
• Applicant requested to/ needs to attach a minimum amount of
documentation to make claim credible
│ 20
European Research Council
“Co-Investigator projects” – AdG only
• Exceptionally, for Interdisciplinary proposals, the PI can include one or
more “Co-Investigators”
• These projects are subject to a higher financial limit (3.5 M€) BUT the
Co-Is are subject to the same re-submission rules as PIs!
• Co-Is have to complete Scientific leadership profile, CV and 10 year
track record in Part B
• Scientific added value of including the CO-I to be assessed by
evaluation panel
• No formal link between Co-Is (scientific issue) and existence of
partners (administrative issue)
│ 21
European Research Council
Key Documents & Information Sources
• Most important
ERC 2009 Work Programme
Guide for Peer Reviewers
• Less important
Call Text
Guide for Applicants
Rules for Submission
│ 22