Research Excellence Framework
The Research Excellence Framework is a UK wide exercise which assesses the quality (and volume) of research and the results are used to allocate research funding to UK Universities (approx £2bn per year, of which Keele receives approximately £7m).
The funding bodies’ shared policy aim for research assessment is to secure the continuation of a world-class, dynamic and responsive research base across the full academic spectrum within UK higher education. We expect that this will be achieved through the threefold purpose of the REF:
- To provide accountability for public investment in research and produce evidence of the benefits of this investment.
- To provide benchmarking information and establish reputational yardsticks, for use within the HE sector and for public information.
- To inform the selective allocation of funding for research.
The REF is a process of expert review, carried out by expert panels for each of the 34 subject-based units of assessment (UOAs), under the guidance of four main panels. Expert panels are made up of senior academics, international members, and research users.
For each submission, three distinct elements are assessed: the quality of outputs (e.g. publications, performances, and exhibitions), their impact beyond academia, and the environment that supports research.
Preparation (2017 to 2019) The UK funding bodies appoint the REF expert panels, consult the sector and publish the criteria and guidelines for the exercise.
Submission (2019-2020) Each University decides which UoAs to submit to and prepares their submissions to be made by November 2020
Assessment (2021) Expert panels review the submissions and produce quality profiles, which are the results (published late 2021)
So far, there have been two key documents published by the REF team:
- ‘Decisions on staff and outputs’ (2017/04),
- Initial decisions on the Research Excellence Framework 2021 (REF 2017/01)
Key aspects from the REF publications above are:
Three elements contribute to the overall score and are weighted as follows: Outputs 60%, Impact 25%, Environment 15%. The panel criteria and working methods will give details of how each of these elements will be assessed.
All staff with significant responsibility for research are returned to the REF (provided they are independent researchers). Recognising there are staff with more responsibility for research than others, the REF team will work with HEIs, with their staff and with guidelines, to identify who is ‘in scope’ for submission among staff meeting core criteria. Staff categories:
- Category A eligible: building on previous definition: academic staff with contract of employment, minimum 0.2 FTE, on payroll on census date 31.7.2020, primary employment function ‘research and teaching’ or ‘research only’ - this includes independent researchers, but not research assistants.
- Category A submitted: those from the eligible pool who have been identified as having a significant responsibility for research on the census date (31.7.2020). This includes research only staff who are eligible.
- For those UoAs where not all eligible staff have a significant responsibility for research, the institution need to implement a process to determine this (i.e. Code of Practice).
- Significant responsibility for research: Those for whom explicit time and resources are made available to engage actively in independent research, and that is an expectation of their job role.
- REF team and main panels will provide further guidance on identifying staff with significant responsibility (in guidance on submissions and panel criteria).
Decoupling staff from outputs
- Number of outputs required for each submission determined by the FTE of ‘Category A submitted staff’.
- Average number of outputs per FTE is 2.5.
- A minimum of one output per person and maximum of five outputs per person.
- No minimum requirement regarding outputs for staff who have left the institution.
- Outputs can be submitted by both institutions, where the employee has moved institutions during the REF period
- The same output will be assessed once within the same UoA (if submitted by multiple institutions), but the eligibility criteria will need to be met individually by each institution submitting the output.
- Minimum of 2 case studies required (up to 15 FTE), 3 up to 30 FTE, 4 up to 45 FTE etc
- Impact template moved and assessed in environment section, therefore just the impact case studies contribute to the final score impact quality profile (25%)
- Impact is not portable with staff when they leave the institution (as in REF 2014)
- Evidence to support each impact case study is being requested in advance of the submission (rather than by audit afterwards as in REF 2014)
Confirmation of REF Open Access policy, i.e. outputs to be deposited no later than 3 months after the date of acceptance (from April 2018). A deposit exception allows those within 3 months of date of publication to remain compliant.
These are due to be published for consultation in Summer 2018, and will be finalised in Spring 2019.
The primary outcome of the assessment will be an overall quality profile awarded to each submission, showing the proportion of the submission which meets each of the starred levels. In forming the overall quality profile, sub-panels will develop a sub-profile for each of the three elements of the assessment - outputs, impact and environment, weighted by 60%, 25% and 15% respectively.
4* is defined as: quality that is world-leading in terms of originality, significance and rigour.
3* is defined as: quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.
2* is defined as: quality that is recognised internationally in terms of originality, significance and rigour.
1* is defined as: quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified research is that where the quality falls below the standard of nationally recognised work, or the work does not meet the published definition of research for the purposes of this assessment.
- The external REF timetable can be found at www.ref.ac.uk/about/Timetable/
All the results and submissions across the UK can be found here: www.ref.ac.uk/2014/
This is currently under development and more information will be available here (anticipated Summer/Autumn 2018)
Faculty of Natural Sciences
|3*||Allied Health Professions, Dentistry, Nursing and Pharmacy||Interim: Clare Holdsworth|
|4||Psychology, Psychiatry and Neuroscience||Professor Nicola Edelstyn|
|5||Biological Sciences||Professor Dave Furness|
|9||Physics||Professor Coel Hellier|
|10||Mathematical Sciences||Professor Julius Kaplunov|
|11||Computer Science and Informatics||Dr Alastair Channon (moving to Professor Fiona Polack)|
|12*||Engineering||Interim: Clare Holdsworth|
|14||Geography and Environmental Studies||Dr Ralf Gertisser|
|20*||Social Work and Social Policy||Professor Simon Pemberton (for advice only, the expectation is that Human Geography will be returned in UoA 14).|
Faculty of Humanities and Social Sciences
|17||Business and Management Studies||Professor Marylyn Carrigan (KMS Director of Research, as of 3rd Janaury 2018)|
|18||Law||Tsachi Keren-Paz (Marie-Andree Jacob will take over in July 2018)|
|19||Politics and International Studies||Liz Carter (on research leave January-July 2018; covered by Helen Parr)|
|20*||Social Work and Social Policy||Farzana Shain|
|27||English Language and Literature||Susan Bruce|
|33||Music, Drama, Dance, Performing Arts, Film and Screen Studies||Alastair Williams|
Faculty of Medicine and Health Sciences
|2||Public Health, Health Sciences and Primary Care||George Peat|
|3*||Allied Health Professions, Dentistry, Nursing and Pharmacy||Julius Sim|
|20*||Social Work and Social Policy||Interim: Elaine Hay|
The UoA lead will be responsible for coordinating their UoA submission in REF 2021. This includes oversight of the submission’s collective research outputs, environment statement and impact case studies. This will need to be done in accordance with the University’s REF Code of Practice, which will be drafted following guidance from the REF team at HEFCE, expected in Autumn 2018. UoA leads will be in regular contact with Head of Research Quality, to whom they will deliver regular updates on activities.
Outputs: Following the important baseline that has been established from the 2016/17 audit of outputs, the University will now move to a continuous mode of assessment. This expectation of continuous mode will be framed by regular points of reporting, to be agreed in due course by the Vice-Chancellor’s REF Strategy Steering Group. The UoA lead will be responsible for ensuring that all newly published outputs are being assessed/scored and included in a ranked outputs list for each UoA (if they meet open access requirements which will be checked by Research operations). The ranked outputs list will enable modelling of outputs profiles and grade point averages (GPA), the outcomes of which will be continually reported to the Vice Chancellors REF Strategy Steering Group and Faculties.
Environment: The UoA lead will be responsible for drafting the environment statement for their unit (previously called the REF5), following the format set out by the REF guidance (due Autumn 2018).
Impact: The UoA lead will have an overview of the impact case studies in their UoA, but will not be responsible for drafting them (this will be led by the case study lead and supported by the University Impact Steering Group)
Support for UoA leads: The PVC R&E will provide leadership and advice to the UoA leads, including cross-University statements around research environment and infrastructure. REF information and data analysis will be supported by Head of Research Quality and the Research Operations team in the Directorate of Research, Innovation and Engagement and their related databases such as Symplectic publications database and the institutional repository.
The Vice Chancellor’s REF steering group makes the final decisions about our REF 2021 submissions. To support this group there is a REF impact steering group (Chaired by Dr Mark Bacon, Director of Research, Innovation and Engagement) and a REF outputs steering group (Chaired by Professor David Amigoni, PVC Research and Enterprise), all of which meet at least four times a year.
Each Faculty has its own REF steering groups as follows:
- Medicine & Health Sciences: REF Executive Group (Chaired by Professor Elaine Hay, Associate Dean for Research)
- Faculty of Natural Sciences: REF Steering Group (Chaired by Professor Clare Holdsworth, Associate Dean for Research)
- Humanities & Social Sciences: REF Steering Group (Chaired by Professor Derek McGhee, Associate Dean for Research)