This post is by Research Support Advisor Barbara Lancho Barrantes. If you would like advice on bibliometric analysis please submit an enquiry to firstname.lastname@example.org
UK Research and Innovation (UKRI) has demonstrated their commitment to responsible practices in research evaluation by signing the San Francisco Declaration of Research Assessment (DORA)
UKRI works in partnership with universities, research organisations, businesses, charities, and government to create the best possible environment for research and innovation. It operates across the UK bringing together the seven Research Councils, Innovate UK and Research England.
The Higher Education Funding Council for England (HEFCE) and RCUK (the Research Councils’ parent body before the formation of UKRI) had both previously signed DORA, while in February 2018 RCUK published a statement on the responsible use of metrics. HEFCE has now also been incorporated into UKRI. The statement includes a commitment to review and update the principles and guidance for researchers applying for funding, for reviewers assessing applications, and for evaluation of funded research outputs.
The seven Research Councils under the UKRI umbrella are all DORA signatories in their own right:
- Arts and Humanities Research Council (AHRC)
- Biotechnology and Biological Sciences Research Council (BBSRC)
- Engineering and Physical Sciences Research Council (EPSRC)
- Economic and Social Research Council (ESRC)
- Medical Research Council (MRC)
- Natural Environment Research Council (NERC)
- Science and Technology Facilities Council (STFC)
The San Francisco Declaration of Research Assessment (DORA)
With a pressing need to improve the ways in which the outputs of scientific research are evaluated by funding agencies, academic institutions, and others, a group of editors and publishers of scholarly journals met during the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco on December 16, 2012. The group developed a set of recommendations that became the San Francisco Declaration on Research Assessment.
“do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.” DORA (2012)
In all, there are 18 statements classified into recommendations for funding agencies, institutions, publishers and organizations that supply metrics for researchers.
The responsible use of metrics
According to the UK Forum for Responsible Research Metrics “metrics form part of an evolving and increasingly digital research environment, where data and analysis are important. However, the current description, production, and use of these metrics are experimental and open to misunderstanding. They can lead to negative effects and behaviours as well as positive ones.”
The Metric Tide outlines the key principles of responsible metrics as:
- Robustness – basing metrics on the best possible data in terms of accuracy and scope
- Humility – recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment
- Transparency – that those being evaluated can test and verify the results
- Diversity – accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system
- Reflexivity – recognising and anticipating the systemic and potential effects of indicators, and updating them in response
Wellcome Trust and responsible metrics
In 2018, as part of their open access policy alignment with Plan S, the Wellcome Trust made an announcement that they will only fund institutions that have publicly committed to content-based evaluation approaches such as DORA, the Leiden Manifesto or equivalent.
Russell Group and UK universities
A number of Russell Group as well as several MillionPlus universities have already signed DORA. These include Oxford, UCL, Imperial, Edinburgh, Manchester, and King’s College London from the Russell Group, Edinburgh Napier, Dundee, SGUL, Salford and Bedfordshire from MillionPlus.
For a searchable list of all signatories, organisations and individuals see DORA Signers
University of Leeds and responsible metrics
The University of Leeds is committed to the responsible use of metrics and has convened a working group comprising managers, academics and Library staff to explore the issue and make recommendations.