This post is by Research Support Advisor Barbara Lancho Barrantes. If you would like advice on bibliometric analysis please submit an enquiry to research@library.leeds.ac.uk

UK Research and Innovation (UKRI) has demonstrated their commitment to responsible practices in research evaluation by signing the San Francisco Declaration of Research Assessment (DORA)

UKRI signs San Francisco Declaration of Research Assessment

UKRI works in partnership with universities, research organisations, businesses, charities, and government to create the best possible environment for research and innovation. It operates across the UK bringing together the seven Research Councils, Innovate UK and Research England.

The Higher Education Funding Council for England (HEFCE) and RCUK (the Research Councils’ parent body before the formation of UKRI) had both previously signed DORA,  while in February 2018 RCUK published a statement on the responsible use of metrics. HEFCE has now also been incorporated into UKRI. The statement includes a commitment to review and update the principles and guidance for researchers applying for funding, for reviewers assessing applications, and for evaluation of funded research outputs.

The seven Research Councils under the UKRI umbrella are all DORA signatories in their own right:

The San Francisco Declaration of Research Assessment (DORA)

With a pressing need to improve the ways in which the outputs of scientific research are evaluated by funding agencies, academic institutions, and others, a group of editors and publishers of scholarly journals met during the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco on December 16, 2012. The group developed a set of recommendations that became the San Francisco Declaration on Research Assessment.

dora

“do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.” DORA (2012)

In all, there are 18 statements classified into recommendations for funding agencies, institutions, publishers and organizations that supply metrics for researchers.

The responsible use of metrics

According to the UK Forum for Responsible Research Metrics “metrics form part of an evolving and increasingly digital research environment, where data and analysis are important. However, the current description, production, and use of these metrics are experimental and open to misunderstanding. They can lead to negative effects and behaviours as well as positive ones.”

Although DORA is the most widely known, other initiatives for responsible metrics include the Leiden Manifesto and The Metric Tide report.

The Metric Tide outlines the key principles of responsible metrics as:

  • Robustness – basing metrics on the best possible data in terms of accuracy and scope
  • Humility – recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment
  • Transparency – that those being evaluated can test and verify the results
  • Diversity – accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system
  • Reflexivity – recognising and anticipating the systemic and potential effects of indicators, and updating them in response

Many research funders have already signed DORA, including the Wellcome Trust, Cancer Research UK, and the Daphne Jackson Trust.

Wellcome Trust and responsible metrics

In 2018, as part of their open access policy alignment with Plan S, the Wellcome Trust made an announcement that they will only fund institutions that have publicly committed to content-based evaluation approaches such as DORA, the Leiden Manifesto or equivalent.

Russell Group and UK universities

A number of Russell Group as well as several MillionPlus universities have already signed DORA. These include Oxford, UCL, Imperial, Edinburgh, Manchester, and King’s College London from the Russell Group, Edinburgh Napier, Dundee, SGUL, Salford and Bedfordshire from MillionPlus.

For a searchable list of all signatories, organisations and individuals see DORA Signers

University of Leeds and responsible metrics

The University of Leeds is committed to the responsible use of metrics and has convened a working group comprising managers, academics and Library staff to explore the issue and make recommendations.