After the last event in May exploring barriers to practicsing open research and some ideas of how to overcome them, June’s open lunch offered a funder’s perspective with Sonya Towers from the Wellcome Trust, along with a case study from Robert Smith, PhD candidate at the Wellcome Trust Doctoral Training Centre for Public Health, Economics and Decision Science at the University of Sheffield.
The Wellcome Trust have been leading efforts to make research more open for more than 20 years, ever since working to make sure the results of the Human Genome Project were released immediately into the public domain. They were also the first research funder to introduce a mandatory open access policy, with more than 150 global research funders having since followed their lead. More recently, they have developed the Wellcome Open Research platform, which allow their researchers to rapidly publish and share their findings openly and transparently, and encourage researchers to cite preprints in their grant applications.
A world where there are transformative improvements to human health because research outputs are managed, shared, and used in ways that unleash their full valueThe Wellcome Trust vision for Open Research
Those of us that work in research support and open access know that where Wellcome goes, other funders often follow, so it’s worth paying attention to Sonya clearly explaining the rationale behind their commitment to open research, and various associated initiatives.
Wellcome have a dedicated open research team that works with other teams across Wellcome, and with partners, to make all the outputs of their funded research accessible, so they can be combined and used to accelerate discovery and application. ‘Outputs’ encompass data sets, code and software to ensure research findings can be validated and reproduced, and increase efficiency in research by reducing duplication. It also means that research outputs are available to a full range of users, including the public, policy makers and healthcare professionals.
Driving the Transition to Open Access
Wellcome was the first major funder to have an open access policy, first introduced in 2005. In 2018 they joined cOAlition S, a group of funding organisations working to to implement ‘Plan S‘, an initiative to speed up the transition to open access, where funded research is available immediately in OA journals or via repositories. In January 2021 Wellcomes’ new OA policy, aligned with Plan S, came into effect.
In addition, all research articles must include a statement explaining how other researchers can access any data, original software or materials underpinning the research.
Wellcome are signatories to the San Francisco Declaration on Research Assessment (DORA) and are committed to ensuring that the assessment of research outputs during funding decisions is based on the intrinsic merit of the work and not the ‘prestige’ of the journal. All Wellcome funded institutions must also commit to this principle and you can read the University of Leeds Statement in Support of Responsible Metrics here.
Wellcome Open Research publishing platform
Wellcome Open Research is a publishing platform that offers a new way for Wellcome-funded researchers to rapidly publish any results they think are worth sharing, including negative or null results.
In 2020 Wellcome Open Research was the primary venue for welcome funded researchers to publish their research articles. Turnaround is very quick with articles published within 7 days of submission and researchers can publish all their research outputs from standard research articles to data sets, case reports, null and negative results. It is transparent with open peer review where you can choose referees from an approved list.
After submission, articles are subject to a number of ‘objective tests’, after which they are immediately published with a formal citation and DOI. Peer review then takes place after the article is published. Authors can respond to comments and produce new versions which are deposited into Europe PMC.
The ‘generic’ nature of the Wellcome Open Research Platform also represents Wellcome’s commitment to the principles of responsible metrics, whereby existing research assessment processes, for example based on Journal Impact Factor (JIF), are increasingly recognised as barriers to open research. Responsible metrics is part of Wellcome’s broader work to reimagine research culture.
Output Management Plan support service
While many funding organisations refer to Data Management and may require a Data Management Plan (DMP), Wellcome’s Output Management Plan (OMP) emphasises the broader range of research outputs.
An OMP should clearly describe:
- What outputs will be generated and/or how will existing data/resources be re-used
- What metadata or documentation will accompany the outputs e.g. via a data note
- When and where will the outputs be made available
- How will the research outputs be discovered and accessed by the community
- If there are restrictions on data sharing
- How the data will be stored backed up and preserved
- What resources are required
Researchers should also include where the outputs will be made available, from a recognised repository for particular data types where they exist e.g. GenBank or github for code, or a general community repository such as Zenodo or Figshare. Researchers can also use an institutional repository like Research Data Leeds. Where there are restrictions on data sharing, for ethical or commercial reasons for example, this should also be highlighted in the plan.
A recent review found that Outputs Management Plans vary widely in quality, while in some cases were not included at all when required. This has led to the development of an ‘Outputs Management Plan support service’ launched in October 2020 and running in the first instance as a pilot for 1 year. The service will be available to all applicants where the quality of the plan has been flagged as requiring improvement AND their application has been recommended for funding. Following award, grant holder will be contacted to inform them that they have been suggested to take part in the pilot. The University of Leeds is taking part in this pilot, please get in touch for further information.
Open Research & COVID-19
In January 2020, Wellcome coordinated a joint statement on sharing research data and findings relevant to COVID-19, calling on researchers, journals and funders to ensure that research findings and data relevant to the outbreak are shared rapidly and openly to inform the public health response and help save lives. Over a hundred journals and funders signed the statement. This was followed by a similar statement specifically aimed at publishers and created in partnership with the White House Office of Science and Technology Policy and the US National Library of Medicine. Wellcome have also implemented special conditions for grants funded via COVID-19 funding initiatives in line with these statements.
Looking ahead, Wellcome plans to examine any lessons learned around managing research outputs related to the COVID-19 pandemic and to explore approaches to accelerate access to research findings and data more broadly. They will continue to work with researchers, institutions and cOAlition S partners to implement the new open access policy and to support innovation.
In partnership with other funding organisations and their funded institutions, Wellcome will explore how to embed incentives for open research, which is an area we are also very interested in at the University of Leeds. If you have any ideas please get in touch!
Robert Smith described work with R Shiny and data science techniques to improve the transparency of healthy economic models in the context of two papers published with Wellcome Open Research in 2020 while a PhD student at the Wellcome Trust Doctoral Training Centre for Public Health, Economics and Decision Science. Both papers have been very successful and have had an impact beyond just an academic audience:
- Smith R, Schneider P, Bullas A et al. Does ethnic density influence community participation in mass participation physical activity events? The case of parkrun in England [version 2; peer review: 3 approved]. Wellcome Open Res 2020, 5:9 (https://doi.org/10.12688/wellcomeopenres.15657.2)
Source code available from: https://github.com/ bitowaqr/DoPE
Archived source code at time of publication: https://doi.org/10.5281/zenodo.3596841
- Smith R and Schneider P. Making health economic models Shiny: A tutorial [version 2; peer review: 2 approved]. Wellcome Open Res 2020, 5:69 (https://doi.org/10.12688/wellcomeopenres.15807.2)
A more extensive tutorial, all code, and data are provided in a GitHub repository.
Park Run is a national, free, weekly running event that have been very successful in engaging people in physical activity who might otherwise not participate in exercise. The project looked at all of the (500 plus) different locations of Park Run events across the country to explore engagement across different communities. Rob and his colleagues developed a model which predicted Park Run attendance based on a number of different characteristics of a local area.
The team of medical doctor, health economist and mathematician worked together to combine a large amount of data from ONS and Park Run, primarily working in R to analyse the data and write a report using R markdown, then simply copy/pasting into a Google doc for the supervisory team to review. The report was later input to a Latex editor to publish through Wellcome Open Research. They published all code and the aggregated data on Zenodo to enable anybody to understand and reproduce the analysis. They also created a user interface to present the results of the analysis and did some sensitivity analysis to enable Park Run or other interested people to engage with their model.
Rob described the process of working with Wellcome Open Research as very quick and convenient with short time scales. From the original submission to having the publication peer-reviewed took just five months, which meant that a peer-reviewed publication was available to inform policy. A more ‘traditional’ publishing process might take well over a year which would have reduced, or even negated, any impact and be too late for the model to be trusted as a methodology.
On the back of the peer reviewed publication, Rob and his colleagues were invited to a House of Common select committee (that didn’t go ahead due to COVID!). However they have given talks on podcasts and got to work very closely with triathlete and Park Run ambassador Chrissy Wellington. Park Run used the results from the analysis to inform where to locate new events funded by by Sport England.
The second paper has been used by industry, and the team have been approached to teach the methodology. As everything is openly accessible it has also been incorporated into teaching from other groups including UCL and the R for Health Technology Assessment (HTA) working group.
The work has also been reproduced at a “ReproHack” event. A ReproHack is an event where researchers come together and try to reproduce some research: