‘ Octopus represents an entirely new publishing model, with the potential to transform research communication and research culture.’

Liz Bal, Director of open research services, at Jisc

The current model of academic publishing does not necessarily work in the service of science and scholarship. As discussed at a previous event with cOAlition S, it is expensive and full open access has still not been achieved. Science is potentially distorted by the commercial models and the proxy methods of research quality that go with it.

Octopus is an innovative new open science platform created by Dr Alexanda Freeman that is designed to replace journals and papers as the place to establish priority and record your work in full detail.

In March, Dr Freeman joined us with colleagues from Jisc to tell us more about the platform due to launch in June. She was introduced by Leeds’ Dean for Research Culture, Dr Cat Davies. A full recording is available on YouTube:

Why do we do it like that?

As discussed elsewhere on the Library blog, the problems with the current academic publishing system are many and varied…so why do we do it like that? This was the question Alex found herself asking on returning to academia after a career with the BBC.

The current publishing system is too slow, too expensive, and vulnerable to bias. There are blockers to the quick and open sharing of research with poor measures of quality. Then there are problems of reproducibility, discoverability and accessibility. A hierarchical system that prioritises a mythical high impact publication can lead to a range of negative behaviours ranging from ‘questionable research practices’ to people just losing that sense of fulfillment and control and even to outright bullying.

Octopus aims to address many of these issues and which also have a knock-on effect to the broader culture of research.

The academic journal: primary research record or tool for dissemination?

Alex suggests that a major factor that contributes to these problems is that journals are trying to fulfil two different roles that pull them in different directions:

  • disseminating useful findings to practioners and researchers
  • acting as the primary research record: what has been done, by whom, when and with what results, in full detail

The first of these inevitably focuses on results and their potential impact. It tends to emphasise a simple ‘narrative’ which can discount negative results and contribute to questionable research practices.

On the other hand, the journal’s role as the primary research record makes it the mechanism to establish scientific priority, as well as the place to record comprehensive methodology and full data sets. This represents a more specialised form of communication that is easily neglected as journals prioritise dissemination to build a broad audience, which is ultimately what drives their reputation and revenue.

Alex argues that it is these competing aims that lead to some really serious problems whereby research quality is judged on the basis of publications that are essentially designed to ‘sell a story’.

Octopus: a new primary research record

Octopus disaggregates the journal article into its component parts in a structure that more closely matches the actual process of research, which generally occurs in a defined sequence with each step often requiring different skills and resources. It is designed to maximise access to primary research – digital first and free to use – and to set a new incentive structure to support good science. Leaving journals free to focus on dissemination without incentivising questionable research practices.

Whereas the journal model artificially forces people to get right to the end of the process before sharing any of it, it is intended that Octopus will encourage collaboration among researchers with different skills to get to the same end point without a retrospective linear narrative. The journal model can also drive questionable research practices by encouraging you to publish only those data and findings that support your initial hypothesis, or perhaps ignoring tangents and dead ends along the way, all of which will be available through Octopus.

Octopus: Eight types of publication

Octopus is structured with eight different publication types for different parts of the research process with each potentially linked to (multiple iterations) of the one before it in the chain.

The eight publication types are:

  • Problem
  • Hypothesis
  • Protocol
  • Data
  • Analysis
  • Interpretation
  • Real world use
  • Review

Note: the last of these, ‘reviews’ are peer reviews that can be attached to any other type of publication. They are treated and valued in the same way as all the other types of publication in recognition that reviewing is a research skill like any other that needs to be incentivised.

Alex describes it as like taking the concept of preregistration to its ultimate conclusion. Preregistration, and formal registered reports, mean that hypothesis, methods and analytic protocol are ‘preregistered’ prior to data collection, which helps to avoid questionable research practices such as HARKing (hypothesising after results are known) and P-Hacking (running different statistical analyses until a positive result is found).

Octopus is similar though doesn’t necessarily expect the same researchers to be working on each of the 8 publication types; researcher A might be a brilliant hypothesis generator but lack the resources or skills to collect data or devise a protocol to test that hypothesis, which can instead be undertaken by researcher B.

The prototype also includes a rating system for users to quickly share their judgments of the quality of research (in addition to peer review), and an option to ‘red flag’ concerns about plagiarism or statistical errors for example.

Improving the research environment

Fast: instant publication. Just click a button and as soon as approved by any co-authors it goes live.

Free: to read, access and publish.

Fair: publishing in smaller groups makes it easier to see exactly who is doing what, which is more meritocratic and increases accountability. If you are responsible for data analysis but there was a problem with data collection it doesn’t reflect on your analysis but allows you to get credit for the work that you did with the data that was provided.

Without the need for a narrative structure, there is no pressure to make data retrospectively fit a hypothesis and researchers can share more of their work, such as a small data set for example that can be built upon by others.

The main takeaway from Alex is that Octopus represents a fundamental change in emphasis, away from findings and impact to the *process* of good science. Of course research needs to be usable and useful but first it must it must be rigorous, and open, so that other scientists can bring their own professional skills to bear to build a solid research base from which important research findings will emerge.

A note on metrics

Alex acknowledges that metrics to measure research quality is hard, but the current system is based on proxy measures such as Journal Impact Factor and a couple of peer reviewers’ assessment which is still regarded as the gold standard, though flawed and unsustainable.

Octopus’ rating system is intended to identify the criteria of quality for each specific publication type, enabling multiple peers to rate on those criteria alongside more traditional reviews (which are certainly still important and explicitly incentivised as a publication type in its own right, with the potential for multiple in-depth critiques.)

Openness and transparency

Alex hopes that the openness and transparency of the Octopus model will encourage quality over quantity that is beneficial for different stakeholders throughout the research environment. It will improve reproducibility by making it easier for researchers to publish, for institutions to assess researchers and for funders to assess research and to identify potential funding priorities, in more specialised areas for example.

At the same time, Alex recognises some of the challenges of this approach, including quality control and potentially enabling ‘trolling’ from bad actors. Octopus is designed to minimise bias, so there aren’t any profile photos or first names or institutions on publications. The system uses ORCID or a user could look up someone on Google, but the intention is to mitigate immediate, unconscious bias. Using ORCID should also help to avoid trolling as it is connected to a professional identity and institution rather than the anonymous login of a social media platform.

Changing research culture

In many ways, developing the platform is the easy bit. Ultimately though, we need people to change the way that they work and to actually use the platform. Inevitably, though, as researchers and related stakeholders, we are all locked into the way we work. We all need to think about our place within the research environment. How can you publish your work and think in an ‘Octopus way’ (which, like preprints, does not preclude a journal publication further down the line)?

It doesn’t necessarily take long for things to change however, where a platform serves a real need, and Octopus has certainly generated interest and excitement across the sector, as evidenced by the funding from Research England and partnership with Jisc and the UK Reproducibility Network.

Call for user testing

We had a quick demo at the event in March, though the Octopus team are working hard with Jisc to rapidly iterate the platform ahead of official launch towards the end of June. They are very keen to engage with individuals or small groups for user testing in May, including non-STEM disciplines. They would need just 45 minutes of your time. Please get in touch with Nick Sheppard (n.sheppard@leeds.ac.uk) if you are interested.