Same data – different results? ConSci 2017 introduces AQMCS!

A question that I have always had, is what happens when you give different scientists the same data and ask them to analyses those data. Do different scientists come up with different answers? Do they ask different questions? How much does our scientific interpretations depend on individual perspectives?

In the Conservation Science course we set out to test this question in our activity “Same data, different results?” We used data from the Niwot Ridge Long-term Ecological Research Site – a montane site whose flora and environment have been monitored for decades to understand ecological processes in high-elevation mountain ecosystems. Students and tutors worked in small groups to complete a speed data analysis and write-up activity – a quick and exciting journey though picking a research question, deciding on the methods that best address it, opening the data present, interpreting what it all means, AND writing a one page manuscript.

We love quantitative analysis, and we don’t shy away from statistics and coding – in the past we have counted Pokemon to calculate different biodiversity metrics, we have tested island biogeography theory, and we have gone through live coding exercises in class. Among all those blog posts, we often say we are “getting quantitative” – well, it is an exciting time of the year now, and not just because the holidays are approaching, but because now we can say that we are quantitative! So quantitative, in fact, that we couldn’t resists day-dreaming about a conservation journal highlighting different quantitative methods. And of course, Gergana couldn’t resist making a logo, so we are proud to present AQMCS (pronounced ack-mecs),  Advanced Quantitative Methods in Conservation Science, our course fictional journal!

aqmcs

Our dream journal – Advanced Quantitative Methods in Conservation Science. Though an official impact factor hasn’t been calculated yet, we have observed many students honing their quantitative skills, which can be quite significant.

We all rushed to write up our reports and submit them to AQMCS’s editors – after all, the sooner you submit a manuscript, the sooner you will get your reviews back. We had to wait only a mere three weeks, so as far as journals go, we would like to commend AQMCS’s quick turnaround time. All of out five manuscripts got sent out to review (phew, no straight-away rejections!), and in the final Conservation Science session, we all got to experience the peer-review process firsthand! And though the journal might be fictional, let us assure you that our editorial boards scrutinised the manuscripts and explored them in detail, making for some intense peer-reviewing!

DSCN5669

AQMCS’s editors-in-chief, Myers-Smith and Keane, setting the scene for an intense round of peer-review.

Tasked with selecting just one manuscript for publication (publication in AQMCS is very competitive!), students worked away in small groups. Manuscript after manuscript, students thought critically of the studies’ key questions and findings, as well as the methods used to obtain them. Does the manuscript meet the journal guidelines (e.g. a one page format), is the science exciting and novel, does the take-home message contribute to our understanding of biodiversity change?

Some of the submitted manuscripts were over the one page limit – we would like to remind their authors that should they need to present extra information, they can do so in Supplementary Information, which would be published online alongside the manuscript, should it be accepted for publication. One manuscript did not include a methods section – given our journal’s strong methodological focus (it is after all, Advanced Quantitative Methods in Conservation Science), we recommend those authors to implement our suggested revisions (i.e. outline what methods were used and how they advance conservation science) and re-submit. Just like in the real world, co-authorship dynamics were interesting to ponder – who is first author, who is last? One group included an authorship contribution statement, which we appreciated.

Each editorial group presented the criteria they used in the selection process, and finally announced the manuscript selected for publication. It was a close race, and it was up to the last editorial group to break the tie between “Long-term study in Niwot Ridge, Colorado reveals greater increases in spatial and temporal homogeneity within, but not between sites” and “Unprecedented biodiversity changes in continental divide at Niwot Ridge, Co, US”. Though informative, the group felt that the title of the first manuscript was a bit too long, and in the end, they decided that in its next issue, AQMCS will publish “Unprecedented biodiversity changes in continental divide at Niwot Ridge, Co, US” – the first publication in this exciting new journal!

aqmcs2

Accepted for publication in AQMCS – “Unprecedented biodiversity changes in continental divide at Niwot Ridge, Co, US”!

The two activities, “Same data, different results” and the AQMCS peer-review session, stirred up many thoughts on biodiversity change, how we quantify it and how we can attribute it to different drivers. We also honed our speed writing skills, and finally, our critical thinking skills – a winning combination!

Isla has been collected the titles of the same data different results papers throughout the years and here they are below.

2017:

  • Unprecedented biodiversity changes at the continental divide at Niwot Ridge, CO, US
  • Long-term study in Niwot Ridge, Colorado reveals greater increases in spatial and temporal homogeneity within, but not between sites
  • Baby It’s Hot Outside – Change in alpha diversity over elevational gradients due to temperature increase?
  • Nowhere to go: Biodiversity change at Niwot Ridge, CO, US
  • Potential trends and effects of temperature on species richness on four sites

2016:

  • Evidence of adaptation of high elevation plant species to dramatic climate change
  • Increased variance in species richness over time in montane forest-tundra environments
  • Disturbance causes varying levels of species richness change across alpine latitudinal gradients
  • Human activities cause species declines and increases across elevational gradients
  • Higher temperatures decrease biodiversity in alpine habitats
  • Local mountain biodiversity increases by 7% over time

2015:

  • Evidence of high-elevation plant community shifts to dramatic climate change
  • Increased variance in species richness over time in montane forest-tundra environments
  • Disturbance causes varying levels of species richness change across alpine latitudinal gradients
  • Human activities cause species declines and increases across elevational gradients
  • Higher temperatures decrease biodiversity in alpine habitats
  • Local mountain biodiversity increases by 7% over three decades

You can also check out our 2015 blog post about the “Same data, different results” activity: Same data different interpretations?

I think this experiment is telling us that different scientists do make different interpretations when presented with the same data. You can check out this study that found the same result with analyses of football (soccer) data. We at AQMCS think that the way forward is to make sure our data, code and science are as open as possible, so that we can promote thorough investigations of data and transparent interpretations in the literature.

By Gergana, Isla and the Cons. Sci. 2017 class

5 thoughts on “Same data – different results? ConSci 2017 introduces AQMCS!

  1. Pingback: Team Shrub – 2017 in Review | Tundra Ecology Lab – Team Shrub

  2. Pingback: What kind of a conservations are you? The Future of Conservation Survey | Cons. Sci.

  3. Pingback: What kind of a conservationist are you? The Future of Conservation Survey | Cons. Sci.

  4. Pingback: Team Shrub – 2017 in Review – Team Shrub

  5. Pingback: Highlights of the 2018 Conservation Science course | Cons. Sci.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s