Sometimes I think there might be such a thing as too much data.
I expanded my research program a few years ago from just taxonomy and systematics, into community ecology of insects. This meant I had to change the way I view fieldwork and sampling and data management and publication. Mostly, it changed the way I deal with samples, because in ecological analyses, it’s not just the presence or absence of a species that matters, but the relative abundance of species from place to place, time to time. Obviously – and this might just be the biggest difference in how “Taxonomist Me” collects compared to “Ecologist Me” – I can’t simply stop collecting a particular species when I have “enough”. To do so would mess up a lot of the quantitative analyses we need to do in order to answer our ecological questions.
Our ongoing Diptera work in the Northern Biodiversity Program is a fantastic example of the challenges with Big Data Sets. Two years of large scale sampling (plus a partial third), 12 main sites, 2 weeks per site, 6 Malaise trap samples, 54 pan traps, 54 pitfall traps and multiple sweep samples per site. And flies are the most abundant and diverse group of insects in the arctic. By a LONG way. Don’t let moth people or wasp people tell you any different. We are drowning in flies. I’m thinking about this today in particular because Sabrina and Élodie, our summer stars of sorting, are expanding and rearranging the drawers of NBP Diptera to make room for more material. Again. And not for the last time.
Field work is a delightful and high-profile, but very brief, component of a project like this one. The vast majority of the time and effort happens in the lab, as samples are sorted, specimens are processed, and species are identified and arranged for eventual data entry and analysis.
And it’s when the drawers of identified material start to pile up that we realize just how much work there is still to do.
A lot of us here in the Lyman (Meagan, Anna, Sabrina, Élodie, Amélie, Stéphanie, Terry) as well as some colleagues from the Canadian National Collection in Ottawa have already spent a lot of time identifying flies from the NBP’s arctic sites. We’ve found new species in some families, northern records of groups we didn’t expect to see up there, some intriguing patterns of change over space and over time in northern flies. And we’ve barely scratched the surface.
We’ll be mining this monster data set for years to come. And great things (taxonomic, ecological, genetic) will come out of it.
I just have to remind myself not to look at the entire enormous pile of insect drawers all at once. It’s, well, too scary.