Twice this year I’ve had the opportunity to spend three weeks, 7 days a week, 24 hours a day, interacting in the field with students. The first was on a Desert Ecology field course in April-May with 20 students and 3 other staff; the second was my recent trip to the Yukon Territory with two of my own students from the lab. One of the great things about being with a group of students 24/7 is that it gives you the opportunity to go beyond the usual set of classroom questions about the material, or the exams, or the expectations of the course or project. Sitting around waiting for meals, or going on a hike, or waiting out a rain storm, or a five-hour drive to the next campground or collecting spot gives lots of time for deeper conversations.
I’ve had a lot of interesting exchanges with some of the students about the nature of science, specifically the nature of questions and answers. It was an eye-opener for both them and me. One of the great misconceptions that many undergrad students carry through their programs (thanks, mostly, to people like me — professors) is that science is about finding the answers. And a good scientist is someone who has lots of good answers. The problem is that science doesn’t always work that way; only exams do. In the world of science, finding the answers is sometimes easier than coming up with the questions.
I’ve always told my grad students that a good thesis project should generate more questions than answers. Because in the process of answering the original question, a good scientist will discover new questions along the way. And not all the new questions are big, or earth-shattering, or even answerable. But I bet a lot of them are good questions just the same. But grad students start out as undergrad students who, in too many courses, are dispensed a series of questions and answers and are expected to generate answers to another set of questions at an appointed and future time, usually a final exam. Sometime between being an undergrad and a grad student or other flavour of researcher, this view of science has to be at least partly deprogrammed. A few episodes on our desert field course brought this home to me.
We had a Research Day. Here’s how it worked: each group of four students had to come up with a question about desert ecology — patterns, gradients, interactions — and figure out how to collect the necessary data to address that question using the rudimentary tools in our trailer, and their brains. They had to analyse their data and give a group presentation about their question. They had about a day and a half to do this. And they all came up with great questions. And they all showed great ingenuity in figuring out how to collect the data they needed using string and rulers and sticks and compasses and thermometers and binoculars and brooms and plastic bags and flashlights. And several of them were disappointed in how their projects turned out. Why? Because they didn’t find “significant results”. What they meant by that was that the analyses of their data did not produce a difference that was statistically significant at a probability value of less than 0.05. Sigh.
We (the Desert Ecology staff who spent the day drinking tea and reading and wandering around checking in with the groups and cooking the supper) disagreed. We knew they did come up with something significant – in less than 36 hours they thought up a good research question out of thin air, they devised a way to answer that question, and they set about doing it. How is that not significant? Especially when you only have a few hours to collect data. Although we didn’t tell them this in advance, the assignment was about finding the question and figuring out how to answer it. That’s science. In that sense, they all passed with flying colours. The students, on the other hand, had been so conditioned to think that the game of science is about finding answers (“significant” answers) so some of them felt like they had failed. I sometimes wonder if requiring undergrad students to take a basic stats course, in the absence of a good course about science culture, does more harm than good. Of course, having uttered that blasphemous statement, I can hear the teeth gnashing and hair pulling now. I’ll stand behind it though. We should be cautious about launching students on a misguided quest for 95% significance too early in their careers without also telling them that that number is not the Holy Grail, it’s just one tool in a big toolbox of science.
One exchange in particular from the Desert Ecology course sticks in my mind. I was talking after supper one night with a student who had impressed me throughout the course because she frequently came up with really interesting insights and observations. She asked me for some career advice because she wasn’t sure about potential career paths other than grad school. Now I recognize that there are many options for good scientists besides grad school, but I was curious to know why she wasn’t considering it.
“I’m worried that I can’t come up with any good questions”
Let me put that answer in context: this same student had, in a series of conversations with me and others over the previous few days, generated probably enough good research questions to successfully keep a lab of 3–4 grad students doing productive research for 5 years. Those conversations had plotted out not just a question, but a research program. Granted, many of these questions grew out of group discussions and other students had input, but that’s the way science is supposed to work anyway.
We did a lot of the same thing on the Yukon field trip, the three of us. We walked around looking at things. We collected data, to be sure, but we won’t really know what we have until we get the specimens back to the lab and sorted, mounted and identified. But one things we did a lot was to talk about questions. We were all asking questions (even me): questions about patterns in fly diversity up here; about whether or not flies become active later in the day up here than down south (even with 21 hours of daylight); about why some flies are over on the willows and others in the sedges, even though they eat neither; about why those little shiny flies on the rotting mushrooms move their wings in that way, and are they all males?; about why taxonomy and ecology still often persist in acting like they’re completely unrelated fields; about how many species and how many specimens we might need for a good, reliable molecular phylogeny.
Finding answers is rewarding and satisfying, but who knows — maybe it’s also secondary. Being a good scientist isn’t just about finding the “right” answers; it’s about asking good questions.