I just wrote a story for the Simons Foundation Autism Research Initiative about a new study of the intestinal bacteria of autistic children. A team of Australian researchers studied more than 50 children with autism and discovered no significant differences between their gut microbes and the bacteria present in the digestive systems of their typically developing siblings.
The study contradicts some earlier research that has found that autistic children have microbiomes that differ from those of other kids. For instance, one study concluded that autistic children have elevated levels of Clostridia bacteria, and another detected Sutterella bacteria in the guts of children with autism but not in typically developing controls. The new Australian study, however, found no differences whatsoever in the microbes present in fecal samples from children with autism and those of control children. In my story, I explore some possible reasons why this one new study may not match earlier ones, but in the course of doing my research and reporting, I began thinking about the larger issue of why we see so many contradictory research findings, across many different fields.
So let’s look more closely at this particular issue. The autism-gut microbe connection has been particularly hard to unravel, and there have been a number of contradictory studies. The research question seems like it should be straightforward: Do the guts of autistic children have different kinds or concentrations of microbes than the bodies of typically developing children? Scientists should be able to answer the question by taking a few simple steps: Collect bacterial samples from the guts of children with autism and children without, determine the type and number of bacteria present in each sample, and compare.
The trick is that when researchers are actually designing these studies, they have to make countless small decisions about how to collect and analyze data, each of which could affect the ultimate conclusion.
For instance, in studies of autism and microbes, investigators must decide what kind of control group they want to use. Some scientists have chosen to compare the guts of autistic kids to those of their neurotypical siblings while others have used unrelated children as controls. This choice of control group can influence the strength of the effect that researchers find–or whether they find one at all. As I write in my story:
Some experts say comparing the gut microbes of children who have autism with those of their unaffected siblings, rather than with unrelated controls, may make it more difficult to detect subtle differences.
“Autism is a complex etiology,” says Catherine Lozupone, a postdoctoral fellow in Rob Knight’s lab at the University of Colorado at Boulder. “There’s a genetic basis to it and there’s an environmental basis to it.”
Siblings share many genes and environmental exposures, would be expected to have similar microbiomes. In fact, some research has shown that whereas the gut microbes of children with autism are significantly different from those of unrelated controls, siblings have microbial populations that fall somewhere in the middle.
Scientists also know that antibiotics can have profound and long-lasting effects on our microbiomes, so they agree on the need to exclude children from these studies who have taken antibiotics recently. But what’s recently? Within the last week? Month? Three months? Each investigator has to make his or her own call when designing a study.
Then there’s the matter of how researchers collect their bacterial samples. Are they studying fecal samples? Or taking samples from inside the intestines themselves? The bacterial communities may differ in samples taken from different places.
Finally, there’s data analysis to contend with. As I explain in my story:
Some scientists compare the presence of specific species or genera, and others look for differences in families or classes of bacteria, which may obscure subtle differences at the finer, species level.
“It’s really hard to interpret microbiome results,” Lozupone says. “It’s a new field — we’re still trying to figure out how to analyze data.”
This is just one example from one field, but it provides a glimpse of how tiny decisions about experimental design can affect the outcome of a study–and begins to illuminate why studies may contradict one another. That’s not to say that researchers are deliberately trying to sway the results through their study designs, though unscrupulous scientists certainly could do so. The point is merely that it is difficult, especially in an emerging field where there is not yet a consensus about procedures and best practices, to know exactly how to conduct research and analyze data. Of course, none of this means that there’s not some actual objective “Truth” out there–it just means that the process of discovering it can be a messy one indeed.
Gondalia, S., Palombo, E., Knowles, S., Cox, S., Meyer, D., & Austin, D. (2012). Molecular Characterisation of Gastrointestinal Microbiota of Children With Autism (With and Without Gastrointestinal Dysfunction) and Their Neurotypical Siblings Autism Research DOI: 10.1002/aur.1253
Parracho, H. (2005). Differences between the gut microflora of children with autistic spectrum disorders and that of healthy children Journal of Medical Microbiology, 54 (10), 987-991 DOI: 10.1099/jmm.0.46101-0
Williams, B., Hornig, M., Parekh, T., & Lipkin, W. (2012). Application of Novel PCR-Based Methods for Detection, Quantitation, and Phylogenetic Characterization of Sutterella Species in Intestinal Biopsy Samples from Children with Autism and Gastrointestinal Disturbances mBio, 3 (1) DOI: 10.1128/mBio.00261-11