Study Buddies

One of the things that strikes me as interesting about the way science is depicted in fiction is that it's rarely presented in terms of what is known, but rather it tends to be symbolized by its trappings. We know a person is a scientist not by what they say (unless it's "I'm a scientist."), but by their environment or what they wear.

A tour through a movie lab involves a whole lot of glass containers full of blue liquid which eventually turns a sympathetic character into a monster. Generally, there's not a lot of, "Oh my God! That idiot just drank my toxic copper sulfate solution! Copper Sulfate is a useful fungicide and is what they use to test your hemoglobin levels! Get a doctor, he's seizing!" Of course there wouldn't be anything like that in movies. Sure, sometimes you're in the mood for an informative science lecture about gamma rays. Really though, most of the time you just want to see Hulk smash (second one was decent).

Where this becomes an issue is when non-scientists get blinded by incomprehensible data. the average person wouldn't be able to toss a study out on its ass and call it cargo cult science just by flipping through it. In fact to a lawyer, a banker, a street vendor, or to *cough* a science journalist any kind of layman a crap study looks the same as a good one: Lots of graphs, equations, and sciencey type er... things. Yet sometimes, the graphs and diagrams are actually diversions. It's like someone got the idea, "Hey! let's throw lots of data in here and maybe no one will notice that we fail to prove our point!"

Take, as an example, just one of the many things wrong with the "ZOMG! 9/11 was an inside job!" study I mentioned a while back. They spend, if I recall correctly, ~25 pages talking about the properties of a sample ostensibly obtained from the day of the attacks. Yet, they don't go through a lot of trouble to ensure this is in fact dust from the Center. A lot of things got blown off by the wind that day, and they don't have any real way of knowing if the substance they claim to have found was from the buildings or blown off something else near the buildings. Then they do strange things, like treat a "control" paint chip in solvent and compare the effects. Yet, there are more than a few formulations for a paint, and more than few things it could be besides paint. Don't even get me started on the volatility of paint binders and the potential differences between old chips with evaporated solvent and new paint chips with residual solvent. I don't attack the experimental data proving the composition of the dust because I don't have to. The study lacks one of the most important things in science: rigor.

When a scientist say, discovers a new insect repellent. It's not enough to spray one piece of food with water and one with the repellent and put both on an anthill and then say it works. They might do this of course- if only to get a rough idea of what to expect and to help form a hypothesis, but they wouldn't get heavy into collecting data just yet.

In a simplified example involving one ant, they would take the two pieces and place them equidistant from the place the ant is going to be released (for the purposes of this example, I want you to imagine a tiny ant cage, mainly because I find the idea amusing). The two pieces of food are weighed to be equal and sprayed with the same amount of both the water (as a control) and the repellent. The ant is then released. This is then repeated hundreds of times until it becomes apparent that the ant is actually repelled. But wait! The light bulb in the test chamber makes one side brighter! There's a draft that may be pushing the scent one way and not the other! The ant is a lefty! A whole host of factors that have to be controlled for can easily render any experiment meaningless if you're not careful.

So how do we know when a scientific study is any good? Mostly, you don't- we have to rely on what scientists tell us- unfortunately. It is unfortunate that we can't always make truly informed decisions. This isn't much different from relying on a lawyer to tell you the law. There are a lot of things you can understand on your own. It may even be in your best interests or fun for you to learn those things. However, you wouldn't defend yourself if you were accused of murder. In a serious situation- you eventually resort to faith in an authority. Even if that authority isn't as competent as you would like. This is the unavoidable consequence of not being able to specialize in everything. The good new is that you don't have to trust just one person.

I once read a comment somewhere saying that cutting scientific funding only affects 0.000001% of the population. Being a complete geek, I knew he was full of shit. That would mean that in a population of ~300,000,000 Americans, there are 3 scientists. In truth scientist PhD's in the US come in at around 0.2%. By contrast construction workers come in at around 5%. They number in the hundreds of thousands, and that number only gets bigger when you take into account science is an international effort. It is a part of every scientist's job to hold others in the community to the standards of their field. So while a scientific consensus may eventually turn out to be wrong, by and large, trusting isolated cranks who don't interact (frequently by choice) with the community at large is not the wiser decision.