For the last few decades there has been a growing cultural phenomenon among jurors regarding forensic evidence. Reasonable and intelligent people confuse what’s on TV with what is physically in front of them. Unfortunately these assumptions have dramatic consequences every day in court rooms across the country. The phenomenon comes from the endless barrage of criminal investigation series on television. Shows like C.S.I., N.C.I.S., Criminal Minds, Law and Order, Bones, and Dexter all revolve around clever investigators who are able to catch horrible criminals by using almost magical scientific abilities. The phenomenon also called the “C.S.I. effect” is the belief that all the hocus pocus in daytime TV actually exists and is readily accessible to modern law enforcement agencies. At its core, the “C.S.I. effect” comes from everyday citizens blindly believing that accurate scientific methods exist to identify everything from ballistics (firearms and bullets), hair, fingerprints, blood spatter, breath tests, field sobriety tests, and even DNA evidence without knowing how these disciplines actually come to their conclusions. The “C.S.I. effect” is nurtured by real world forensic analysts and law enforcement with over confidence and biased data.

Let’s take fingerprint identification as an example. It’s assumed by the public that finger prints are unique to an individual and that a match is conclusive evidence of placing someone at a crime scene.

In actuality, there is a great deal of disagreement on how many points of similarity should be observed before a match is declared. There is also disagreement as to whether fingerprints are unique, but let’s take that assumption at face value. In fingerprint identification matches are made by finding similar points (the little squiggly curves and lines in your finger print) in a print taken from a crime scene with an already existing sample from a suspect. The first issue, how many points of similarity do we need? Do we need 5, 9, 14, or 21 points of similarity to get a clear match? Forensic scientists can’t seem to agree. The logical question is then: why don’t they just check 30 points to be sure? Unlike TV, most forensic labs have only a few people qualified to test samples. Most forensic labs also have large jurisdictions sometimes covering an entire state or region. Agencies are handling large numbers of samples with limited staff. Given that reality, it’s easy to see why law enforcement and forensic labs would be content to have a reduced number of points to make an identification. When called into trial, analysts testify they have so many hours of training or went to some institute of forensic science. They’ll discuss the procedures of their lab and how it’s double checked with other forensic scientists. The word science is thrown in as much as possible. In truth most forensic disciplines are not peer reviewed by outside groups and more importantly, neither is the data they rely on. Studies that back forensic research are often paid for and conducted by law enforcement agencies or labs that have a clear law enforcement bias. When cornered on inconsistencies the response is usually “well it’s an applied science and experts get better with more experience”. That kind of phrase is a red flag. Scientific results shouldn’t depend on subjective experiences. Once a method is known, a student and teacher should have the same statistical results using the same method. It’s not to say that there is no place for forensic studies. It is fair to say that forensics is more of an art than a science. So analysts who testify in court about fingerprints are making educated guesses that two pictures have similar points.

Another problem with fingerprints is the lack of data on how frequent certain characteristics are in a given population. For example, human beings look different. To find an individual we need to identify certain traits that identify that person from the rest of the species. So let’s take 7 individual factors to make an identification. The factors are: height, hair color, eye color, skin color, foot size, number of teeth, and number of fingers on each hand. There are some variances in these points of identification but many are so common they are not reliable for blind match. It may help an investigation, but does it really close the case to know a 5’9, brown haired, blue eyed, white person with 32 teeth, 10 fingers, and size 10 shoe may have been at the scene? Notice the sex of the suspect wasn’t even identified. This is called frequency testing. Much of the controversy revolving around forensic analysts is due to a lack of frequency testing.

Even for respected peer review sciences like genetic identification with DNA evidence, where frequency testing is supposed to be part of the process there are still problems. Ideally analysts can run allele frequency testing to see how common a particular allele is in a population. But the analyst will only check frequency if it’s requested. Law enforcement agencies will often not request DNA frequency testing because they assume (as do many jurors) that a match of alleles is a solid DNA identification. When law enforcement forget to ask for frequency testing we end up with DNA alleles matching but without an indication of whether an allele has a 99.99% chance of being unique to the individual or if the allele identified is extremely common and could match any sample from any person in the court room.

So why don’t defense attorneys just say all this to the jurors? They do, but one of the most difficult things to overcome when discussing these issues is the blind trust and romantic fascination people have for the Federal and State agencies that analyze samples. Jurors see the expert and think of Ducky from N.C.I.S., Dexter, or that lady from Bones. Now the defense attorney has the difficult job of telling 12 people that’s just a fictional story. The result, silent broken hearts in the jury box (for some people this is like being told about Santa Claus and the Easter Bunny), and the defendant is often blamed. It makes sense. What if Dexter or Ducky from N.C.I.S. were just using junk science to determine a fact? The good guys go from being educated investigators to mystic gamblers fighting crime. The certainty of being right is gone. Dexter isn’t as romantic if he’s torturing innocent people based on a hunch. To be fair, most forensic analysts are trying to do their part in a criminal investigation and generally have good intentions. That being said, good intentions don’t equate to reliable results or remove subjective inaccuracies.  The Washington Post recently ran a story about F.B.I. analysts exaggerating testimony about hair identification. The story stated “Of 28 examiners with the FBI Laboratory’s microscopic hair comparison unit, 26 overstated forensic matches in ways that favored prosecutors in more than 95 percent of the 268 trials reviewed so far.”

We may be tempted to think law enforcement still got the right guy because of other evidence, but already 4 cases have been exonerated. This means there is uncertainty, one may even say reasonable doubt in these identifications. The Washington Post also states “cases include those of 32 defendants sentenced to death. Of those, 14 have been executed or died in prison.” Hopefully there was other evidence to convict, but now a society that values justice and fairness must spend millions of dollars reinvestigating hundreds of cases across the country. Those resources don’t come out of thin air. Scandals like the F.B.I. hair analyst story are uncovered every few years and yet society still clings to the magic of forensic science. We can make reasonable fixes that both support law enforcement and protect defendants from voodoo methods of identification:

• If an area claims to be science, open all the data to the public and let it be peer reviewed by the scientific community as a whole.
• Teach scientists who are not obligated to law enforcement these techniques and see if their conclusions are statistically reliable.
• Hire scientists (people that understand statistics, biology, and chemistry) not analysts to conduct these tests (some forensic specialists have only a high school diploma). When a Defendant is evaluated for competency to stand trial in a criminal hearing a Doctor does the evaluation. Imagine if we let a high school graduate with a few classes in psychology determine a competency evaluation?
• Finally, if the government and law enforcement agencies want to use these methods let’s stop over working and under paying staff poorly trained analysts. Instead, let’s use some of that forfeiture money to properly hire the right number of people who have biology, chemistry, and statistical backgrounds to conduct these scientific tests.

Everyday forensic evidence is being used to convict people in the criminal justice system from in cases ranging from D.U.I. and drugs to rape and murder. Let’s do it right the first time so society doesn’t have to say “sorry, we destroyed your life, we weren’t as sure as we thought.”