You don’t have to be a professional journalist or a scientist to have an interest in evaluating the accuracy of the information you receive – from newspapers, magazines, television, on-line, or anywhere else. As citizens, it is important to be able to evaluate the accuracy of information we receive, since we all have the potential to be actively involved in determining the direction of our government.

William Greider said in his book, “Come Home America” that “Democracy begins within the self by thinking and saying what we truly feel and believe”. Thus, a good understanding of what constitutes accurate information will help us in expressing or arguing our points of view with other people whom we would like to influence. More important, it will help us to develop a well informed point of view to begin with.

There is a very good explanation as to why money has been able to corrupt politics in our country and why so many Americans sit passively by while our elected leaders lead us into immoral wars and tilt the playing field in favor of those who shower them with money. That reason is that too many Americans are not good enough at independently evaluating information. Consequently, they have too great a tendency to believe what the corporatocracy and their government leaders condition them to believe through the propaganda that they spew out on television, radio, newspapers, etc.

I am not a journalist. But as an epidemiologist and public health worker since 1977, an ability to accurately evaluate the reliability of information has been essential to my job responsibilities for all of my working life. Therefore, I have had to think a great deal about this issue. In this post I would like to share what I see as some of the most important criteria for evaluating the quality of the information we receive.


Number of sources in the chain

I begin with this issue, not because it is the most important, but because it is useful in clarifying some terminological issues:

First hand information is information that we directly witness ourselves. Obviously, most of the information we receive that is of political importance is not first hand information. If we get the information directly from a person who witnessed the information first hand, then we have second hand information. If we get it from someone who interviewed the person who witnessed the information first hand, then we have third hand information. Etc., etc., etc.

As a general rule, the fewer sources in the chain of information, the more likely the information is to be accurate. The reason for that is that if any of the sources in the chain are inaccurate then the information itself is inaccurate. However, a relatively long chain of sources does not necessarily invalidate information, depending upon the reliability of each source in the chain. For example, consider:

In 2005 Senator Richard Durbin read a report from on the floor of the U.S. Senate written by an FBI agent who had witnessed torture at Guantanamo Bay. If we assume that the report by the FBI agent was accurate, then the torture represented first hand information to the FBI agent, since he had witnessed it directly. It was second hand information to Senator Durbin, since he had to rely on the accuracy of the FBI agent’s testimony. To a journalist who heard and reported Senator Durbin’s remarks it was third hand information (unless the journalist actually saw the FBI report), since the journalist had to rely on Senator Durbin’s speech. And to those of us who read the journalist’s report in a newspaper it was fourth hand information, since we had to rely on the journalist’s accurate reporting of Senator Durbin’s remarks. Fourth hand information probably doesn’t sound very good.

However, in this case I don’t see that as a problem. That is because it seems highly unlikely to me that an FBI agent would misrepresent something like that in an official FBI report, that Senator Durbin would misread or fabricate the report, or that a journalist would misrepresent Senator Durbin’s remarks. Any of this would have been very easy to dispute if it was false, and despite all the furor over the episode by rabid Republicans, nobody that I am aware of disputed the accuracy of what the FBI agent witnessed.

Nevertheless, it is well to remember that where a long chain of sources is involved, the reliability of each one of them needs to be considered.


Anonymous sources

Although sometimes it is necessary to use anonymous sources in an article, they tend to weigh against its accuracy. That is because they can’t be verified. Because the reader has little or no idea who the source is, it is very difficult for him/her to evaluate their credibility. In fact, it is generally impossible to even prove whether or not the author of the article fabricated the “anonymous sources”. Nevertheless, it is possible for a reader to make some assessment of the potential negative effect of anonymous sources on the accuracy of a journal article.

One of the best examples of the potential problematic use of anonymous sources was Judith Miller’s use of them in 2002 to help the Bush/Cheney administration make its case for war against Iraq. The problem is explained by Russ Baker:


How could a discerning reader have viewed this situation? One of best ways to assess the use of an anonymous source is to ask yourself why the source is anonymous and what possible motive the anonymous source would have for exaggerating or lying. In this case it was evident that the Bush administration was desperately trying to make a case for war at the same time that Miller came out with her anonymous sources, which she admitted were Bush administration sources (How could anyone have access to such information if not through the Bush administration?)

We should ask ourselves why these sources were anonymous. It should have been evident that they were saying what the Bush administration wanted us to believe. So they could not have had anything to fear from openly making their statements. The only other motive that comes to mind for being anonymous is that their information was false, exaggerated, or flimsy. If you provide information of such monumental importance (making a case for war) someone is bound to question how you arrived at your conclusion. If you’re not able to defend your information, the solution is to provide it anonymously, so that you can’t be questioned about it.

On the other hand, when Seymour Hersh came up with his scoops about the Bush administration making plans to go to war against Iran just the opposite situation applied. The information in Hersh’s article was an embarrassment to the Bush administration, and the anonymous sources could have been targeted for retribution if they had identified themselves. So in that case there was a perfectly plausible reason for remaining anonymous other than that the source provided false or flimsy information. That awareness should have made the information more credible to a discerning reader.

When anonymous sources are used, the integrity of journalist becomes all the more important. In the case of Seymour Hersh I have no problem accepting the information he provides whether or not the source is anonymous, since his previous reporting has shown him to be a journalist of exceptional integrity.


Quality of the source

Consideration of the quality of the source applies to each of the sources along the source chain, from the initial source to the person who reports it to the reader.

My preference is to base my evaluation of the source’s quality mainly on what I know of its past history. In many cases I know nothing of the source’s past history – in which case I judge it neither positively nor negatively.

In my opinion, many people place way too much value on the source’s “reputation” in “respectable” circles. I’ll tell you a personal story to demonstrate what I mean by that:

When I was a resident in Preventive Medicine and Public Health, I became interested in the use of the low carbohydrate diet in the treatment of obesity. So I did a lot of reading on that subject, as well as some research. The diet had become popularized through Dr. Robert Atkins and his books. However, highly “prestigious” medical journals either ignored the subject or wrote negatively about it.

Nevertheless, I found some very good research on the subject in some relatively obscure medical journals. The methodology used and the depth of knowledge demonstrated in explaining the subject in some of those articles was very impressive in my opinion, and I came to the conclusion that the diet should be given a lot more respect than it was given. I gave very little credence to the fact that it was mainly ignored or mocked in the most highly “prestigious” circles.

Why is that? Whereas it is true that the more prestigious medical journals tend to have a more stringent peer review process than the more obscure journals, that does not by any means mean that excellent research doesn’t sometimes appear in more obscure journals. There are many criteria that the most prestigious journals use in determining what articles to publish, including whether or not the research results fit in with current paradigms. If they don’t, they’re unlikely to be published in the most prestigious journals.

So I did some of my own clinical research on the subject, had some very successful results, published a manuscript in a very obscure medical journal, finished my residency, and then forgot about the subject. Many years later (I can’t remember exactly when, but here’s one study), favorable research on the diet did begin to become published in highly prestigious journals, which of course caused a resurgence of the diet in the American public at large as well.

The same principle applies to political news. Way too many Americans give too much credence to so-called “respectable” news sources simply because they receive national attention. It doesn’t matter how many times the source has been wrong – If it is on nation-wide TV it is generally though to be highly credible. Larry Summers, for example, is a former Secretary of the Treasury, so his opinion is highly sought after and probably always will be, regardless of how many times he turns out to be wrong.

Nor do enough people consider the bias that the good majority of corporate media sources demonstrate in favor of the wishes of their corporate masters.


Motivations

We generally don’t know the motivations of the sources who write the things we read. But when we do, or if we can ascertain them, it behooves us to consider what motivations a source may have for lying or exaggerating. As I said above, anything we hear from a corporate media source should be taken with a grain of salt while considering the potential bias of the corporations behind the news.

That rationale applies to medical research as well. In recognition of that fact, most mainstream medical or public health organizations that sponsor conferences now require all their speakers to begin their talks with a disclosure of potential conflicts of interest.

Long before this problem was widely recognized, I observed that epidemiologists (I’m sorry to say because this slanders my profession) who worked for tobacco companies were on the front lines of the effort to convince the public that there was no proof that cigarettes cause lung cancer. They lost that battle, but it should serve as a reminder to consider potential motivations and conflicts of interest of authors when reading anything.


Internal consistency

In reviewing medical literature I often come across journal articles in which the numbers in the tables don’t add up, or in which the description in the text contradicts the tables. That’s always a bad sign, and provides a good reason to lose confidence in the accuracy of the article.


Consistency with external knowledge

Especially for very important issues, we should always consider the consistency of what a person says with what is currently known about the subject. That is not to say that we should automatically disregard what a person says if it contradicts existing opinions or presumed facts. But certainly our suspicions should be aroused to the extent that the information is out of synch with what we already know (or think we know) about the subject.

One of the biggest examples of how a person can discredit herself by publicly spouting out lies is Sarah Palin’s appearance at the Republican National Convention of 2008. My God, she told so many whoppers that were easily checked out that she must have forgotten all about Google! Within days most of the whole world knew that she was an unrepentant liar. Because of a corporate news media willing to give her some degree of protection and a segment of the American population that is so right wing that they have lost all ability to recognize reality, she still managed to retain some popularity after that fiasco. But still, the lies that were exposed did great harm to the McCain-Palin ticket in 2008.

The Bush administration making the case for war in Iraq is another great example of inconsistency with what is known about a subject. One of many examples that could be provided on this issue involves Bush’s January 28, 2003 State of the Union address. In making his case that Iraq posed a nuclear threat to us, Bush said that in that address, “The British government has learned that Saddam Hussein recently sought significant quantities of uranium from Africa”. Yet, someone who was knowledgeable on this subject would have known that: the only U.S. official sent to Africa to check out this story (Joe Wilson) said that there was no indication that Iraq is buying yellowcake; French intelligence had told the Bush administration in October 2002 “Bullshit. It doesn’t make any sense”; and our own National Intelligence Estimate of October 2002 had said “Claims of Iraqi pursuit of natural uranium (i.e., yellowcake) in Africa are highly dubious”. And, on September 7, 2002, George Bush claimed that a new U.N. International Atomic Energy Agency (IAEA) report stated that Iraq was six months from developing a nuclear weapon – though no such report existed.

It pays to be aware of information of that kind when evaluating a case for war.


Detail

In the absence of known internal or external inconsistencies it is very helpful to have a lot of well explained detail from an information source. If a journal article says something to the effect that so-and-so agreed with me on this issue, that’s not much to go on. There is so little information provided in a statement of that nature that even if the person whom the author referred to later denies that he agreed with the author on the subject, the author can avoid being caught in a lie by simply saying that she misinterpreted the source.

Providing a full quote that demonstrates the claimed agreement would be much more valuable than merely asserting agreement. The reader can then evaluate the quote himself and come to an independent opinion on whether or not it really demonstrates agreement with the author. In that case the author is putting her reputation on the line. If the quote turns out to be seriously inaccurate (especially if it came from a written document) she can’t later claim misinterpretation. The more detail the reader has to go on, the more he can come to an independent and valid conclusion as to the accuracy of the author’s information.

I recently received an e-mail from President Obama, in which he asked for my support of his health plan. The e-mail gave the principles of the plan, including 1) Lower health care costs; 2) Choice of our own physician; and 3) Affordable quality health care for everyone. I am fine with the principles, but skeptical of the fact that so few details were provided. In particular, if the Obama plan intends to achieve affordable quality health care for everyone without providing a public option for everyone (meaning that many of us would be forced to deal with private insurance companies in order to obtain our “affordable low cost health care”), I would have very little confidence in the plan. The lack of detail in that case is troubling.


Coherency

Sometimes I read articles that are very difficult for me to understand. That is sometimes because I don’t know enough about the subject. Or, my difficulty in understanding the article could be because the author does not write very coherently. In either case, if I can’t understand an article I generally won’t have much confidence in its conclusions – unless I have a great amount of confidence in the author’s integrity, judgment and intelligence.


Verifiability

The issue of verifiability is closely related to that of detail because in general, the more detail that is supplied by an author the easier it is to verify what s/he says. The same can be said about the number of references. I’ll use the assassination of John F. Kennedy to illustrate some points on this issue.

Verifiable evidence that JFK was shot from the front
David Lifton, in his book “Best Evidence – Disguise and Deception in the Assassination of John F. Kennedy”, supplies hundreds of references and meticulous detail in making the point that Kennedy was shot from the front. This is of vital importance to the question of Lee Harvey Oswald as the lone gunman because the official finding of the Warren Commission places Oswald in the book depository behind Kennedy at the time of the shooting.

Lifton notes that three persons offered eye-witness accounts of the movement of brain material. All three say that the brain material and blood exited the President’s head to the left and rear of the head. Two of them were motorcycle police officers and were actually splattered with the material. One account was obtained from a contemporary newspaper article, one was obtained from Warren Commission testimony, and one was obtained from a book author.

Nine physicians and a nurse who treated the President at Parkland hospital are quoted (four in Warren Commission testimony, three in their official medical reports, one in a contemporary newspaper account, and Lifton doesn’t provide the source for the other two) as saying that the fatal wound produced a large hole (5-7 centimeters by one account) in the back right side of the head. The skull at the back of the head was noted to have “exploded outwards”. All of the physicians characterized this wound as an exit wound.

Lifton supplies way too much verifiable detail to be accused of making this stuff up, exaggerating it, or misinterpreting it. As noted above, the evidence can be found in contemporary newspaper accounts, Warren Commission testimony, and medical reports.

Autopsy evidence contradicting the evidence that JFK was shot from the front
But Lifton explains that the autopsy evidence contradicts the eye-witness accounts of the brain material and the Parkland doctors, indicating that the fatal bullet came from behind. So which is correct? If all the bullets came from behind, that is consistent with the lone gunman theory. If any of them came from the front (i.e. the grassy knoll area), that indicates a conspiracy.

The House Select Committee on Assassinations (HSCA) contradicts the Warren Commission – reports “probable conspiracy”
In 1976 a House Select Committee on Assassinations (HSCA) was established to reinvestigate the JFK assassination (among other things), releasing its final report in 1979. Based on newly available acoustical evidence that indicated the existence of a fourth shot which probably came from the front, the report contradicted the Warren Commission findings by concluding that the assassination was “probably a conspiracy”.

Nevertheless, the HSCA report continued to lay the blame principally on Oswald and exonerated the U.S. government in the conspiracy, saying that the fatal wound came from behind (i.e. the book depository where Oswald was supposed to be) and that the conspirator who shot at Kennedy from the grassy knoll area must have been conspiring with Oswald alone. In coming to the conclusion that the fatal shot came from behind Kennedy, the HSCA favored the autopsy evidence over the evidence provided by the doctors at Parkland Hospital. Lifton quotes from the HSCA on that issue:


In other words it all boils down to the opinion of the Parkland doctors versus the testimony of Secret Service agents – for whom many, including Lifton, have shown evidence for complicity in the assassination – to the effect that the body was not altered prior to the autopsy.

Evidence for alteration of the body prior to autopsy
On that score, much of Lifton’s book is devoted to showing the likelihood that the body was indeed altered prior to the autopsy and that the “constant vigilance over the body” claimed by the secret service agents provides little or no assurance that the body was not altered prior to autopsy. With respect to the “constant vigilance over the body”, Lifton notes several persons who saw what appeared to be Kennedy’s body brought into the morgue in a plain gray coffin, very different than the fancy bronze coffin into which Kennedy’s body was placed in Dallas after he was pronounced dead, and which was televised being unloaded from the plane that carried the body from Dallas to Washington. (Additional evidence for a fraudulent autopsy can be found in this post.)


Plausibility

Plausibility is closely related to coherency, and to consistency with external knowledge as well. But it also has to do with a certain type of common sense. We should always ask ourselves if what the author says makes sense to us.

Typically a journal article (or other information source) will contain a conclusion, along with additional information and maybe some reasoning to show how the author arrived at the conclusion. Sometimes the conclusion will be crystal clear to us, while it is unclear how the author (or speaker) arrived at the conclusion. Either few details are provided to back up the conclusion, the details that are provided to back up the conclusion don’t form a coherent pattern in our minds, or we have reason to doubt the veracity of the details. In order to arrive at a valid assessment of an article’s conclusions, we should always try to understand how the author used the information at her disposal to arrive at the conclusion. And we should figure out if the logic used in that process makes sense to us.

The Bush administration told us frequently that we must continue our war and occupation of Iraq because if we don’t fight them over there we’ll have to fight them over here. He never provided any further explanation on that bizarre statement. It made no sense to me whatsoever. Shipping our military half way around the world, to a country where no enemies existed before we invaded it is supposed to protect us from attack here? Same thing with Bush’s claim that we were spreading democracy to Iraq. What good will democracy do for them when we kill about a million of them, make refugees out of four million, and ruin their infrastructure?


Summary

There are numerous criteria that can be used to assess the accuracy of information that we receive. No single criterion is necessarily a deal breaker (although it can be). Rather, a reasonable evaluation of information accuracy depends upon an assessment of all the relevant criteria taken together as a whole.