Dr. Reddy's Pediatric Office on the Web^TM

Snake Oil: The Accuracy of Medical Information on the Internet

If you are reading this you probably know that the Internet -- particularly the World-Wide Web -- is a widely-used source of medical information for the lay public (patients and their families) as well as a great resource for medical professionals. However, there's a problem: there is no guarantee that the information you find on the Internet is accurate, much less useful.

There are many reasons for this.

• Technical

  The Internet was designed as an anarchy.

  It was not intended for scheduling travel, meeting people, or selling fake (and sometimes real) drugs via junk mail. It was designed to allow computer A on a network of computers to send data to computer B -- even if half of the intermediate computers were nuked. (Really. The Internet started as a Cold War-era Defense Department project on reliable computer-to-computer communications. For that matter, digital computers themselves originated in part as a spinoff of the Manhattan Project.)

  The Internet has very loose central control.

  This was part of the redundant design, and became entrenched during the original expansion of the Internet to academia and industry, long before the general public had access.

  It is incredibly easy to connect new computers.

  Just about anyone can buy a computer, take it home, and have it connected to the Internet within 30 minutes (or less) of getting home.

  7-year-olds can set up Web sites.

  (And some of them do a better job than some of the professionals.)

  Sites come and go rapidly.

  (Some only last a few days...)

• Economic

  Computers are cheap.

  You can buy a new Internet-capable computer for well under $1000 these days -- and a used Internet-capable computer for only a few hundred. (And if you build them yourself -- as I do -- they're even cheaper.)

  Internet access is cheap.

  (As little as $9.95 a month. Some companies even give Internet access away for free -- as long as you put up with the online ads. And public libraries and schools have Internet-capable computers available to the public for free.)

  Web pages are cheap

  (In fact, they can be free, either from your Internet service provider or from companies who give away Web space in return for ads appearing on your pages.)

• Social

  (Almost) everybody uses computers

  (Just look at E-mail, Amazon.com, and eBay!)

  E-commerce was nonexistent in 1990

  (Again... look at Amazon and eBay!)

  Many patients and families spend more time on the Net than talking with their doctors

  (which explains a few things...)

The Dangers of Medical Misinformation

Can bad Web pages hurt someone?

Unfortunately, since the Internet is in essence an anarchy, there is no guarantee -- and no way to guarantee -- that the information you find on the Web is accurate. For some things, this isn't a problem: knitting instructions are most likely to be used by people who can easily detect errors, and the exact wording of the Monty Python Spam sketch is only of interest to us truly obsessed purists. (For you purists out there: the Green Midget Cafe did not serve lobster thermidor aux crevettes in a Provencale manner, with or without Spam. If you have doubts, look at the original Spam sketch on DVD.)

Medical information is another matter entirely -- bad information can hurt someone. An example (from Crocco, Villasis-Keever, and Jadad, Pediatrics (the journal of the American Academy of Pediatrics), vol.109 (issue 3), pages 522-523, March, 2002):

A 1-year-old boy developed nausea and vomiting, followed by diarrhea. He was seen at a local emergency room, diagnosed with an ear infection, and given antibiotics.

Two days later he still had diarrhea, and his parents took him back to the ER. They were told that he had gastroenteritis with dehydration, and were instructed to

• stop the antibiotics
• stop giving him solid foods, and
• rehydrate with soft drinks or fruit juice.

(This is not the advice I usually give in this situation: my advice, especially regarding fluids, is rather different -- and much more in keeping with generally accepted treatment for gastroenteritis and dehydration.)

When the parents brought their child home, they looked on the Web for information about diarrhea in children. They found a Web site with information "from a purported tertiary care pediatric medical center in the US" (I'm quoting the article here -- and, no, I do not know which medical center it was.) The instructions from the "purported tertiary care ... center" were to

• Give clear fluids ("These may include Pedialyte, flat cola, ginger ale, tea with sugar, Kool Aid, or Jello") every 2-3 hours.
• For children drinking regular milk: give no milk or milk products for 1 week.
• "Call your doctor if your child does not urinate for over 12 hours."

Over the next 7 days, the parents followed this advice. The little boy continued to have diarrhea and became increasingly weak.

On day 10 they took him to his regular doctor, who immediately put him in the hospital. He was reportedly "pale, irritable, and appearing wasted" but, fortunately, his electrolyte (sodium and potassium) levels were still in the normal ranges.

He was started on Pedialyte and restarted on solid foods, and was better enough to go home in two days.

His parents, of course, were rather upset -- especially as they adamantly maintained that they followed the ER and Web instructions carefully ... and "could not understand why their child had not improved... as they had followed the recommendations carefully." They were able to show their son's doctors a printout of the Web page they had relied on, and the doctors found that, indeed, "(they) had not misinterpreted the information on the site, which did not conform to standards of care outlined in any available clinical practice guideline (for the treatment of gastroenteritis)."

The parents gave permission for their child's case to be reported. The paper's authors claim -- and I agree -- that this case may be the first published case in which a patient was harmed because of bad advice posted on the Internet.

How much bad information is out there -- and how can you tell it's bad?

So, how does a Web surfer figure out what medical advice is good and what advice is bad? Not easily, unfortunately. And there are a number of studies showing that there is a lot of bad information out there. One example was published by Pandolfini, Impicciatore, and Bonati (Pediatrics, vol. 105 (issue 1), pages e1-e8, January, 2000). The authors were trying to see if "technical" criteria which any reader could check, such as:

• whether the author identified him/herself on the Web page
• whether the author's qualifications (MD? RN? others? none?) were given on the page
• whether the page shows the date on which it was last revised
• whether the page states that it is not a substitute for your own doctor's advice
• whether there are links to other pages on the same subject, and
• whether the page lists references

It didn't. Even worse, not only was there no correlation between technical completeness and completeness of information, or between technical completeness and accuracy of information, but there was also no correlation between completeness of information and accuracy of information. The subject they chose was cough in children. They looked at how complete each page they reviewed was as far as information on causes of cough, and at how accurate each page was on how to treat cough (the gold standards for accuracy were the recommendations of the AAP and the World Health Organization).

Of the 19 pages they looked at,

• only 4 pages met over half of the technical criteria.
• only 9 pages met over half of the completeness criteria.
• only 6 pages had more accurate than inaccurate information on treatment.
• only one page had more than 50% accurate information on treatment.

• Only 2 of the 3 most technically-complete pages had complete information on causes of cough.
• Only 1 of the 3 most technically-complete pages had more accurate than inaccurate treatment information (of 6 treatment options the authors considered one page had a 0 score, and one had a -4).
• Only 1 of the 3 pages with complete information on causes of cough had more accurate than inaccurate treatment information (the other two had -4 accuracy scores).

• that I did not include references or links to other resources (can you blame me, after these results?), and
• that I did not mention multiple-ingredient cough-and-cold medicines (which was also deliberate -- I don't think they work any more than AAP and WHO do, so I didn't waste the storage and bandwidth discussing them).

I might add that at the time the article was published in Pediatrics I received several calls from the lay press asking me to comment on the findings. I confess I was flattered by the attention. It is sad, though, that I would attract national attention merely because I was the only Webmaster reviewed in the paper who actually did everything right.

Unscientific and unsupported information

Web sites that oppose routine vaccination of children against common diseases

Antivaccination Web sites have been examined in the medical literature: one report, by Wolfe, Sharp, and Lipsky, (published in the Journal of the American Medical Association, vol. 287 (issue 24), pages 3245-3248, June, 2002), describes 22 Web sites examined by the authors. They found that:

• every one of the 22 sites claimed that vaccines, or their preservatives, cause "idiopathic" illness. ("Idiopathic" is a term we use in the medical profession to refer to diseases whose pathology (causes) cannot yet be figured out by us idiots.) Among the disorders these sites blame on vaccines and/or preservatives are:
  • autism,
  • SIDS,
  • immune dysfunction,
  • diabetes,
  • neurologic disorders including seizures, brain damage, learning disorders, ADD, and antisocial behaviour, and
  • atopic disorders (asthma, eczema, and allergic rhinitis)
  Some of these problems might be vaccine-related. However, there is no credible evidence that they are related to vaccine use. (Criteria for scientific evidence does not seem to bother most of the people who post material like this on the Web: they tend to pounce on any information that supports their beliefs, while dismissing any evidence that they are wrong as having been biased by vested interests such as drug and vaccine manufacturers. I have wondered how they account for all the companies that have left the vaccine business in the last few years -- one reason we now have a shortage of some vaccines, including the influenza vaccine.)
• 12 of the 22 sites claimed that supposedly contaminated lots of vaccine are associated with an increased risk of adverse effects. Again, there is no credible evidence that this is the case. Occasional "bad" lots (due to possible bacterial contamination, or merely not as effective as they are supposed to be) are occasionally identified, and are very quickly pulled from the market, and so are vaccines that prove to have bad side effects, such as the first rotavirus vaccine after we discovered that some babies develop bowel intussusception.
• 11 of the 22 claimed that there is an increased risk of adverse effects with simultaneous administration of several vaccines at one office visit. No one has shown this to be true in a well-designed study.
• 21 of the 22 claimed that vaccines (or their preservatives) erode immunity and induce autoimmunity -- again, without credible evidence.
• 18 claimed that vaccine-induced immunity is temporary or ineffective. That is true for some vaccines (such as tetanus vaccine), but not in general -- most vaccines to childhood illnesses give lifetime immunity to those diseases just as natural diseases do, without having to have the illnesses.
• 21 sites claimed that adverse reactions are underreported.
• 16 sites claimed that we don't need to vaccinate against many diseases because their incidence has declined -- without mentioning that the incidence has declined precisely because we vaccinate all susceptible children. (If these people are right about this, then why aren't we still vaccinating against smallpox?)
• 15 sites promote homeopathy as an alternative to vaccination. There have been studies showing that homeopathy may be useful for some diseases, but none showing that homeopathy prevents infectious diseases.
• 20 sites claimed that vaccine policy is motivated by profit or other vested interests. Many, if not most, of these sites dismiss studies showing the safety and efficacy of vaccination -- if not because of study flaws, then because the studies were funded by vaccine manufacturers or by governments (who, as everyone knows, have hidden vested interests in vaccine manufacture and usage).
• 17 sites claim that mandatory vaccination is a violation of civil liberties. This might possibly be true, but I haven't heard many other people question the right of the state to protect the majority of its citizens from dangers posed by a minority -- including from crime or from communicable diseases. Would these people object to the quarantine of someone who had smallpox or anthrax?
• 7 of the sites claimed that vaccines are manufactured using aborted fetal tissue. Most vaccines are made by growing the viruses in culture, usually with animal tissue (such as eggs), or by recombinant DNA techniques, using baking yeast to hold the recombined DNA and produce antigens. Three vaccines (for chickenpox, hepatitis A, and rubella (German measles) are grown on cultured human tissue. The chickenpox vaccine was originally grown in cultures of lung cells from human embryos, but my understanding is that it is now grown using other human tissue cell cultures; the rubella and hepatitis A vaccine viruses are also produced with cultured tissue, and the cultures were originated from tissue samples taken from volunteers. (I'm still looking into where the tissue cultures come from, but they are not from wholesale harvest of human embryos.)

Web sites that promote unhealthy and illness-related behaviour

I am experienced at treating medically-unstable patients with eating disorders (anorexia and bulimia), and treat such patients on a referral basis.

Treating two or more eating disorder patients at once has always been difficult, because when we have two anorexics together they tend either to compete to see how much more weight they can lose, or to start swapping tips on how to avoid eating and how to continue losing weight. The Internet has made things much worse: there are numerous Web sites run by anorexics or bulimics that post advice on how to continue being anorexic or bulimic. In my opinion, that is not much different than posting instructions on how to commit suicide. I have never had one of my eating-disorder patients die, but I've certainly had some come close and I really don't need to see one succeed -- and neither do my patients, their families, or their friends.

How can we find good medical information on the Web?

It's not easy. An awful lot of Web-based medical information isn't worth the time it takes to download. The problem is sorting out the wheat from the chaff.

And you can't depend on external forces to get rid of bad medical information. Not only is regulating the Web legally difficult in most democratic countries (it's certainly unconstitutional in the United States), but it's also technically impossible. (Remember: the Internet was designed to transmit information reliably even under nuclear attack.) Suing people who harm others by posting bad information may get rid of some of the garbage, but it may also drive the good medical Webmasters off the Web (there's a reason for that disclaimer at the end of every one of my Web pages...) -- and, given the international nature of the Internet, where would one file such a suit?

There are some things you can do to lessen the chances of running into bad information.

Ask your doctor to suggest sites s/he thinks are well-written and accurate.

Most of us are happy to recommend good sites -- whether or not we wrote them ourselves (*smile*).
(Seriously, one reason why my Office on the Web exists is so that my patients and parents would have a source of information on common problems that I felt comfortable steering them to. When I first set up this site there weren't that many resources on the Web; now there are many more, but I still start parents out here because I know -- and they know, too -- that what I say here is exactly what I would tell them in person.)

Browse the medical professional organizations' Web sites.

Most of the professional-organization sites have sections written specifically for patients and families. The American Academy of Pediatrics Web site is one good example. (And, yes, I am a Fellow of the AAP -- but I was recommending their site long before I became a Fellow.) The information on the professional-organization sites has been reviewed by members of the organization for accuracy, and the sites are updated regularly. They also check the accuracy of many or most of the sites to which they link.

Browse public-health Web sites...

including

• the US Centers for Disease Control and Prevention (CDC)
• the US Food and Drug Administration
• the World Health Organization

Unless you actually are into conspiracy theories, public health authorities will prove to have up-to-date information on many health-related issues. CDC in particular is one of the best and most current sources of information on infectious diseases -- both domestic and for foreign travel. (My Web pages on infections and immunizations are based largely on information from the CDC and the AAP.)

Check each Web site you read for the author's name and qualifications and the date when the page was last revised.

Although this may not assure accuracy, it helps to know who wrote the page, how old or new it is, and what qualifications the author has to say what's said there.
The Health on the Net Foundation, based in Switzerland, is dedicated to improving both the usability and the accuracy of medical information on the Internet. They have developed a code of conduct, the HONCode, to which many medical Webmasters subscribe. The foundation reviews Web sites to insure that the code of conduct is observed, and Webmasters are not allowed to use the foundation's logo unless they meet the code's requirements, which include

• identifying the author of each page and stating his/her qualifications.
• showing the date of last revision
• stating that the page is not a substitute for professional care
• revealing the source of financing for the site, including any potential conflicts of interest (like drug-company-financed sites)
• an explicitly-stated policy on privacy for site visitors, and
• providing an electronic means (which may be E-mail or a feedback form) for a visitor to contact the Webmaster.

(The HONCode link here is to my site's listing at the foundation; I have subscribed to the code for many years.)

Be careful out there!

The Internet is a great place to find medical information -- but you need to be careful about where you find that information. When you have doubts, your doctor is the best person to ask for advice -- after all, your doctor knows you a lot better than a Webmaster does.

Search the Office for:

10 25 50 100 250 500 all Results

Copyright © 2005, 2006, 2007 Vinay N. Reddy, M.D. All rights reserved.
Written 04/29/05; last revised 09/17/07