Contact the Association for Medical Ethics with any questions, or to find out more about our organization. Click here to become a Member of the Association for Medical Ethics

Evidence Based Medicine (EBM) is a universally accepted system to objectively rate the validity of any medical study in terms of quality. For example, two different studies may have opposite conclusions about whether a new device is good for patients or not. Yet, one study may just be a doctor documenting he did 10 of the procedures (or cases) and said the results are good and patients happy. The other could be where patients were compared to a group that had a standard procedure, or maybe nothing at all, and then evaluated by a third independent party for comparing the groups. The second study's conclusion may be infinitely more valid and therefore predict what ones real chances of doing well with a new procedure are. Patients, and even many doctors, may think both are equal, but they are not. One is an objective valid study for predicting results, and the other may be one step above hearsay.

To make matters worse, the studies may have authors that receive hundreds of thousands, or even millions, of dollars from the company that makes the device, yet people only know of a vague disclosure, as its called, that just says the name of a company the author works with. This may create bias whether intended or not.

To address the first problem, namely the quality and predictive value of a study, we use Evidence Based Medicines five levels of rating for categorizing a study. We do this for EVERY study published in the English literature in peer reviewed publications as found on Pub Med, the NIH site (National Institute for Health, of the Dept. of Health and Human Services) for citing medical publications. We then rate the studies AS A GROUP under our AME system of 5 stars where 5 is the best and most predictable data, and 1 is the worst. For example, if more than 25% of the  studies supporting  a new procedure are rated as  the highest Level  I of Evidence Based Medicine , then this would be an OVERALL AME 5 star rating which has a excellent level of predictive value for success if one chooses to get the procedure. If on the other hand the studies were all the lowest levels, such as Levels IV and V, then this would get an overall rating of 1 Star * which has a very poor level of prediction for success. In that case, one should give very serious pause as to having such a procedure done.

OVERALL AME STAR RATING for Prediction of Success for a Surgical Procedure, Device or Test

Predictive value for success for procedure, device or test	Star Rating
Excellent	5 star: > 25% of all studies are Level I
Good	4 star: > 25% of all studies are Level II or better
Fair	3 star: > 25% of all studies are Level III or better
Poor	2 star: > 25% of all studies are Level IV or better
Very Poor	1 star: <25% of all studies are Level V or better

INDIVIDUAL AME STAR RATING for Individual Studies

Level I
Randomized, controlled clinical trials
This is the most objective and valid study results. This study has posted the highest level of quality for evidence based medicine.

Level II
Prospective non-randomized, comparative studies
This is the second most objective and valid study results.

Level III
Retrospective comparative or case controlled studies
This third level is not a reliable standard for determining the validity of a study
These studies look back historically and compare them with other cases in the past.  The problem with this system is that these past patients may not have been chosen or treated – or not treated at all – on equal standings.

Level IV
Case Series
This is where a group of patients have a particular treatment, and then observations are made about them.  This does not differentiate between the success or failure of the treatment compared to other treatments or no treatment at all.  For example, treating back pain for 10 days by drinking pomegranate juice may show all patients got better and conclude this should be the treatment for back pain.  However, usually all back pain gets better in 10 days no matter what one does.

Level V
Expert Opinion
This is a person’s opinion only.  It’s an editorial, and is considered the lowest level of evidence to support a position.  AME considers this to be comparable to hearsay.

Detailed Summary of INDIVIDUAL Studies Rating:

1.- All patients were enrolled at the same point in their disease course (inception cohort) with 80% follow-up of enrolled patients.
2.- A study of results from two or more previous studies.
3.- Patients were compared with a control group of patients treated at the same time and institution.
4.- The study was initiated after treatment was performed.
5.- Patients with a particular outcome ("cases" with, for example, a failed total arthroplasty) were compared with those who did not have the outcome ("controls" with, for example, a total hip arthroplasty that did not fail).

*The Journal of Bone and Joint Surgery (American) 85:1-3 (2003) © 2003 The Journal of Bone and Joint Surgery, Inc.

Introducing Levels of Evidence to The Journal

Note: Levels converted to Star rating as indicated in text.

James G. Wright, MD, MPH, FRCSC, Marc F. Swiontkowski, MD and James D. Heckman, MD

Click here for AME's Star Rating System Reports.