Stanley Feld M.D., FACP, MACE Menu


How Should Healthcare Quality Be Measured?

Stanley Feld M.D., FACP,MACE

 The U.S. Preventive Services Task Force’s (USPSTF) grading system for measuring the quality of healthcare is an wrong. It results in a way to limit physicians’ judgment and treats medical care as a commodity. It enables a computer program to judge if physicians have followed an algorithm to treat patients.

 “The U.S. Preventive Services Task Force (USPSTF).[57][citation needed has developed grading systems for assessing the quality of evidence for making judgments about treatments. 

  • Level I: Evidence obtained from at least one properly designed randomized controlled trial.
  • Level II-1: Evidence obtained from well-designed controlled trials without randomization.
  • Level II-2: Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group.
  • Level II-3: Evidence obtained from multiple time series designs with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence.
  • Level III: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.”

The grading system is wrong. I will lead to shabby medical care. If quality of care should be measured, it should be measured by using Evidence-Based Behavioral Practice evaluations

 “Evidence-based behavioral practice (EBBP) "entails making decisions about how to promote health or provide care by integrating the best available evidence with practitioner expertise and other resources, and with the characteristics, state, needs, values and preferences of those who will be affected.”

Empirically supported treatments (ESTs) in some clinical settings are defined as "clearly specified psychological treatments shown to be efficacious in controlled research with a delineated population" [3]

Only when physicians’ clinical judgment and observations are included in the assessment of their quality of medical care should evaluation of that quality of care be measured.

The narrow criteria of the USPSTF will not define quality. It will only serve to restrict access to care and penalize physicians for using clinical judgment and consumers’ from receiving medical care.

Suddenly, it becomes easy to see how difficult it is to assess the quality medical care.

There is little question that an occasional physician practices terrible medicine. This is obvious to the medical community. The mechanism for improving a bad physicians quality of care is in place but not well executed.

Few, especially healthcare policy wonks, seem to understand the difficulty of assessment of medical care.

It should be easy for policy wonks to understand the limitations and criticisms of evidence based medicine.

Yet the Obama administration regards evidence-based medicine as measured presently as the gold standard of clinical practice,

Limitations and Criticisms of Evidence-Based Medicine (EBM)

  • EBM produces quantitative research, especially from randomized controlled trials (RCTs). Accordingly, results may not be relevant for all treatment situations.[67]

This is obvious to most physicians.

  • The theoretical ideal of EBM (that every narrow clinical question, of which hundreds of thousands can exist, would be answered by meta-analysis and systematic reviews of multiple RCTs) faces the limitation that research (especially the RCTs themselves) is expensive; thus, in reality, for the foreseeable future, there will always be much more demand for EBM than supply, and the best humanity can do is to triage the application of scarce resources.

The reasons for EMS shortcoming are listed below. The list is not complete.

  • Because RCTs are expensive, the priority assigned to research topics is inevitably influenced by the sponsors' interests.
  • There is a lag between when the RCT is conducted and when its results are published.[68]
  • There is a lag between when results are published and when these are properly applied.[69]
  • Certain population segments have been historically under-researched (racial minorities and people with co-morbid diseases), and thus the RCT restricts generalizing.[70]
  • Not all evidence from an RCT is made accessible. Treatment effectiveness reported from RCTs may be different than that achieved in routine clinical practice.[64]
  • Published studies may not be representative of all studies completed on a given topic (published and unpublished) or may be unreliable due to the different study conditions and variables.[71]
  • Research tends to focus on populations, but individual persons can vary substantially from population norms, meaning that extrapolation of lessons learned may founder.
  •  Thus EBM applies to groups of people, but this should not preclude clinicians from using their personal experience in deciding how to treat each patient. One author advises that "the knowledge gained from clinical research does not directly answer the primary clinical question of what is best for the patient at hand" and suggests that evidence-based medicine should not discount the value of clinical experience.[56] Another author stated that "the practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research."[72]
  • Hypocognition (the absence of a simple, consolidated mental framework that new information can be placed into) can hinder the application of EBM.[73]
  • Valid enthusiasm for science should not cross the line into scientism, losing critical perspective.
  •  Although clinical experience and expert opinion are insufficient by themselves, neither are they valueless, as EBM fervor that approaches scientism sometimes tends to paint them.

This last point is repetition of a very important shortcoming.

  • An informed clinician can weigh confounding variables in a clinical case and decide that following a population-based guideline to the letter feels inadequate for the situation. Thus clinical backlash against "cookbook medicine" is not always misguided, and "guidelines are not gospel."[74]
  •  Conceptual models, by having fewer variables than always-multivariate reality, face limits of predictive accuracy, just as even the best supercomputer simulations cannot predict the weather with 100% accuracy, whether because of the butterfly effect or otherwise.
  •  Thus, just as clinical judgment alone cannot give epistemological completeness, neither can RCTs and systematic reviews alone.”

The answer to the reader’s last comment and question, “I believe a carrot and stick approach may be necessary with more carrot and less stick.  Your thoughts?” is

I believe that government must learn how to evaluate quality medical care accurately, if they want to base healthcare payments on the quality of medical care. Presently, the government is far from achieving that goal.

It could also be that measuring quality medical care is not President Obama’s goal.

 The opinions expressed in the blog “Repairing The Healthcare System” are, mine and mine alone

Please have a friend subscribe






  • Thanks for leaving a comment, please keep it clean. HTML allowed is strong, code and a href.