Measures That Matter to Primary Care
The Measures That Matter are a suite of clinical quality measures that the American Board of Family Medicine endorses for simplifying and improving measurement of Primary Care. The Center’s aim in developing Measures That Matter for primary care practices is to better align assessment and payment policies with what patients and clinicians know to be valuable, to reduce burden, and to reduce burnout.
Measures That Matter
The Measures That Matter are a suite of four clinical quality measures that the American Board of Family Medicine endorses for simplifying and improving measurement of Primary Care. The Center’s aim in developing Measures That Matter for primary care practices is to better align assessment and payment policies with what patients and clinicians know to be valuable, to reduce burden, and to reduce burnout.
Measures That Matter Brochure
(pdf 278K)
Continuity of Care
Brochure
(pdf 219K)
Person-Centered Primary Care Brochure
(pdf 1.48MB)
On this Page
An Introduction to Clinical Quality Measures
The Measures That Matter are a suite of four clinical quality measures that the American Board of Family Medicine endorses for simplifying and improving measurement of Primary Care.
An Introduction to Clinical Quality Measures
The Measures That Matter are a suite of four clinical quality measures that the American Board of Family Medicine endorses for simplifying and improving measurement of Primary Care.
An Introduction to Clinical Quality Measures
The Measures That Matter are a suite of four clinical quality measures that the American Board of Family Medicine endorses for simplifying and improving measurement of Primary Care.
- Person-Centered Primary Care
- Continuity of Care
- Comprehensiveness of Care
Low Value Care
A More Meaningful Standard for Primary Care
Whole-person clinical quality measures are the underpinning of what matters in primary care. They are relevant to all communities, in all public health situations, and across all diseases, providing a way to quantify patient-centered quality care.
Measures That Matter focus on more personalized care, taking into account what matters to both patients and clinicians. While current clinical quality measures focused on disease specific care have value, they are not aligned with the foundations of primary care or the needs of patients, communities, and health systems.
Measures That Matter are designed for use across multiple levels of the health care system:
Primary Care Practices
- Meet patient needs by focusing attention on what matters
- Reduce burnout and burden by organizing practice around the reason they went into patient care
Employers
- Assess if the care they are purchasing is doing what patients, clinicians and payers have identified as what matters
- Require that systems support aspects of primary care that matter
Patients
- Participate in health care improvement
- Provide information important for caring for them as a whole person
Insurers and Healthcare Systems
- Identify where to invest to support those delivering high quality primary care
- Develop systems that support integrating, personalizing and prioritizing care
Current Projects
Gordon and Betty Moore Foundation Grant Funding
The Institute of Medicine labeled continuity of care a defining characteristic of primary care, one that Starfield and others demonstrated as essential to primary care’s positive impact on health equity, cost reduction, and improved quality of care. Described as an implicit contract between physician and patient in which the physician assumes ongoing responsibility for the patient, continuity frames the personal nature of medical care, in contrast to the dehumanizing nature of disjointed care. Building on the idea that knowledge, trust, and respect have developed between the patient and provider over time, allowing for better interaction and communication, continuity at the patient level is associated with a host of benefits.
The American Board of Family Medicine has been awarded funding by the Gordon and Betty Moore Foundation to demonstrate that continuity is related to better diagnosis, and to specifically demonstrate how a clinician-level Continuity of Care measure is a valid and robust measure that can be associated with reduced diagnostic errors and improved quality of care. We hope to demonstrate this relationship not only in the diagnosis of cardiovascular disease and its risk factors (e.g., Diabetes and Hypertension), but also for Cancer and Infectious Disease, two other core areas of interest for the Foundation. Second, we aim to advance the Continuity of Care measure through the CMS Merit-Based Incentive Payment Program.
The overall Aims of the grant include:
Aim: Estimate the impact of continuity on intermediate predictors of cardiovascular outcomes influenced by timely and accurate diagnosis. These predictors are high value treatment for early and on target diagnosis of comorbidities, including metformin, statins, and ACE-inhibitor use
Aim: Refine ABFM’s continuity of care quality measure for submission to CMS’ MUC list for endorsement and implemented in public and private payment and accountability programs
For more information on the grant deliverables, contact: Jill Shuemaker.
The Continuity of Care Measure
The Continuity of Care measure was developed in collaboration with the Robert Graham Center. Continuity of Care is defined as seeing the same primary care clinician over time and remains one of the pillars of a high functioning health care system. The Continuity of Care measure has been vetted through the National Quality Forum’s (NQF) rigorous endorsement process and received NQF endorsement in 2021.
High care continuity is shown to improve patient outcomes and physician well-being and is associated with decreased health care costs including total costs, ED costs, inpatient costs, primary care costs, and costs for specific conditions or treatments. It is also associated with decreased health care utilization such as ED visits and hospitalizations. As a byproduct of building a continuous, trusting relationship over time, issues of equity and social risk can be addressed.
The Continuity of Care measure is available in the PRIME Registry, a qualified clinical data registry (QCDR) open to all primary care clinicians and is endorsed by CMS for use in the Merit-Based Incentive Payment System (MIPS) quality payment program. The PRIME Registry supports MIPS and other CMS quality payment programs.
For measure specifications, please click here.
The Person-Centered Primary Care Measure (PCPCM) Patient Reported Outcome Performance Measure (PRO-PM)
The PCPCM PRO-PM is a patient reported measure of exemplary primary care that has been developed in collaboration with the Larry A. Green Center based on extensive development work with patients, clinicians and health care payers. The PCPCM PRO-PM has been vetted through the National Quality Forum’s (NQF) rigorous endorsement process and received NQF endorsement in 2021.The measure is also the winner in the Patient-Reported Outcomes category of the NQF Next-Generation Innovator Abstract Award.
The PCPCM focuses attention and support on the integrating, personalizing, and prioritizing functions that patients and clinicians say are important. A measure based on these principles may reduce both the de-personalization experienced by patients, and the measurement burden, burnout and crisis of meaning experienced by clinicians.
The PCPCM uses a survey to ask patients to assess 11 distinct yet highly interrelated items regarding their assessment of the care they receive. The 11 items were developed with input from hundreds of patients and physicians, and are associated with better personal and population health, equity, quality and costs.
The PCPCM has been approved for broad use in the CMS Merit-based Incentive Payment System (MIPS) Quality Payment Program (QPP). Additionally, it has been approved:
- As part of the Family Medicine measure set
- As part of the Internal Medicine measure set
- As part of the Optimizing Chronic Disease Management MVP for CY 2023
For measure specifications, please click here.
The Comprehensiveness of Care Measure
The Comprehensiveness of Care measure was developed in collaboration with the Robert Graham Center. Comprehensiveness is lauded as 1 of the 5 core virtues of primary care. When measuring associations between variations in comprehensiveness of practice among family physicians and healthcare utilization and costs for their Medicare beneficiaries, we found that increasing family physician comprehensiveness of care, especially as measured by claims measures, is associated with decreasing Medicare costs and hospitalizations.
The Comprehensiveness of Care measure is currently being tested in the PRIME Registry and is slated for submission for National Quality Forum (NQF) endorsement in 2023 and for the CMS Merit-Based Incentive Payment System (MIPS) Quality Payment Program (QPP) in 2024.
Value Care Measure
The -Value Care measure is in the conceptualization phase and is being funded through an AHRQ grant. Primary care is where most outpatient healthcare is delivered and where more than one-third of all visits take place. However; how primary care clinician decisions and behaviors affect total costs of care is being studied.
Trust Patient Reported Outcome Performance Measure (PRO-PM)
The Trust PRO-PM is in the conceptualization phase. Trust is a bedrock of medicine as an institution, as a profession and as a set of personal relationships between patients and clinicians and within care teams. Trust is also a key constituent of an organizations’ ability to delivery high-quality health care.
The Five Phases of the Measure Development Lifecycle
Phase 1: Conceptualization
Information Gathering
Measure development begins with information gathering – the measure developer conducts an environmental scan, develops a business case and requests input from a broad group of stakeholders including patients. The developer then narrows down the concepts to specific measures.
Phase 2: Specification
Draft Specifications
After the information gathering phase, we begin to draft the measure specifications. Measure specifications provide the comprehensive details that allow the measure to be collected and implemented consistently, reliably, and effectively. The specifications identify the population, the recommended practice, the expected outcome and determine how it will be measured. They also may include age ranges, performance time period and allowable values for medical conditions or procedures, code systems, descriptions.
Measure technical specifications will address the following questions: How will the measure be named? Does the name mean anything to people when they read it? Do they understand what that measure is about? What would the setting of the measure be (e.g., ambulatory office)? How will the data be collected? These questions have to be answered before testing begins.
Harmonization is all about reducing burden. Look at measures currently in practice and determine if there are places where our measure could be harmonized with the existing measure(s).
Phase 3: Testing
Measure Testing
Measure testing assesses the suitability of the quality measure’s technical specifications and acquires empirical evidence to help assess the strengths and weaknesses of a measure. Measure testing involves testing the components of the quality measure such as the data elements, the scales (and items in the scales if applicable), and the performance score.
There are two parts to measure testing: alpha and the beta testing.
Alpha testing helps identify early issues and often begins as early as the conceptualization step and is repeated during the development of the measure specifications.
Beta testing, which is also referred to as field testing, generally occurs after the initial specifications have been developed, and strives for representative sample sizes – multiple sites/settings. The primary purpose for beta testing is to understand the usability of the measure and to test the scientific acceptability of the measure.
After the testing ends, the results are analyzed with a return to the specification phase, or even the conceptualization phase, to rework the measure before testing again.
The PRIME registry is ABFM’s Qualified Clinical Data Registry (QCDR) and serves as our measure testing bed.
Phase 4: Implementation
Measure Implementation
The measure is then submitted for NQF endorsement (not a requirement for use by CMS) and for use in the 18 CMS quality reporting and payment programs.
What’s the difference between submitting to CMS versus submitting to NQF?
The National Quality Forum (NQF) submission is about the endorsement process where a consensus-based entity reviews the measure using five evaluation criteria to assess the measure on its own merit and independent of a CMS program. It essentially gives it that stamp of approval, and so endorsement/NQF submission is separate from CMS implementation. NQF endorsement is valued for measures in CMS programs, but it is not a requirement.
The CMS implementation process takes the measure from being in development to being actively used in 18 of the CMS quality payment programs (QPP).
Phase 5: Use, Continuing Evaluation, and Maintenance
Re-evaluation
This step ensures that the measure continues to add value to quality reporting measurement programs and that its construction continues to be sound. The regular reevaluation of measures is vital as the science and other factors are always changing (e.g., development of new clinical guidelines, new technologies for data collection, discovering a better way to calculate measure results). Continually reviewing the measure will ensure it remains relevant and meaningful. Measures that stop being useful are retired.
Interaction Among Measure Lifecycle Phases
The Measure development lifecycle is not a linear process. Once the measure is conceptualized, it can move throughout the various phases in the measure development lifecycle.
The following decision criteria throughout the measure development cycle ensures a measure meets the applicable standards before moving to the next phase:
Importance to measure and report—including analysis of opportunities for improvement such as reducing variability in comparison groups or disparities in healthcare related to race, ethnicity, age, or other classifications.
Scientific acceptability—including analysis of reliability, validity, and exclusion appropriateness.
Feasibility—including evaluation of reported costs or perceived burden, frequency of missing data, and description of data availability.
Usability—including planned analyses to demonstrate that the measure is meaningful and useful to the target audience. This may be accomplished by the Technical Expert Panel (TEP) reviewing the measure results such as means and detectable differences, dispersion of comparison groups, etc. More formal testing, if requested by CMS, may require assessment via structured surveys or focus groups to evaluate the usability of the measure (e.g., clinical impact of detectable differences, evaluation of the variability among groups).
Measure Specifications
Continuity of Care Measure Specifications
This is a process measure evaluating primary care physicians; for each physician, their denominator is all of the patients they saw during the evaluation period who had at least 2 PCP visits (could include visits to other PCPs), and the numerator is the number of those patients whose Bice-Boxerman Continuity of Care Index is >= 0.7. The Bice-Boxerman index is a validated measure of patient-level care continuity that ranges from 0 to 1; 0 reflects completely disjointed care (a different provider for each visit) and 1 reflects complete continuity with the same provider for all visits.
2022 Person Centered Primary Care Measure Specifications
The Person-Centered Primary Care Measure Patient Reported Outcome Performance Measure (PCPCM PRO-PM) uses the PCPCM Patient Reported Outcome Measure (PROM) a comprehensive and parsimonious set of 11 patient-reported items – to assess the broad scope of primary care. Unlike other primary care measures, the PCPCM PRO-PM measures the high value aspects of primary care based on a patient’s relationship with the clinician or practice.
Recommended Reading
A New Comprehensive Measure of High-Value Aspects of Primary Care
Rebecca S. Etz, PhD
Stephen J. Zyzanski, PhD
Martha M. Gonzalez
Sarah R. Reves, MSN, FNP-C
Jonathan P. O’Neal
Kurt C. Stange, MD, PhD
To develop and evaluate a concise measure of primary care that is grounded in the experience of patients, clinicians, and health care payers… Read the article.
The Lost Pillar: Does Continuity of Care Still Matter?
By: David Loxterkamp, MD
Continuity of care has long held a hallowed place in the halls of family medicine. Indeed, it is one of the 4 pillars of an ideal family practice, along with first contact, comprehensive, and coordinated care… Read the article.
More Comprehensive Care Among Family Physicians is Associated with Lower Costs and Fewer Hospitalizations
Andrew Bazemore, MD, MPH
Stephen Petterson, PhD
Lars E. Peterson, MD, PhD
Robert L. Phillips Jr, MD, MSPH
Comprehensiveness is lauded as 1 of the 5 core virtues of primary care, but its relationship with outcomes is unclear. We measured associations between variations in comprehensiveness of practice among family physicians and healthcare utilization and costs for their Medicare beneficiaries… Read the article.
Higher Primary Care Physician Continuity is Associated With Lower Costs and Hospitalizations
Andrew Bazemore, MD, MPHStephen Petterson, PhD
Lars E. Peterson, MD, PhD
Richard Bruno, MD, MPH
Yoonkyung Chung, PhD
Robert L. Phillips Jr, MD, MSPH
Continuity of care is a defining characteristic of primary care associated with lower costs and improved health equity and care quality. However, we lack provider-level measures of primary care continuity amenable to value-based payment, including the Medicare Quality Payment Program (QPP)… Read the article.
Interpersonal Continuity of Care and Care Outcomes: A Critical Review
John W. Saultz, MD
Jennifer Lochner, MD
We wanted to undertake a critical review of the medical literature
regarding the relationships between interpersonal continuity of care and the outcomes
and cost of health care… Read the article.
Interpersonal Continuity of Care and Patient Satisfaction: A Critical Review
John W. Saultz, MD
Waleed Albedaiwi, MD
We wanted to review the medical literature regarding the relationship
between interpersonal continuity of care and patient satisfaction and suggest
future strategies for research on this topic… Read the article.
Physician versus Practice-Level Primary Care Continuity and Association with Outcomes in Medicare Beneficiaries
Mingliang Dai, Ph.D., MS
Jill Shuemaker, RN, CPHIMS
Lars Peterson, M.D., Ph.D.
Robert Phillips, M.D., MSPH
YoonKyung Chung, Ph.D.
To compare physician versus practice-level primary care continuity and their
association with expenditure and acute care utilization among Medicare beneficiaries and evaluate if continuity of outpatient primary care at either/both physician or/and practice level could be useful quality measures… Read the article.
Primary Care Physician Characteristics Associated with Low Value Care Spending
Tyler W. Barreto, MD
Yoonkyung Chung, PhD
Peter Wingrove, BS
Richard A. Young, MD
Stephen Petterson, PhD
Andrew Bazemore, MD, MPH
and Winston Liaw, MD, MPH
Previous work has shown that $210 billion may be spent annually on unnecessary medical services and has identified patient and hospital characteristics associated with low value care (LVC). However, little is known about the association between primary care physician (PCP) characteristics and LVC spending. The objective of this study was to assess this association… Read the article.
Primary Care Physicians and Spending on Low-Value Care
Aaron Baum, PhD
Andrew Bazemore,MD, MPH
Lars Peterson,MD, PhD
Sanjay Basu,MD, PhD
Keith Humphreys, PhD
Robert L. Phillips,MD, MPH
Low-value services account for $75 billion to
$100 billion of U.S. health care spending (1). Primary care physicians
(PCPs) have been conceptualized as potential gatekeepers
for efforts to reduce low-value spending, but the share of
low-value spending directly related to their services and referral
decisions remains unclear… Read the article.
The Lost Pillar: Does Continuity of Care Still Matter?
By: David Loxterkamp, MD
Continuity of care has long held a hallowed place in the halls of family medicine.
Indeed, it is one of the 4 pillars of an ideal family practice, along with first contact,
comprehensive, and coordinated care. But what does it mean to the patient,
the doctor, and our sense of identity and purpose? Read the articleRead the article.
Continuity of Care bibliography
By: Zachary Merenstein
Pre-Med Student, University of Maryland
With guidance and assistance from Alison Morris, Mikel Severson, and Andrew Bazemore, ABFM & the Center for Professionalism & Value Download the Bibliography.
Comprehensiveness–The Need to Resurrect a Sagging Pillar of Primary Care
Tracey L. Henry MD, MPH, MS,
Eugene C. Rich MD
Andrew Bazemore MD, MPH
Journal of General Internal Medicine (2021)
The COVID-19 pandemic highlighted both the importance of primary care and the fragility of its current infrastructure in the United States (US). Within its first 2 months, stark reminders of racial injustice, unaddressed health disparities, and grossly inequitable access to healthcare further underscored the current lack and future importance of universal access to high-performing primary care… Read the article.
Clinical Quality Measures in a Post-Pandemic World: Measuring What Matters in Family Medicine (ABFM)
Jill C. Shuemaker
Robert L. Phillips
Warren P. Newton
The Annals of Family Medicine (July 2020)
COVID-19 altered the way the American public lived their lives; the way they worked, ate, socialized, traveled, and ultimately received their health care. Family Medicine largely closed its doors to face-to-face preventive and chronic care visits and made a large shift to telephone and online video visits… Read the article.
Our Partners
We are grateful for the collaboration and support of our current partners in this initiative.
- American Board of Family Medicine (ABFM)
- The ABFM Foundation
- The PRIME Registry
- The Robert Graham Center
- The Larry A Green Center
- Stanford University
- Virginia Center for Health Innovation
- American Academy of Family Physicians (AAFP)
Interested in opportunities to collaborate or support this initiative?
Contact Jill Shuemaker
Related Publications
A New Comprehensive Measure of Primary Care
Higher Primary Care Physician Continuity is Associated With Lower Costs and Hospitalizations:
More Comprehensive Care Among Family Physicians is Associated with Lower Costs and Fewer Hospitalizations:
Primary Care Physician Characteristics Associated with Low Value Care Spending:
An Introduction to Clinical Quality Measures
The Measures That Matter are a suite of four clinical quality measures that the American Board of Family Medicine endorses for simplifying and improving measurement of Primary Care.
- Person-Centered Primary Care
- Continuity of Care
- Comprehensiveness of Care
Low Value Care
A More Meaningful Standard for Primary Care
Whole-person clinical quality measures are the underpinning of what matters in primary care. They are relevant to all communities, in all public health situations, and across all diseases, providing a way to quantify patient-centered quality care.
Measures That Matter focus on more personalized care, taking into account what matters to both patients and clinicians. While current clinical quality measures focused on disease specific care have value, they are not aligned with the foundations of primary care or the needs of patients, communities, and health systems.
Measures That Matter are designed for use across multiple levels of the health care system:
Primary Care Practices
- Meet patient needs by focusing attention on what matters
- Reduce burnout and burden by organizing practice around the reason they went into patient care
Employers
- Assess if the care they are purchasing is doing what patients, clinicians and payers have identified as what matters
- Require that systems support aspects of primary care that matter
Patients
- Participate in health care improvement
- Provide information important for caring for them as a whole person
Insurers and Healthcare Systems
- Identify where to invest to support those delivering high quality primary care
- Develop systems that support integrating, personalizing and prioritizing care
Current Projects
Gordon and Betty Moore Foundation Grant Funding
The Institute of Medicine labeled continuity of care a defining characteristic of primary care, one that Starfield and others demonstrated as essential to primary care’s positive impact on health equity, cost reduction, and improved quality of care. Described as an implicit contract between physician and patient in which the physician assumes ongoing responsibility for the patient, continuity frames the personal nature of medical care, in contrast to the dehumanizing nature of disjointed care. Building on the idea that knowledge, trust, and respect have developed between the patient and provider over time, allowing for better interaction and communication, continuity at the patient level is associated with a host of benefits.
The American Board of Family Medicine has been awarded funding by the Gordon and Betty Moore Foundation to demonstrate that continuity is related to better diagnosis, and to specifically demonstrate how a clinician-level Continuity of Care measure is a valid and robust measure that can be associated with reduced diagnostic errors and improved quality of care. We hope to demonstrate this relationship not only in the diagnosis of cardiovascular disease and its risk factors (e.g., Diabetes and Hypertension), but also for Cancer and Infectious Disease, two other core areas of interest for the Foundation. Second, we aim to advance the Continuity of Care measure through the CMS Merit-Based Incentive Payment Program.
The overall Aims of the grant include:
Aim: Estimate the impact of continuity on intermediate predictors of cardiovascular outcomes influenced by timely and accurate diagnosis. These predictors are high value treatment for early and on target diagnosis of comorbidities, including metformin, statins, and ACE-inhibitor use
Aim: Refine ABFM’s continuity of care quality measure for submission to CMS’ MUC list for endorsement and implemented in public and private payment and accountability programs
For more information on the grant deliverables, contact: Jill Shuemaker.
The Continuity of Care Measure
The Continuity of Care measure was developed in collaboration with the Robert Graham Center. Continuity of Care is defined as seeing the same primary care clinician over time and remains one of the pillars of a high functioning health care system. The Continuity of Care measure has been vetted through the National Quality Forum’s (NQF) rigorous endorsement process and received NQF endorsement in 2021.
High care continuity is shown to improve patient outcomes and physician well-being and is associated with decreased health care costs including total costs, ED costs, inpatient costs, primary care costs, and costs for specific conditions or treatments. It is also associated with decreased health care utilization such as ED visits and hospitalizations. As a byproduct of building a continuous, trusting relationship over time, issues of equity and social risk can be addressed.
The Continuity of Care measure is available in the PRIME Registry, a qualified clinical data registry (QCDR) open to all primary care clinicians and is endorsed by CMS for use in the Merit-Based Incentive Payment System (MIPS) quality payment program. The PRIME Registry supports MIPS and other CMS quality payment programs.
For measure specifications, please click here.
The Person-Centered Primary Care Measure (PCPCM) Patient Reported Outcome Performance Measure (PRO-PM)
The PCPCM PRO-PM is a patient reported measure of exemplary primary care that has been developed in collaboration with the Larry A. Green Center based on extensive development work with patients, clinicians and health care payers. The PCPCM PRO-PM has been vetted through the National Quality Forum’s (NQF) rigorous endorsement process and received NQF endorsement in 2021.The measure is also the winner in the Patient-Reported Outcomes category of the NQF Next-Generation Innovator Abstract Award.
The PCPCM focuses attention and support on the integrating, personalizing, and prioritizing functions that patients and clinicians say are important. A measure based on these principles may reduce both the de-personalization experienced by patients, and the measurement burden, burnout and crisis of meaning experienced by clinicians.
The PCPCM uses a survey to ask patients to assess 11 distinct yet highly interrelated items regarding their assessment of the care they receive. The 11 items were developed with input from hundreds of patients and physicians, and are associated with better personal and population health, equity, quality and costs.
The PCPCM has been approved for broad use in the CMS Merit-based Incentive Payment System (MIPS) Quality Payment Program (QPP). Additionally, it has been approved:
- As part of the Family Medicine measure set
- As part of the Internal Medicine measure set
- As part of the Optimizing Chronic Disease Management MVP for CY 2023
For measure specifications, please click here.
The Comprehensiveness of Care Measure
The Comprehensiveness of Care measure was developed in collaboration with the Robert Graham Center. Comprehensiveness is lauded as 1 of the 5 core virtues of primary care. When measuring associations between variations in comprehensiveness of practice among family physicians and healthcare utilization and costs for their Medicare beneficiaries, we found that increasing family physician comprehensiveness of care, especially as measured by claims measures, is associated with decreasing Medicare costs and hospitalizations.
The Comprehensiveness of Care measure is currently being tested in the PRIME Registry and is slated for submission for National Quality Forum (NQF) endorsement in 2023 and for the CMS Merit-Based Incentive Payment System (MIPS) Quality Payment Program (QPP) in 2024.
Value Care Measure
The -Value Care measure is in the conceptualization phase and is being funded through an AHRQ grant. Primary care is where most outpatient healthcare is delivered and where more than one-third of all visits take place. However; how primary care clinician decisions and behaviors affect total costs of care is being studied.
Trust Patient Reported Outcome Performance Measure (PRO-PM)
The Trust PRO-PM is in the conceptualization phase. Trust is a bedrock of medicine as an institution, as a profession and as a set of personal relationships between patients and clinicians and within care teams. Trust is also a key constituent of an organizations’ ability to delivery high-quality health care.
The Five Phases of the Measure Development Lifecycle
Phase 1: Conceptualization
Information Gathering
Measure development begins with information gathering – the measure developer conducts an environmental scan, develops a business case and requests input from a broad group of stakeholders including patients. The developer then narrows down the concepts to specific measures.
Phase 2: Specification
Draft Specifications
After the information gathering phase, we begin to draft the measure specifications. Measure specifications provide the comprehensive details that allow the measure to be collected and implemented consistently, reliably, and effectively. The specifications identify the population, the recommended practice, the expected outcome and determine how it will be measured. They also may include age ranges, performance time period and allowable values for medical conditions or procedures, code systems, descriptions.
Measure technical specifications will address the following questions: How will the measure be named? Does the name mean anything to people when they read it? Do they understand what that measure is about? What would the setting of the measure be (e.g., ambulatory office)? How will the data be collected? These questions have to be answered before testing begins.
Harmonization is all about reducing burden. Look at measures currently in practice and determine if there are places where our measure could be harmonized with the existing measure(s).
Phase 3: Testing
Measure Testing
Measure testing assesses the suitability of the quality measure’s technical specifications and acquires empirical evidence to help assess the strengths and weaknesses of a measure. Measure testing involves testing the components of the quality measure such as the data elements, the scales (and items in the scales if applicable), and the performance score.
There are two parts to measure testing: alpha and the beta testing.
Alpha testing helps identify early issues and often begins as early as the conceptualization step and is repeated during the development of the measure specifications.
Beta testing, which is also referred to as field testing, generally occurs after the initial specifications have been developed, and strives for representative sample sizes – multiple sites/settings. The primary purpose for beta testing is to understand the usability of the measure and to test the scientific acceptability of the measure.
After the testing ends, the results are analyzed with a return to the specification phase, or even the conceptualization phase, to rework the measure before testing again.
The PRIME registry is ABFM’s Qualified Clinical Data Registry (QCDR) and serves as our measure testing bed.
Phase 4: Implementation
Measure Implementation
The measure is then submitted for NQF endorsement (not a requirement for use by CMS) and for use in the 18 CMS quality reporting and payment programs.
What’s the difference between submitting to CMS versus submitting to NQF?
The National Quality Forum (NQF) submission is about the endorsement process where a consensus-based entity reviews the measure using five evaluation criteria to assess the measure on its own merit and independent of a CMS program. It essentially gives it that stamp of approval, and so endorsement/NQF submission is separate from CMS implementation. NQF endorsement is valued for measures in CMS programs, but it is not a requirement.
The CMS implementation process takes the measure from being in development to being actively used in 18 of the CMS quality payment programs (QPP).
Phase 5: Use, Continuing Evaluation, and Maintenance
Re-evaluation
This step ensures that the measure continues to add value to quality reporting measurement programs and that its construction continues to be sound. The regular reevaluation of measures is vital as the science and other factors are always changing (e.g., development of new clinical guidelines, new technologies for data collection, discovering a better way to calculate measure results). Continually reviewing the measure will ensure it remains relevant and meaningful. Measures that stop being useful are retired.
Interaction Among Measure Lifecycle Phases
The Measure development lifecycle is not a linear process. Once the measure is conceptualized, it can move throughout the various phases in the measure development lifecycle.
The following decision criteria throughout the measure development cycle ensures a measure meets the applicable standards before moving to the next phase:
Importance to measure and report—including analysis of opportunities for improvement such as reducing variability in comparison groups or disparities in healthcare related to race, ethnicity, age, or other classifications.
Scientific acceptability—including analysis of reliability, validity, and exclusion appropriateness.
Feasibility—including evaluation of reported costs or perceived burden, frequency of missing data, and description of data availability.
Usability—including planned analyses to demonstrate that the measure is meaningful and useful to the target audience. This may be accomplished by the Technical Expert Panel (TEP) reviewing the measure results such as means and detectable differences, dispersion of comparison groups, etc. More formal testing, if requested by CMS, may require assessment via structured surveys or focus groups to evaluate the usability of the measure (e.g., clinical impact of detectable differences, evaluation of the variability among groups).
Measure Specifications
Continuity of Care Measure Specifications
This is a process measure evaluating primary care physicians; for each physician, their denominator is all of the patients they saw during the evaluation period who had at least 2 PCP visits (could include visits to other PCPs), and the numerator is the number of those patients whose Bice-Boxerman Continuity of Care Index is >= 0.7. The Bice-Boxerman index is a validated measure of patient-level care continuity that ranges from 0 to 1; 0 reflects completely disjointed care (a different provider for each visit) and 1 reflects complete continuity with the same provider for all visits.
2022 Person Centered Primary Care Measure Specifications
The Person-Centered Primary Care Measure Patient Reported Outcome Performance Measure (PCPCM PRO-PM) uses the PCPCM Patient Reported Outcome Measure (PROM) a comprehensive and parsimonious set of 11 patient-reported items – to assess the broad scope of primary care. Unlike other primary care measures, the PCPCM PRO-PM measures the high value aspects of primary care based on a patient’s relationship with the clinician or practice.
Recommended Reading
A New Comprehensive Measure of High-Value Aspects of Primary Care
Rebecca S. Etz, PhD
Stephen J. Zyzanski, PhD
Martha M. Gonzalez
Sarah R. Reves, MSN, FNP-C
Jonathan P. O’Neal
Kurt C. Stange, MD, PhD
To develop and evaluate a concise measure of primary care that is grounded in the experience of patients, clinicians, and health care payers… Read the article.
The Lost Pillar: Does Continuity of Care Still Matter?
By: David Loxterkamp, MD
Continuity of care has long held a hallowed place in the halls of family medicine. Indeed, it is one of the 4 pillars of an ideal family practice, along with first contact, comprehensive, and coordinated care… Read the article.
More Comprehensive Care Among Family Physicians is Associated with Lower Costs and Fewer Hospitalizations
Andrew Bazemore, MD, MPH
Stephen Petterson, PhD
Lars E. Peterson, MD, PhD
Robert L. Phillips Jr, MD, MSPH
Comprehensiveness is lauded as 1 of the 5 core virtues of primary care, but its relationship with outcomes is unclear. We measured associations between variations in comprehensiveness of practice among family physicians and healthcare utilization and costs for their Medicare beneficiaries… Read the article.
Higher Primary Care Physician Continuity is Associated With Lower Costs and Hospitalizations
Andrew Bazemore, MD, MPHStephen Petterson, PhD
Lars E. Peterson, MD, PhD
Richard Bruno, MD, MPH
Yoonkyung Chung, PhD
Robert L. Phillips Jr, MD, MSPH
Continuity of care is a defining characteristic of primary care associated with lower costs and improved health equity and care quality. However, we lack provider-level measures of primary care continuity amenable to value-based payment, including the Medicare Quality Payment Program (QPP)… Read the article.
Interpersonal Continuity of Care and Care Outcomes: A Critical Review
John W. Saultz, MD
Jennifer Lochner, MD
We wanted to undertake a critical review of the medical literature
regarding the relationships between interpersonal continuity of care and the outcomes
and cost of health care… Read the article.
Interpersonal Continuity of Care and Patient Satisfaction: A Critical Review
John W. Saultz, MD
Waleed Albedaiwi, MD
We wanted to review the medical literature regarding the relationship
between interpersonal continuity of care and patient satisfaction and suggest
future strategies for research on this topic… Read the article.
Physician versus Practice-Level Primary Care Continuity and Association with Outcomes in Medicare Beneficiaries
Mingliang Dai, Ph.D., MS
Jill Shuemaker, RN, CPHIMS
Lars Peterson, M.D., Ph.D.
Robert Phillips, M.D., MSPH
YoonKyung Chung, Ph.D.
To compare physician versus practice-level primary care continuity and their
association with expenditure and acute care utilization among Medicare beneficiaries and evaluate if continuity of outpatient primary care at either/both physician or/and practice level could be useful quality measures… Read the article.
Primary Care Physician Characteristics Associated with Low Value Care Spending
Tyler W. Barreto, MD
Yoonkyung Chung, PhD
Peter Wingrove, BS
Richard A. Young, MD
Stephen Petterson, PhD
Andrew Bazemore, MD, MPH
and Winston Liaw, MD, MPH
Previous work has shown that $210 billion may be spent annually on unnecessary medical services and has identified patient and hospital characteristics associated with low value care (LVC). However, little is known about the association between primary care physician (PCP) characteristics and LVC spending. The objective of this study was to assess this association… Read the article.
Primary Care Physicians and Spending on Low-Value Care
Aaron Baum, PhD
Andrew Bazemore,MD, MPH
Lars Peterson,MD, PhD
Sanjay Basu,MD, PhD
Keith Humphreys, PhD
Robert L. Phillips,MD, MPH
Low-value services account for $75 billion to
$100 billion of U.S. health care spending (1). Primary care physicians
(PCPs) have been conceptualized as potential gatekeepers
for efforts to reduce low-value spending, but the share of
low-value spending directly related to their services and referral
decisions remains unclear… Read the article.
The Lost Pillar: Does Continuity of Care Still Matter?
By: David Loxterkamp, MD
Continuity of care has long held a hallowed place in the halls of family medicine.
Indeed, it is one of the 4 pillars of an ideal family practice, along with first contact,
comprehensive, and coordinated care. But what does it mean to the patient,
the doctor, and our sense of identity and purpose? Read the articleRead the article.
Continuity of Care bibliography
By: Zachary Merenstein
Pre-Med Student, University of Maryland
With guidance and assistance from Alison Morris, Mikel Severson, and Andrew Bazemore, ABFM & the Center for Professionalism & Value Download the Bibliography.
Comprehensiveness–The Need to Resurrect a Sagging Pillar of Primary Care
Tracey L. Henry MD, MPH, MS,
Eugene C. Rich MD
Andrew Bazemore MD, MPH
Journal of General Internal Medicine (2021)
The COVID-19 pandemic highlighted both the importance of primary care and the fragility of its current infrastructure in the United States (US). Within its first 2 months, stark reminders of racial injustice, unaddressed health disparities, and grossly inequitable access to healthcare further underscored the current lack and future importance of universal access to high-performing primary care… Read the article.
Clinical Quality Measures in a Post-Pandemic World: Measuring What Matters in Family Medicine (ABFM)
Jill C. Shuemaker
Robert L. Phillips
Warren P. Newton
The Annals of Family Medicine (July 2020)
COVID-19 altered the way the American public lived their lives; the way they worked, ate, socialized, traveled, and ultimately received their health care. Family Medicine largely closed its doors to face-to-face preventive and chronic care visits and made a large shift to telephone and online video visits… Read the article.
Our Partners
We are grateful for the collaboration and support of our current partners in this initiative.
- American Board of Family Medicine (ABFM)
- The ABFM Foundation
- The PRIME Registry
- The Robert Graham Center
- The Larry A Green Center
- Stanford University
- Virginia Center for Health Innovation
- American Academy of Family Physicians (AAFP)
Interested in opportunities to collaborate or support this initiative?
Contact Jill Shuemaker
Related Publications
A New Comprehensive Measure of Primary Care
Higher Primary Care Physician Continuity is Associated With Lower Costs and Hospitalizations:
More Comprehensive Care Among Family Physicians is Associated with Lower Costs and Fewer Hospitalizations:
Primary Care Physician Characteristics Associated with Low Value Care Spending:
