First: Before starting the Use Case Development Guidelines, please read and review the attached PowerPoint slides. These slides provide a high-level overview on:

  1. Introduction to the Use Case Guidelines
  2. Overview of a Successful Use Case
  3. High-level Content of the Use Case Guidelines

If a use case team has any questions or comments after reviewing the attached slides, please reach out to the CodeX Program Management team (hl7codex@gmail.com).

Date of Last EditsSlides

 


Second: Review a summary of the Use Case Development Guidelines content, attached below:


Third: Complete the Use Case Development Guidelines, found here:

Use Case Development Guidelines Checklist

CategoryInstructionsCurrent StatusQM Member Team FeedbackCaroline's comments
Use Case Concept

Web Presence

This information provides a link to information available to the public and use case members

  • Create public and Member-only UC Web areas on Confluence using standard templates
  • Add content from sections below (as appropriate for the public vs member-only audiences)
  • Add content from the public Confluence use case homepage 
    • Also include Confluence link
  • Include content from member only Confluence use case homepage
    • Also include Confluence link

Public page available here: https://confluence.hl7.org/display/COD/Quality+Measures+for+Cancer

Member-only UC web page available here: https://confluence.hl7.org/display/CPP/Quality+Measures+for+Cancer


Is there guidance somewhere else on what content should be public vs. member-only?


AD: I think we can simply state that any content a UC team is comfortable having available out in the public can be in Public. Anything that would be considered to have a level of restriction should be in private/member-only section. 

Problem Statement

This information provides an overview of why this use case is needed, including a description of the challenges with the current state.

  • Provide a nearly complete description of the problem on the public Confluence page
  • Provide references to studies referencing the quantitative and/or qualitative extent of the problem (if available)
  • Digital oncology measure development/authoring has been constrained by an ecosystem of disparate and unstructured data, as well as a lack of digitalized collections and oncology models to support interoperability.
  • Consequently, the majority of oncology quality measures data collection has required manual abstraction resulting in significant burden, and an overall lack of opportunity to share this unstandardized information across systems or organizations. 


Relevant articles supporting problem statement:

ASCO Comment (SJ): Specific to the 3rd bullet, I don't think the structured data challenge is entirely unique to oncology. I would lean towards removing it.

No additional resources to add from ASCO.

Resolved


Proposed Solution/Workflow

This information provides an overview of the use case proposed solution.

  • Create a proposed solution that summarizes the use case overarching goal
  • Create a workflow diagram that visually represents the use case proposed solution
    • CodeX PM/Comms can help with graphics. Should be a novel solution, but doable.
  • Ensure that the proposed solution leverages FHIR (and mCODE where applicable)

Proposed Solution

  • Create a solution that demonstrates the ability to author and evaluate digital measures using FHIR data model with mCODE profile extensions and data elements for value-based programs and clinical quality improvement in the Oncology domain.
  • This use case will focus on the beginning of a measure’s life cycle and start with quality measure authoring.
  • Initial project scope will focus efforts to:
    • Author and develop a process for setting up necessary data elements in discrete data fields, to be used for data collection and aggregation. 
    • Prove that measures can be authored and executed using mCODE, and FHIR, and the correct measure result can be obtained.
  • This use case will start with the authoring and development of test case bundles using a single measure initially for phase 0.


Workflow Diagram:


Note: Regarding our proposed solution, if the measure steward(s) we work with (i.e., ASCO, VA, ASTRO) gives us the authority to publish the measure publicly:

  1. We will test a measure at either a Connectathon (or some other testing event)
  2. The tested and validated measures will be made available via GitHub (we can use our CodeX GitHub account for this; as well as CQI’s)
  3. And the measure content will be included on GitHub, including, but not limited to:
    1. Human readable metadata (specification)
    2. CQL file
    3. ELM
    4. Measure library bundle, as well as supplemental libraries
  4. We will offer a public demonstration/recording that highlights how the measures can really be used in practice via FHIR


Scope of Work

The use case team should define, in further detail, the full scope of work for the use case. This should include the overarching use case scope, the associated phases of work (high-level), and how the scope intends to solve the problem identified in the earlier criterion.

  • Provide background context to any existing work that helped catalyze and kick-off this CodeX use case scope
  • Define the high-level scope of work – which includes the overarching goal(s), phases of work (high-level), and how the scope intends to solve the problem
  • Also briefly describe any existing work that will be blended or incorporated into the use case scope of work
PhaseMilestone
Phase 0

Test the Telligen Quality Measures Engine (QME) Tooling, as a proof of concept, to determine if QME is successful in authoring and evaluating:

  1. the digitally available 2022 MIPS Measure #143: Oncology: Oncology: Medical and Radiation – Pain Intensity Quantified
  2. ASCO proposed eCQM #1: Antiemetic Therapy for Low and Minimal Emetic Risk Antineoplastic Agents in the Infusion Center
  3. ASCO proposed eCQM #2: Antiemetic Therapy for High and Moderate Emetic Risk Antieneoplastic Agents in the Infusion Center


Phase 1

Expand portfolio of VA quality measures and repeat measure mapping, authoring, test bundle creation and testing steps

Current Measures include:

  1. ASTRO measure: Oncology: Treatment Summary Communication - Radiation Oncology
    1. This is the ASTRO eCQM treatment summary measure – linked here
  2. VA measure: Performance Status
    1. This is one of the VA measures - linked here
Phase 2

Expand portfolio by aligning to oncology quality measures in federal models and among oncology steward programs

  • Proposed measure(s) for the Advancing Cancer Care MVP (MIPS) and Enhanced Oncology Model (once measures are identified)
    • Prepare to incorporate this subset into the QME through mapping to mCODE, authoring and support libraries and execution of measures using QME tool for testing


In addition to aligning to federal models, longer-term VA measures include:

  1. 28 Days from Diagnosis to Any Treatment (#All-B-1) ASPIRATIONAL
    1. This will require modeling “systemic therapy” as a FHIR concept
  2. 14 Days from Simulation to First Radiation Treatment
    1. This will require modeling of “simulation” as a FHIR concept (RT-specific)
Phase 3

Source real world Clinical Data Repository endpoints for measure data source

ASCO Comment (SJ): Since we determined early that the pain intensity measure was not a good fit for the project, should we remove it from the Phase 0 Milestone? Of did you intentionally want to leave it in to illustrate why it wasn't a good fit?


No Updates

AD (4/6): We intentionally left pain intensity in to highlight that this team did attempt to model it and determined it was not a good fit. Our report-out of this work is linked in the project plan, as well. 

Resolved


CP (6/1): I think this info is captured in the "Planned Phases" row? 

When I think of scope, I also think about what is NOT included in the use case. Might be helpful to add that to the 2nd bullet?


AD: Great point. I'll add a sub-bullet requesting that UC teams document what is not included in the use case (or what the 2-3 main things are)

Potential Impact

The use case team should provide a description (quantitative and/or qualitative) of how the proposed solution will improve the current situation; this could include better quality, less expensive/burdensome, more equitable care, research, surveillance.

  • Describe (quantitatively and/or qualitatively) how the proposed solution will improve the current situation.
    • Examples include: possible metrics to assess better quality of care, reduced cost/burden, more equitable care, improved research/surveillance.
  • Describe (if applicable) performance metrics that may be used to assess the success of a future use case pilot
  • The clinical oncology subject matter expertise of the CodeX community, paired with the mCODE profiles and FHIR IGs, should allow for the authoring of these quality metrics and provide a proof of concept for measure developers.
  • With the expansion of programs focusing on collection of meaningful oncology quality measure data, such as the VA initiatives, the Advancing Cancer Care MVP (MIPS), in support of the Federal Cancer Moonshot Initiative, and the Enhancing Oncology Model (EOM), this CodeX use case team wishes to provide solutions using HL7 FHIR and mCODE to provide a path to data collection with reduced burden. 
  • Additionally:
    • Target oncology quality measures in federal models and among oncology steward programs
    • Align relevant HL7 FHIR IGs, and any other related, external projects (where applicable)
    • Determine the synergies/direction of the use case that align with CMS’ standards and interoperability roadmap.


The intent of the QM use case is to highlight new quality measures that could be developed and new insights that could be made IF mCODE and other FHIR IGs (QI Core) was implemented and leveraged by an ecosystem of health organizations v. trying to retroactively map FHIR to existing measures. This is the value of using the ASTRO/VA radiotherapy measures to be an exemplar showing what can be achieved, by leveraging mCODE and RT IG, to create and hopefully roll out new/innovative quality measures.


Note: in our current phase of work, the QM team is demonstrating that mCODE/FHIR can be used to author and execute oncology measures, however, it's not currently easier. 

  • Solution is not currently reducing burden to authors but is reducing burden to the providers. 
  • Solution reduces burden to the submitters and receivers of measures because it's easier to extract a measure


Note: profile-informed authoring may be a sub-component of our QM use case. In addition to the support/collaboration with the CQI WG collaboration.

  • Profile-informed authoring would reduce authoring burden, but its not in our team's current scope. 

For the one I bolded, I think you can do this at a high-level during planning, but probably won't have the nitty gritty details until you get in execution.

AD: agreed - I'll note to include "high-level" for planning and "detailed" for execution.

Use Case Team

Meetings

The use case team should provide an overview of the types of planned meetings to ensure that the members are engaged and continuing to formulate or drive towards the use case plan.

  • Share all recurring and/or scheduled use case meetings. This information includes: date/time, participants, purpose of call, type of meeting (Teams, Zoom, Google) and the particular meeting link that will be used

Meetings

  • Biweekly “terminology” meeting
  • Monthly “full member” meeting
    • “technical” meeting (merged with full member meeting, for now)

Channels of work

  • Technical – Telligen leading current efforts
  • Terminology – collaborative effort across all current CodeX member organizations – Telligen, ASTRO, ASCO, Evernorth
  • FHIR Modeling – lead role undetermined
  • Coordination/Project Management – MITRE, CentanniPark

Public Calls

  • Planned after each use case milestone

ASCO Comment (SJ): Agree with Telligen best to keep technical meeting ad hoc for now.


Resolved

Maybe share examples of types of meetings? Use case team, leadership, technical, etc.


AD: great point. I'll add RTTD and QM's current meeting structures, as examples. See Slack message from Karen. 

Member Leadership

The use case team should have a listing of organizations willing to lead and drive the use case scope of work (also known as use case “champions”)

  • Ensure that a leader has been identified from the member community to guide the UC and ensure its success
  • Complete the attached table
Name AffiliationRole

Randi Kudner

ASTRO

Champion / Primary Point of Contact

Samantha Dawes

ASTRO

Champion

Charlotte RaleyASTROASTRO measures and RT Support

Stephanie Jones

ASCO

Champion / Primary Point of Contact

Caitlin Drumheller

ASCO

ASCO Measure SME

Yvette Apura

ASCO

Clinical Informaticist / ASCO Measure SME

Becky Metzger

Telligen

Champion / Primary Point of Contact

Gail Winters

Telligen

Technical SME

Sharon Labbate

Telligen

Technical / Terminology SME

Jessica PughTelligenProject Management for Telligen-focused tasks

Dennis Blair

Evernorth

Champion / Primary Point of Contact

Rick Emery

Evernorth

Champion

Dr. Mary Kay Barton

Evernorth

Medical Oncology SME

Dr. Vik Shah

Evernorth

Medical Oncology SME / Cigna Representative

Dr. Nimi Tuamokumo

Evernorth

Radiation Oncology SME

Sondra Berger

Evernorth

Business Development

Sharon Sebastian

MITRE

Clinical Informaticist / Terminology Support

Rob Dingwell

MITRE

Technical Support

Anthony DiDonato

MITRE

Project co-Coordinator

Doug Williams

CentanniPark

Project co-Coordinator

ASCO Comment (SJ): In keeping with your current role themes, I would also add 'Clinical Informaticist' to Yvette's role.


Resolved

Should there be a section about “project management leadership” or something like that? Thinking about who is going to manage all these meetings, prepare for them, do all the use case leadership roles, etc.


AD: I think PM leadership is a combination of overall UC Coordinator support + technical and terminology leadership roles. I'm open to considering a new role, though. 

Member Representation & Necessary Stakeholders

The use case team should have a list of all members – both current members and aspirational members. This information will help CodeX leadership determine if:

  • (1) there are sufficient member resources for use case sustainability, and
  • 2) the proposed plan can be fulfilled with the existing members.
  • Identify which organizations, and individuals that represent each organization, are contributing to the use case
  • Identify which organizations the use case team aspires to have join the use case.
  • Complete the attached table
Current or AspirationalOrganizationType of OrganizationCodeX Member LevelPrimary Use Case?

Current

ASTRO

Association/Accreditation Program

Premier

Partial

Current

ASCO

Association/Accreditation Program

Benefactor

Partial

Current

Telligen

Quality Improvement Organization/Contractor 

Benefactor

Yes

Current

MITRE

Non-profit

Premier

No

Current

Evernorth

Health Services provider for Cigna

Premier

Partial

Current

CentanniPark

Strategic Consultancy

D/I

Yes

Aspirational

Varian

Oncology Information Systems



Aspirational

Epic

Health System EMR



Aspirational

McKesson (Ontada)

Health System EMR



Aspirational

Pfizer

Pharma



Aspirational

UnitedHealthcare

Payer



Aspirational

Elekta

Oncology Information Systems



Aspirational

Integra Connect

Health System EMR



Aspirational

ONC

Association/Accreditation Program



Aspirational

RaySearch

Oncology Information Systems



Aspirational

Flatiron

Oncology Information Systems




McKesson (iKnowMed)

Oncology Information Systems




US Oncology

Association/Accreditation Program




American Urological Association

Registries/Data Repositories




CMS CCSQ

Value-based Care




CMS Office of Burden Reduction

Value-based Care




CMS Quality Measurement and Value Based Incentives Group

Value-based Care




CMS MIPS/MVP

Value-based Care




Cigna

Payer




Priority Health

Payer




xCures

Patient Encounters




NCQA

Association/Accreditation Program




Roche

Payer




Johnson & Johnson

Payer




MyChart

Patient Portals




FollowMy Health

Patient Portals




Apple Watch

Other Data Sources




Fitbit

Other Data Sources




Allscripts

Health System EMR




TBD – As needed

Providers



CP (6/1) - not all the cells are filled?

Asynchronous Communication

The use case team should identify what asynchronous communication platform to use for communication across all member organizations. The decision should be reflected within this criterion and the asynchronous communication platform should be stood-up and active before a use case can move into Execution stage.

  • Determine an asynchronous communication platform
  • Reflect the team’s decision within this use case criterion
  • Activate the asynchronous communication channel and include all current team members

Presently, the Quality Measures team feels communication via email is sufficient for the success of the use case. Existing CodeX members did note that, because of availability, Teams would be the preferred use of another communication platform, if we decide to move beyond email. 


Member responses

  • Telligen: uses Teams; would prefer email at this time
  • Evernorth: would prefer email list
  • ASTRO: uses Teams
  • CentanniPark: Slack; would prefer email at this time
  • ASCO: uses Teams; would prefer email at this time 
  • ASTRO: uses Teams; would prefer email 
  • MITRE: uses Teams, Slack; would prefer email at this juncture


Resources Requested

The use case team should provide an estimate of resources and funds needed from the CodeX member-fee-based budget (e.g., funding from CodeX Membership fees paid to HL7) that are not supplied from other member sources (in-kind support, etc.). Past experience suggests that a minimum of 0.25- 0.5 FTE (or 10-20 hours per week, on average) for a UC Coordinator plus 0.1-0.2 FTE (or 4-8 hours per week, on average) for overarching PM support is needed to operate UCs successfully.

  • Establish what resourcing is already supported by current CodeX member participation. This support can come from volunteer work, in-kind resourcing, member internal funding, etc.
  • Identify, next, the current resourcing gaps that exist in the use case scope of work. These gaps reflect the identification of tasks or activities that current use case members are unable to tackle, based on either lack of time, funds, or expertise
  • Request, finally, an estimate of resources and funds needed from CodeX member-fee-based-budget that could support and fill (part) of the identified gap(s)

Resourcing that is currently supported by members:

  • ASCO 
    • Measure steward SME / Medical oncology support / champion
    • 4-5 hrs/week (16-20 hrs/month)
  • Telligen - Agree with general estimates though this will need to be evaluated through phases on continuous cycle.  Would also add that specialized collaboration and input from additional HL7 committees (CQI) and potentially stewards (AMA) would be idea. The commitment for CQI might evolve to a member(s) participating in CQI committee meetings on regular basis.
    • Use case coordination / Measure SME – Jessica Pugh and Becky Metzger
    • Technical lead – Gail Winters
    • Terminology support – Sharon Labbate
    • Telligen is dedicating ~40 hours/week (160 hrs/month) across all tasks(will ebb and flow) 
  • ASTRO
    • Radiation oncology leadership / measure steward SME / champion
    • 2.5-4 hrs/week (10-15 hrs/month)
  • Evernorth
    • Medical oncology SME
    • Radiation oncology SME
    • Payer perspective
    • 6-10 hrs/week (24-40 hours/month)
  • MITRE
    • Terminology support
    • Use case coordination
    • Overall project management
    • ~15-20 hrs/week (~60/80 hrs/month)
  • CentanniPark
    • Use case coordination
    • ~3-4 hrs/week (~12-16 hrs/month)


Note from Telligen: if there were additional technical members – then there could be either concurrent or differing work – this may add time to Telligen's current involvement. 


ASCO Comment (SJ): I edited the ASCO bullets to be in alignment with how the other orgs presented their time.


Resolved

Just a FYI comment about the "request for resources/funds"- I think this is going to be hard. Especially thinking about future prospective pilots that require a lot of money. Maybe provide some guidance on how much other prospective pilots for use cases have cost? It's why I bolded it for execution.


AD: Great point. This section may want to just be a high-level assessment of the $/time/resources that are anticipated to be required for a pilot. Caroline Potteiger Do you have prospective pilot #s you'd be able/willing to share?

Member Provided Resources

The use case team should provide a list of potential members who could provide support through in-kind support or membership fees to bridge the gap between what is needed and what is currently available.

  • Identify the current resourcing gaps that exist in the use case scope of work. These gaps reflect the identification of tasks or activities that current use case members are unable to tackle, based on either lack of time, funds, or expertise
  • Provide a list of potential members who could provide support, through either in-kind support or membership fees, to bridge the gaps(s) identified in the above bullet point.
  • Describe how each potential member could successfully contribute, support, and fill (part) of the identified gap(s)


Note: Are there additional needs in terms of in-kind project support (e.g., providing clinical SME input on a QM terminology meeting) that we should consider and quantitatively estimate here (in hours per week/month)


Note: Are there additional needs in terms of paid support (e.g., request to use CodeX member funds to support QM comms, etc.) that we should consider and quantitatively estimate here (in FTE hours per week/month)

  • Vendor perspective – either in-kind or member-funded support – these orgs will help QM use case understand the technical feasibility of use case work. In addition to Telligen.
    • See above “Health System EMR”
  • Terminology lead – either in-kind support or member-funded support – currently, Telligen + MITRE helping coordinate these efforts, but we will need someone to own tasking moving forward
    • TBD – This individual would ideally be a clinical informaticist with FHIR proficiency + has a background in quality measures and HL7
  • Pilot activities – the QM use case will need to have health system buy-in and vendor support to kick-off pilots in later phases of work
    • Telligen can serve as initial vendor organization, however, we’ll want other organizations (as identified above)
    • We’ll also want support from health systems to participate in a pilot
  • UC Coordination split 50/50 between CentanniPark and MITRE
    • Determine a concrete breakdown:
      • Outreach – CentanniPark
      • Strategic alignment – CentanniPark
      • Project coordination – MITRE
      • Project scoping / planning – MITRE
    • Federal alignment / government perspective – the QM use case would benefit from governmental insight to existing and “on the horizon” federal quality programs and those related measures.
      • CMS
      • NCQA

I think “Member Provided Resources” and “Resources Requested” could be combined in one. You could just list all the resources you need and then mark the ones that could be member provided


AD: Agreed – I'll consolidate into one criteria. 

Outside Initiatives

The use case team should identify consortia or other organizations outside of CodeX with which the use case may want to interface, coordinate or partner, or at least remain aware of. Could be competitive.

  • Include a list of relevant outside initiatives that are related to the use case, as this will provide awareness of what has already been accomplished (and can be leveraged)
  • Summarize how and why these initiatives are related to the use case
  • Provide a network of organizations/individuals working towards similar issues. 
  • HL7 CQI Work Group
  • HL7 QI-Core IG team
  • HL7 QM IG team
  • NCQA-related initiatives
  • CMS-related initiatives
  • MITRE Abacus project
  • Gravity FHIR Accelerator - oncology / equity related concepts in Gravity that might be applicable for QM use case
  • ASCO - In 3/9 Presidential Budget in Brief: https://www.hhs.gov/sites/default/files/fy-2024-budget-in-brief.pdf
    • Page 75 mentions what appears to be a new 'proposal' to Expand Cancer Care Quality Measurement, but no specifics are given. I've reached out to Vinitha and Dr. Kline to try to get more intel.
    • 4/6 Update: ASCO had a call with Vinitha and she wasn't even aware of the blurb, but said it simply means CMS is trying to incorporate more oncology measures into more CMS programs (e.g., IQR, OQR, etc.) in response to the Cancer Moonshot.


Note from Telligen: suggest reaching out to AMA for assessment of interest/participation (non-competitive).

ASCO Comment (SJ): Update on the 3/9 Presidential Budget in Brief — we had a call with Vinitha and she wasn't even aware of the blurb, but said it simply means CMS is trying to incorporate more oncology measures into more CMS programs (e.g., IQR, OQR, etc.) in response to the Cancer Moonshot.


Resolved


Use Case Plan

Planned Phases

The use case team will need to define short, agile phases of work that make up the larger use case scope. These phases should be between 6-9 months in length and should define (at a high level) the specific tasks that will be required of the team in order to achieve the phase. Additionally, success metrics and key performance indicators should be defined for each phase.

  • Document the team’s first phase in appropriate detail. “Appropriate” means the team should leverage their Project Planning page on the Confluence private site to detail the specifics of the phase. Namely: a phase objective, overarching timeline, success metrics for the phase, description of tasks that contribute to the phase, primary ownership of each task, timeline for each task, any current gaps in the proposed phase of work, and any other relevant information.
  • Note: while the first phase of work defined in the Execution stage should be the most detailed, the team should also provide a high-level summary of at least 1-2 subsequent phases. The phases should include similar information to what was described above.
PhaseMilestoneTimelineKPI(s)

Phase 0

Test the Telligen Quality Measures Engine (QME) Tooling, as a proof of concept, to determine if QME (using FHIR/mCODE extensions) is successful in authoring and evaluating:

  1. the digitally available 2022 MIPS Measure #143: Oncology: Oncology: Medical and Radiation – Pain Intensity Quantified
  2. ASCO proposed eCQM #1: Antiemetic Therapy for Low and Minimal Emetic Risk Antineoplastic Agents in the Infusion Center
  3. ASCO proposed eCQM #2: Antiemetic Therapy for High and Moderate Emetic Risk Antieneoplastic Agents in the Infusion Center


Process:

  • Map measure to mCODE (MIPS, (2) ASCO antiemetics)
  • Create FHIR transaction test case bundles and load to FHIR Clinical Data Repository (CDR)
  • Author Measure and supporting libraries (translate and create FHIR measure resources)
  • Stratifying denominator by available SDOH characteristics
    • The measure report will automatically stratify on the available data elements: Race, ethnicity, payer, gender at the moment – limited scope
    • Once we have an updated USCDI we will have a more robust demographic pool of information
  • Execute measure and measure report with QME tool


Phase 0 supporting work includes:

·       Convert (1) existing oncology MIPS measure and (2) ASCO eCQM measures to FHIR with use of at least one mCODE extension

·       Create additional “measure” which extracts patient data from mCODE profile resources such that logic to extract extension elements can be reused across measures.

·       Create FHIR transaction test case bundles and load to FHIR Clinical Data Repository (CDR)

·       Author supporting libraries (as needed)

·       Execute measure and measure report in tooling environment

September – March 2023

Proof of concept was successful in demonstrating that the Telligen QME tool could author and evaluate existing MIPS and ASCO measures, using FHIR/mCODE extensions


ASCO measures were successfully modeled in FHIR and test cases were created in JSON and uploaded to testing environment. 


Testing environment was created successfully.


Use case team determined that mCODE would not be required to support pain intensity quantified: Discovery Use Case Concepts (Phase 0)

Phase 1

Expand portfolio of VA quality measures and repeat measure mapping, authoring, test bundle creation and testing steps

Current Measures include:

  1. ASTRO measure: Oncology: Treatment Summary Communication - Radiation Oncology
    1. This is the ASTRO eCQM treatment summary measure – linked here
  2. VA measure: Performance Status
    1. This is one of the VA measures - linked here


Similar to Phase 0, Phase 1 supporting work includes:

·       Convert (1) ASTRO measure and (1) VA measure (identified above)

·       Create additional “measure” which extracts patient data from mCODE profile resources such that logic to extract extension elements can be reused across measures.

·       Create FHIR transaction test case bundles and load to FHIR Clinical Data Repository (CDR)

·       Author supporting libraries (as needed)

·       Execute measure and measure report in tooling environment


April – September 2023?

Proof of concept was successful in demonstrating that the Telligen QME tool could author and evaluate existing ASTRO and VA measures, using FHIR/mCODE extensions


ASTRO and VA measures were successfully modeled in FHIR and test cases were created in JSON


Testing environment was created successfully.

Phase 2

Expand portfolio by aligning to oncology quality measures in federal models and among oncology steward programs

  • Proposed measure(s) for the Advancing Cancer Care MVP (MIPS) and Enhanced Oncology Model (once measures are identified)
    • Prepare to incorporate this subset into the QME through mapping to mCODE, authoring and support libraries and execution of measures using QME tool for testing


In addition to aligning to federal models, longer-term VA measures include:

  1. 28 Days from Diagnosis to Any Treatment (#All-B-1) ASPIRATIONAL
    1. This will require modeling “systemic therapy” as a FHIR concept
  2. 14 Days from Simulation to First Radiation Treatment
    1. This will require modeling of “simulation” as a FHIR concept (RT-specific)

6-9 months


Phase 3

Source real world Clinical Data Repository endpoints for measure data source

·       Aim for real world implementation with vendor(s), such as Telligen, Epic, McKesson, etc., and interested health sites/payers.


TBD



Phased Workflow for measure development:


Measure Authoring Process:

CP (6/1) - add a column for stakeholders required and what's needed?

HL7 FHIR Implementation Guides

The use case team should mention how (if at all) new and/or existing FHIR Implementation Guides might be used or updated within the use case scope.

  • Summarize initial thoughts on the need to either (1) develop a new Implementation Guide as a stand-alone use case artifact and/or supplementary to existing specific IGs or (2) leverage existing FHIR Implementation Guides
  • Summarize initial thoughts to work with existing HL7 Work Groups that manage existing FHIR IGs, outside organizations that are principally involved in the development and management of an IG
  • If applicable, summarize high-level plans to ballot a new FHIR IG and what overarching work will need to be achieved – define new values sets, profiles, data elements, etc.

Existing FHIR IGs the QM team is leveraging include:

  • QMIG (Quality Measures implementation Guide)
  • mCODE STU2 (3, when available)
  • DEQM (Data Exchange Quality Measures)
  • FHIR 4.0.1 and forward


Implementation in Health Information Systems

The use case team should identify existing health information technology systems – either implemented at health sites or external to stand-alone organizations – that might be interested in participating on the use case by implementing aspects of the FHIR standard in their system.

  • Conduct a current state of existing health IT systems that might be interested in contributing to this use case.
  • List the potential health IT systems. Also identify if any initial outreach has been made.
  • Provide a brief description as to why each system might be a good fit.
  • Identify how each health IT system would fulfill the role in a (1) Prototype (2) proof of concept or (3) pilot
    • See definitions on UC Development slides.
  • If no existing health IT systems are identified, describe new health IT systems that might need to be developed to support the success of this use case.

For the purposes of this use case scope – we are focusing efforts on the rectangular portion of the above image. As Telligen works through this technical environment, we’re looking for additional organizations to implement our team’s solution. Thus, we’re looking to Oncology Information Systems (Varian, Elekta, RaySearch, iKnowMed, Flatiron), Health System EMRs (Epic, Allscripts, McKesson, Cerner, etc.), Payers (Cigna, UnitedHealthcare), and Associations/Accreditation Programs (ASTRO, USON, ASCO, etc.) to serve as endpoints for this implementation (Phase 2/3). The QM team is working through an outreach plan to share/demonstrate our completed phase 0 of work with these organizations to understand their level of interest and feasibility to implement.

Telligen: recommends staying with proposed AWS environment. Future-forward phases may want to implement/coordinate with OMOP.

CentanniPark: Ideally, we could also target systems implementing mCODE to provide data inputs for the QMs that can be housed downstream (AWS, etc.)


Execution Environment:



Piloting in Real World Settings

The use case team should describe a plan for how the proposed use case solution could be tested in a real world setting through a pilot project to demonstrate its effectiveness and identify any real-world challenges.

  • Highlight (1) any current organizations that have already expressed interest in future pilot work or (2) an organization “wish-list” of stakeholders that would be best to connect with and include on future pilot work
  • Describe, in high-level, what an ideal pilot in real-world setting might look like: actors, data flow, type of data, estimated timeline of activities, etc.

For a pilot to be successful in a real-world setting, we need to engage with organizations that are willing to be DEQM endpoints (see below diagram). This would allow new organizations to participate in the AWS environment.

Description: Use Case Evaluation Environment is hosted on Amazon Web Services (AWS) cloud-based platform and I is available to authenticated users.  It serves as the source for measure data (FHIR bundles) to be stored, loaded and archived and A FHIR Server FHIR server with mCODE capability and CQF Ruler which offers plugins that provide an implementation of FHIR's Clinical Reasoning Module, serve as a knowledge artifact repository, and a cds-hooks compatible clinical decision support service. FHIR data including measure reports and supporting data are transformed to an analytic database that serves as the source for an AWS Quicksight based dashboard for reviewing performance.


Process Flow (see below diagram):

  1. FHIR bundles are posted to AWS S3 storage by authorized user through web application (1) or may be directly uploaded to S3 storage folder.
  2. A Lambda Orchestrator will process uploaded bundles, posting them to the FHIR Server and then archiving them to a storage folder.
  3. Orchestrator posts bundles to FHIR HAPI Server (with mCODE STU2 and CQF Ruler* plug-ins and archives file(s) to S3 archive bucket 
  4. FHIR Server with mCODE capability and CQF Ruler (RDS db)
  5. FHIR measure report and supporting data is transformed and loaded to an analytic database (PostGres)
  6. AWS Quicksight with analytic database connection used for user access to measure results with filtering on dimensions.



Risks

The use case team should (1) identify any risks or barriers that could impede success of the use case and (2) state, if possible, proposed mitigations to address these risks to ensure success of the use case.

  • Identify any risks or barriers that could impede success of the use case
    • Examples of risks included, but are not limited to: competing efforts in the community, insufficient use case resourcing, divergent views in the community, technical barriers, clinical barriers, legal barriers, etc.
  • State, if possible, proposed mitigations to address these risks to ensure success of the use case

Quality Measures risks include:

  • Allocated resources having competing priorities (i.e., Telligen staff being asked to do contracted work that may compete with our use case) – risk of delay 
  • Lack of participation of endpoint organizations (DEQM endpoints) – this would be a lack of community interest from vendors, payers, health sites – risk of success
  • Lack of regulatory participation/government interest – having CMS, ONC, NCQA, etc. buy-in to our work will create a much easier route to adoption if the federal models are invested in FHIR and interested in our use case approach. – risk of success
  • Changes to federal quality models that negatively affect our use case scope: both in momentum and direction – risk of delay
    • Momentum: CMS loses interest (i.e., if we begin aligning to a CMS model’s quality measures, and the model gets put on pause and/or shut down)
    • Direction: CMS changes draft set of quality measures for a federal model that QM use case already began aligning to
  • Aligning to existing FHIR IGs related to quality measures (QMIG, DEQM) can be risky since this use case team doesn’t have direct control of the IG contents. We will need to stay informed with changes and the direction of both IGs. And we will want to join the CQI WG as an IG team to sponsor our work – so we can stay in harmony with the other related FHIR work. – risk of success
  • ASTRO: we may risk losing track of larger picture - we don't want to wait decades for federal orgs to pick up or use our measures (especially radiation measures), so how can we truly roll out our solution in practice → will we need to re-shift measure focus on certain organizational endpoints (such as health sites, others) and align our future measures to those that the endpoints are interested in piloting?
    • This use case scope, at the moment, is a great proof of concept.
      • We’re testing the pathway but what’s the larger product
    • We've been very hung-up on the availability of data in vendor systems / the aggregation of data in repositories / the lack of CMS interest

      • Could we scope our use case vision longer-term to not be bogged down by these concerns and work through authoring and implementing measures (i.e., the VA measures) with endpoints willing to test (i.e, VA)? 
    • Proposed solution to this risk/concern: Do we want to author brand new measures that can be used in clinical practice? We don't necessarily need to be stuck in old ways of how measures are authored, endorsed, published, and used. 
      • We can go rogue and create our own use case process for this measure development.


Note from Telligen: to truly implement use case in the wild, we will need end point implementation. Future phases really need to focus on relying on engaging from organizations and their commitment to serve as endpoints

Note from ASCO: CMS' main interest is in anticipated MVP measures



Probably helpful to list risks by phase of work. Risks during the proof of concept can be completely different to risks during the prospective pilot


AD: Great point! I'll update the instructions to request that UC teams list risks by phase

Adoption and Scaling

The use case team should identify how the proposed use case solution can be adopted or scaled in a sustainable manner.

  • Ensure use case team has a long-term plan for continued growth and participation in the use case
  • Describe why the use case team is confident that the solution will eventually be adopted by a significant portion of the relevant health community
    • What are drivers of this use case that are significant to highlight here, related to adoption?
  • Explain how the broader health ecosystem may adopt the use case solution (i.e., define more broadly than just the use case scope)
  • Describe how the use case solution will be scaled, long term: both from a use case implementation perspective and also from a use case collaboration perspective

To ensure the use case can be adopted and scaled in a sustainable manner, the QM team identified three key components:

  1.  Regulatory – prioritization (in order to get momentum, we need regulatory backing).
    1. Important to work with CMS in future phases
  2.  Implementation - documented use cases with clear guidelines. Ultimately this use case will describe a vendor-agnostic system in future phases:
    1. We expect/hope that a future state will support open source tooling which the mCODE profiles can be used directly as a data model within a GUI-based authoring tool
      1. After measures are developed and tested with test cases in the IDE environment, they will need to be translated with the CQL to ELM translator (open) and measure bundles will need to be built for measures and libraries following the Quality Measures Implementation Guide and posted to a mCODE-compliant FHIR server with measure evaluation tooling support such as CQF ruler. measures need to be built into Initially, with use of synthetic data, data bundles will also need to be scripted/created and loaded to FHIR resources (profiles) with FHIR api’s. These bundles too will need to be posted to FHIR server with mCODE capability (see evaluation environment)
    2. Current state is as follows (looking to pave the way to this agnostic scope):
      1. QM team is using open-source code using CQI framework. From a measure authoring perspective, we’re envisioning after being defined and mapped, that measures will be developed in CQL in an IDE environment supporting CQL such as VS-CODE using the Clinical Quality language extension (open) value set.
      2. Measures and libraries authorized for consumption can be hosted on a GitHub repository (i.e., CodeX GitHub).
      3. Telligen can assist with measure authoring tooling support in the form of setting up environments and extensions.
      4. QM team expects/hopes that a future state will support open source tooling which the mCODE profiles can be used directly as a data model within a GUI-based authoring tool
      5. After measures are developed and tested with test cases in the IDE environment , they will need to be translated with the CQL to ELM translator (open) and measure bundles will need to be built for measures and libraries following the Quality Measures Implementation Guide and posted to an mCODE-compliant FHIR server with measure evaluation tooling support such as CQF ruler.
    3. Alignment with MADiE (the future replacement for the MAT/BONNIE)
      1. Potential future endpoint for implementation
      2. would include the mCODE profiles in a future version of MADiE
  3. Automation and Standardization - documented benefits and savings (from an economic benefit and patient benefit, we need to clearly define how this use case is benefitting their health systems, organization.
    1. The QM team will focus on describing why an organization would be interested in adopting. Ex) what will appeal to a CFO – this will be one of our focus areas during stakeholder outreach.
ASCO Comment (SJ): 

Do we want to include MADiE (the future replacement for the MAT/BONNIE) here and a future endpoint for implementation would be to include the mCODE profiles in it?


Resolved

I think this is important, but hard to do in detail at the beginning of the use case. We had manyyyy conversations about this for Trial Matching and went back and forth based on what we learned. Like who is going to host things? How much will it cost for people to implement in the long-term (not just pilots)?

AD: IMO - the point of this criteria is to just get the conversations started. It's OK if the details aren't sorted out but this allows the team to start throwing together thoughts/ideas. 

Alignment and Decision Making


Decisions within the Use Case

Use case Leadership team (paying, sponsored, and government members) should define a standard approach for making decisions.

  • Define which meeting(s) decisions are made
  • Define who are the use case decision makers (which types of CodeX member orgs)
  • Define who will be tracking each decision
  • Define how decisions will be tracked and managed
  • Define if/how decisions will be disseminated across the use case team
  • Define how a disagreement related to a potential decision will be resolved
  • QM use case decisions are made during the Quality Measures Full Member Meeting.
    • All paying, government, and sponsored members have decision-making power.
  • The use case coordinator(s) will be tracking all decisions.
  • Decisions will be tracked and managed on our Meeting Summary page on Confluence: https://confluence.hl7.org/display/COD/Quality+Measures+Meetings
  • After each Full Member meeting, meeting notes (along with the associated decisions) will be shared via email to the full member team for review/situational awareness.
  • If there is a disagreement related to a potential decision, then the member participants on the call that the disagreement was raised must come to a majority agreement on the path forward. The voting breakdown must be recorded in addition to the decision results.


Decisions for the Next Stage

This criterion documents the process to request transition, and the subsequent approval, from one stage to another.

Discovery

    • UC leaders request that PM start process of considering transition to Planning, based on input to requirements above.
    • PM acknowledges receipt of request to proposer.
    • PM, within ~1 business day of acknowledging receipt, informs the OC and SC
    • OC and SC have ~5 business days to express concerns about moving the UC to Planning.
    • PM discusses any concerns with the proposer and the OC/SC, and is responsible for disposing of issues and informing all parties as to whether the proposed UC can move into Planning or not (with reasons).

Planning

      • UC leaders requests that PM start process of considering transition to Execution, based on input to requirements above.
      • PM acknowledges receipt of request to proposer.
      • PM, within ~1 business day of acknowledging receipt, informs the OC and SC.
      • OC and SC have ~5 business days to express concerns about moving the UC to Executing
      • PM discusses any concerns with the proposer and the OC/SC.
      • PM convenes SC to make decision regarding moving to Execution.
      • PM is responsible for disposing of any final issues and informing all parties as to whether the proposed UC can move to Execution immediately, or will need to stay in Planning due to lack of resources.

Hi CodeX Program Management Team,


The Quality Measures for Cancer use case team is requesting that we move into the Planning Stage of CodeX. We are introducing the idea/opportunity of moving directly to the Execution stage, given our current state of the use case, if possible. 


Thank you,

The Quality Measures Leadership Team

















***EVERYTHING BELOW THIS LINE IS DRAFT MATERIAL***


Category

During Discovery. Before approval by the Steering Committee to move to Planning.

Current Status

Quality Measures Member Comments


Status: Complete; In-progress; Not started

Concept

WebCreate public and Member-only UC Web areas on Confluence. PM/Coordinator will help set these up using standard templates. Add content as noted below.

Confluence pages for the Quality Measures for Cancer Use Case have been setup - including standard templates describing the overall measures and elements.

Partially complete because the Confluence pages are set-up, however, the content within these pages not yet finalized

Action: CodeX QM team to review home page content and update, as needed


In-progress

  • Web area is available on Confluence. working on populating quality measures content on these specific Confluence pages

Telligen:  We discussed some updates to diagrams in the terminology call.  Once the full team is able to review and we have consensus, we do not have additional updates based on currently defined scope.  As we proceed into phase 0.5 and phase 1, etc. we should be able to update a bit more on target measures, if there are specific issues/problems we can demonstrate solving in those measures, etc.  

ASCO - No edits

CentanniPark - Updated current status column to reflect progress.

Problem

On public page, provide a nearly complete description. References to studies regarding the quantitative and/or qualitative extent of the Problem would be helpful.

Complete 
Solution / workflowOn public page, provide, preferably, a single, proposed solution and workflow diagram (PM/Coordinator can help with graphics) around which Discovery UC Members (and those committed to join) are aligned. Should be a novel solution, but doable.

Solution – Complete 

Workflow – in-progress 

Telligen - See note above re: discussion of diagram on terminology call.  As we get further along into technical solutions, we can provide additional high-level technical architecture diagrams.  
Potential impactOn public page, provide an estimate of quantitative and/or qualitative improvements that might be realized if the solution proves successful. Categories of improvements could include better quality, less expensive/burdensome, more equitable care, research, surveillance.Complete
Scope (in addition to above and below)

Concept and solution are consistent with the vision, principles of CodeX:

  • Vision: Collect patient data once and reuse for multiple purposes
  • Domains (today): oncology, cardiovascular, genomics
  • Member-driven by key representatives of necessary stakeholders (see "Team" below)
  • Accelerate interoperable data modeling and implementation around ​

    the FHIR, HL7 standards, including CodeX products like mCODE and associated artifacts​

Discovery stage → scope in terms of high-level objectives for the broader project work


Planning stage → dedicate more time to defining tasks/goals/owners of tasks/resourcing for Phase 0

While we're in Phase 0 - follow similar approach to defining tasks/goals/owners of tasks/resourcing for Phase 0.5

  • so on and so forth..

QM use case focus is oncology






Team

Meetings

Meetings with individual, potential Members is the best way to discuss specific interests.

Schedule, as early as possible, weekly or bi-weekly public calls to build broader interest and gather ideas that help to form the Concept, Team and Plan. PM/Coordinator will help target participants and schedule.

Schedule CodeX Leadership (Paying, Gov't and Sponsored Members or those who commit) calls to consider all input and make decisions.

See meeting and messaging recommendations - Communication Rhythm

  • biweekly terminology call
  • monthly (but currently biweekly) leadership call
  • ad-hoc technical call – Telligen more or less working offline. 
    • should we consider having a recurring QM call stood up
  • Public Call → agreed to have public call for completion of each phase of work

ASCO - I think given our teams current bandwidth, the schedule of calls now is working well for us.

CentanniPark - current schedule of calls is working well.  Great teamwork and valuable discussion on the calls.

Member leadershipAt least 1 "Member Champion", who commits to lead concept and planning alignment, engagement of stakeholders, etc.  Add names to public CodeX Confluence page.
  • Telligen – Becky Metzger
  • ASTRO – member champion
  • ASCO – member champion
  • Evernorth – member champion
  • MITRE – member champion


ASCO - Stephanie Jones (would be helpful if we could come up with some talking points (smile))
Member representatives of necessary stakeholders and actorsAdditional CodeX Members at any level or those who commit, such that there is at least 1 organization to fill each stakeholder/actor role in the UC.
  • we have a very defined, immediate focus and scope. we will need to engage others as we move through initial phases. 
  • add workflow diagram with current member orgs that fill each actor role
ASCO - Still working to get Integra Connect contacts.
Resources requested from the CodeX Member-fee budget (in addition to the in-kind resources to be provided from UC participants)

Estimate of full-time equivalent (FTE) resources (and skills) being requested to be funded from the CodeX Member-fee-based budget (e.g., funding from CodeX Membership fees paid to HL7). These are resources that are not likely to be provided through in-kind support from UC participants.

Note: Experience suggests that a minimum of 0.25- 0.5 FTE for a UC Coordinator plus 0.1-0.2 FTE for overarching PM support (engagement, governance, communications, education) is important to moving fast and in an organized fashion. If additional expertise is required for terminology, FHIR, architecture, software development, pilot planning/execution, etc.,  another 0.5 - 2.0 FTE could be needed.  


ASCO - Adding up the time now spent by Stephanie, Caitlin and Yvette we'd estimate we are at 0.25 FTE.


Telligen- Agree with general estimates though this will need to be evaluated through phases on continuous cycle.  Would also add that specialized collaboration and input from additional HL7 committees (CQI) and potentially stewards (AMA) would be idea. The commitment for CQI might evolve to a member(s) participating in CQI committee meetings on regular basis.

Sufficient Member-fee and/or grant funding and/or Member in-kind resources (to meet the request in the previous row)

List of specific, potential UC participants who are not yet Members or not committed to join.

Note: For resources that are needed, but not provided as in-kind support, assume that $150K or more of funding could be need through new paying Membership, grant funding and/or through freeing up of resources from existing and/or ending UCs.



Note re: Health Systems: While health systems are most welcome to join CodeX at any level, we are not requiring health systems to formally join CodeX in order to participate in implementations, pilots and other non-decision-making, but important work. 
Outside initiativesConsortia or other organizations outside of CodeX with which the UC may want to interface, coordinate or partner, or at least remain aware of. Could be competitive.
  • CQI Work Group
  • QI-Core IG team
  • QM IG team
  • NCQA-related initiatives
    • XYZ??
  • CMS-related initiatives
    • CMMI Enhancing Oncology Model (EOM)
    • Quality Payment Program (MIPs/Enhancing Oncology Care MVP)
  • Abacus project
  • Gravity - oncology / equity related concepts in Gravity that might be applicable for QM use case


Add additional thoughts

ASCO - In 3/9 Presidential Budget in Brief: https://www.hhs.gov/sites/default/files/fy-2024-budget-in-brief.pdf. Page 75 mentions what appears to be a new 'proposal' to Expand Cancer Care Quality Measurement, but no specifics are given. I've reached out to Vinitha and Dr. Kline to try to get more intel.


Telligen: Suggest reaching out to AMA for assessment of interest/participation.   (non-competitive)





Plan

Agile, short Phases and success metrics for each PhaseOn the public page, list draft, high-level time-frames for Planning work and initial Phases during Execution.
  • DONE

HL7 FHIR IGs

Roughly, how existing FHIR IG might be used or updated.

Initial thoughts on the need to develop new IGs as stand-alone artifacts and/or supplementary to existing specific IGs, working with existing HL7 WGs and/or outside organizations, complex terminology work, balloting in HL7, etc.

  •  FHIR IG leveraging 

QMIG (Quality Measures implementation Guide)

mCODE STU2 (3?)

DEQM (Data Exchange Quality Measures)

FHIR 4.0.1 and forward.


Implementation in health information systems

Existing IT systems that might be candidates to be updated to support existing/new FHIR IGs and functionality during pilots.

New systems that  might need to be developed.


Telligen:  Recommend staying with proposed AWS environment  Future Forward phases may want to implement/coordinate with OMOP.

CentanniPark:  Ideally could target systems implementing mCode to provide data inputs for the QMs that can be housed downstream (AWS etc)
Piloting in real-world settingsInitial thoughts on the location and nature of pilots, preferable sharing data across real-world settings.
Telligen: Need to be exploring willing participants for DEQM endpoints.

CentanniPark:  Ideally could target systems implementing mCode to provide data inputs for the QMs that can be housed downstream (AWS etc)
RisksThe most important, potential challenges to success.  For example, divergent views in the community, competing efforts, insufficient resources, technical or clinical barriers, legal work, etc.    
Telligen:  Most significant risk is allocated resources having competing priorities and participation of endpoint organizations.  Lack of regulatory participation?
Adoption and scalingAssuming successful pilots, the 3-5 most important challenges to be addressed to gain adoption and scale for impact.
Telligen:  Most significant risk is allocated resources having competing priorities and participation of endpoint organizations.  Lack of regulatory participation?

CentanniPark:
  1.  Regulatory - prioritization
  2.  Implementation - documented use cases with clear guidelines
  3.  Automation and Standardization - documented benefits and savings




Aligning and Decision-Making
Meetings within the UCExperience has shown that a combination of some public and separate Member-only meetings is a good way to build broad interest, receive broad input, gain engagement and commitment, and still leverage Member meetings to make decisions.
  • Done

Decisions within the UCDecision makers (approval of all of the above) must be CodeX paying, Gov't and/or Sponsored Members or must submit written commitment to join at one of these levels if/when the UC transitions to Executing.
  • Done


  • what call are decisions made
  • where is our decision log / how are decisions tracked
  • who maintains the decision log

Decision regarding move to the next Stage
  • UC leaders requests that PM start process of considering transition to Planning, based on input to requirements above.
  • PM acknowledges receipt of request to proposer.
  • PM, within ~1 business day of acknowledging receipt, informs the OC and SC
  • OC and SC have ~5 business days to express concerns about moving the UC to Planning.
  • PM discusses any concerns with the proposer and the OC/SC, and is responsible for disposing of issues and informing all parties as to whether the proposed UC can move into Planning or not (with reasons).
 
  • Done

  • No labels