Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Continue testing Quality Measurement use cases.
    1. Evaluate FHIR-based eCQMs written with CQL.
    2. Test eCQM structure, packaging, and reference libraries from draft MAT on FHIR export packages.
    3. Test and validate the use of the QI-Core model in CQL authoring.
    4. Test supplemental data use cases for eCQMs.
    5. Test continuous variable and stratified eCQMs.
  2. Test the use of FHIR resources in alignment with FHIR R4 Implementation Guides (IG).
    1. QI-Core IG.
    2. Quality Measure IG.
    3. Data Exchange for Quality Measures (DEQM) IG.
  3. Test FHIR Clinical Guidelines example content (in coordination with the Care Planning and Public Health tracks).
  4. Test the new `order-select` hook using CDC Opioid Prescribing (in coordination with the CDS Hooks track).

  5. Conduct end-to-end testing: identify a gap, close gap and report measure, specifically Breast Cancer Screening--CMS 125v8 or v9 (in coordination Coordinate with the DaVinci DEQM Gaps in Care track)Track to examine end-to-end testing.
  6. Continue investigation of bulk import support.

  7. Test the ExecutableLibrary profile.

  8. Test the following CMS Measures for R4

Eligible Professional (EP)/Eligible Clinician (EC) Measures
2019 Reporting Measures

2020 Reporting Measures

2021 Reporting Measures

Other Measures for Consideration

  • CMS130v7: Colorectal Cancer Screening
  • CMS125v7: Breast Cancer Screening
  • CMS165v8: Controlling High Blood Pressure
  • CMS349v2: HIV Screening
  • CMS124v8: Cervical Cancer Screening
  • CMS74v10: Primary Caries Prevention Intervention as Offered by Primary Care Providers, including Dentists
  • CMS124v9: Cervical Cancer ScreeningCMS125v9: Breast Cancer Screening
  • CMS149v9: Dementia: Cognitive Assessment
  • CMS153v9: Chlamydia Screening for Women
  • CMS347v4: Statin Therapy for the Prevention and Treatment of Cardiovascular Disease
  • CMS127v9: Pneumococcal Vaccination Status for Older Adults
  • CMS146v9:Appropriate Testing for Children with Pharyngitis
  • CMS154v9: Appropriate Treatment for Children with Upper Respiratory Infection (URI)
  • CMS155v9: Weight Assessment and Counseling for Nutrition and Physical Activity for Children and Adolescents
  • CMS159v9: Depression Remission at Twelve Months

...

Specifications and Artifacts of Focus:

...

Related Tracks: Care Planning, Public Health, CDS Hooks, DaVinci DEQM Gaps in Care and PACIO/eLTSS

Proposed Track Lead: Bryn Rhodes, bryn@databaseconsultinggroup.com

Expected Participants (feel free to add your organization)

Alara Imaging, Inc

American Academy of Neurology

Bellese Technologies

Commure, Inc

DCG

Dynamic Health IT

EBSCO (via EBM-on-FHIR)

eHealth NSW

Epic (via CDS Hooks)

ESAC, Inc

Flexion, Inc

HRSA

IMPAQ International, LLC

Indicina, LLC

iParsimony, LLC

Lantana Consulting Group

Mathematica

SemanticBits

Medisolv, Inc

NCQA

Optum

Oregon Urology Institute and Large Urology Group Practice Association

Philips Healthcare

PJM Consulting, LLC

SAMHSA

SemanticBits

Telligen

The Joint Commission

The MITRE Corporation

UPMC Enterprises

Yale-Center for Outcomes Research and Evaluation

Zulip Stream: https://chat.fhir.org/#narrow/stream/179207-connectathon-mgmt/topic/Clinical.20Reasoning.20Track

Track Orientation:

Getting Started:

  • Getting Started

...

Artifacts During Connectathon:

  • Track Schedule:
  • Track Running Notes: 

System Roles (Quality Reporting)

  • Producer - System of record for clinical information such as an EHR
  • Consumer - A system that aggregates data from multiple sites related to reporting quality measures such as a payer, HIE, reporting vendor or public health registry
  • Reporter - A system that sends quality reporting data and results
  • Receiver - A system that receives quality reporting data and results

Systems capable of playing these roles should implement the exchange and reporting scenarios as described in the Data Exchange for Quality Measures implementation guide.

The CQF Ruler, a reference implementation of FHIR Clinical Reasoning based on the HAPI FHIR Server is available to either play the Consumer, Receiver, and CDS Service roles, or to help guide and test implementations in preparation for the track.

A test instance of the CQF Ruler is available here: http://cqm-sandbox.alphora.com/

Data Exchange Scenarios

For these scenarios, the Producer and Consumer exchange the data-of-interest for a quality measure, either by push or pull

This is the EHR Reference Implementation server endpoint for the Medication Reconciliation Post Discharge use case to test the DEQM $submit-data operation - https://api-v8-stu3.hspconsortium.org/DaVinciMRPProvider/open

This is the Payer Reference Implementation server endpoint for this use case - https://api-v8-stu3.hspconsortium.org/DaVinciMRPPayer/open

Individual Exchange Scenarios

Push Scenario ($submit-data)
  • Action: An EHR playing the role of the Producer uses the $submit-data operation to report the data-of-interest for an MRP measure
  • Precondition: The EHR has appropriate data to produce the data-of-interest for the measure, either based on the test data, or by walking through a simulated workflow that produces the data required
  • Success Criteria: The Consumer receives the data and successfully processes the data
  • Bonus Point: Run the operation multiple times throughout a simulated measurement period to feed incremental data of interest for the measure to the Consumer
Pull Scenario ($collect-data)
  • Action: A Consumer uses the $collect-data operation to gather the data-of-interest for an MRP
  • Precondition: The EHR has appropriate data to produce the data-of-interest for the measure, either based on the test data, or by walking through a simulated workflow that produces the data required
  • Success Criteria: The Producer responds to the $collect-data and the Consumer is able to successfully process the response
  • Bonus Point: Run the operation multiple times throughout a simulated measure period to collect incremental data of interest for the measure

Group Exchange Scenarios

For the Group Exchange scenarios, we focus on the need for payers to track status of members for a screening measure, a Colorectal Cancer Screening measure in this case. The scenarios are based on the case where a payer sends a request to a provider for the screening information for a group of covered patients.

These scenarios will attempt to use the Bulk Data API as specified in the FHIR specification. In particular:

  • $collect-data will use bulk data to communicate the response
  • $submit-data will use an implementation of bulk data import to communicate the payload of the request
Push Scenario ($submit-data)

In the push scenario, the provider - through some interface in their system - gathers the screening information for the set of patients and submits it:

  • Action: An EHR playing the role of the Producer uses a batch of $submit-data interactions to report the data-of-interest for the COL measure for the set of patients
  • Precondition: The EHR has appropriate data to produce the data-of-interest for the measure, either based on the test data, or by walking through a simulated workflow that produces the data required
  • Success Criteria: The Consumer receives the data and successfully processes the data
  • Bonus Point: The provider system uses the Bulk Data format to send the information
Pull Scenario ($collect-data)

In the pull scenario, the payer uses the $collect-data operation to request through the provider's API

Artifacts During Connectathon:

System Roles (Quality Reporting)

  • Producer - System of record for clinical information such as an EHR
  • Consumer - A system that aggregates data from multiple sites related to reporting quality measures such as a payer, HIE, reporting vendor or public health registry
  • Reporter - A system that sends quality reporting data and results
  • Receiver - A system that receives quality reporting data and results

Systems capable of playing these roles should implement the exchange and reporting scenarios as described in the Data Exchange for Quality Measures implementation guide.

The CQF Ruler, a reference implementation of FHIR Clinical Reasoning based on the HAPI FHIR Server is available to either play the Consumer, Receiver, and CDS Service roles, or to help guide and test implementations in preparation for the track.

A test instance of the CQF Ruler is available here: http://cqm-sandbox.alphora.com/

Data Exchange Scenarios

For these scenarios, the Producer and Consumer exchange the data-of-interest for a quality measure, either by push or pull

This is the EHR Reference Implementation server endpoint for the Medication Reconciliation Post Discharge use case to test the DEQM $submit-data operation - https://api-v8-stu3.hspconsortium.org/DaVinciMRPProvider/open

This is the Payer Reference Implementation server endpoint for this use case - https://api-v8-stu3.hspconsortium.org/DaVinciMRPPayer/open

Individual Exchange Scenarios

Push Scenario ($submit-data)
  • Action: An EHR playing the role of the Producer uses the $submit-data operation to report the data-of-interest for an MRP measure
  • Precondition: The EHR has appropriate data to produce the data-of-interest for the measure, either based on the test data, or by walking through a simulated workflow that produces the data required
  • Success Criteria: The Consumer receives the data and successfully processes the data
  • Bonus Point: Run the operation multiple times throughout a simulated measurement period to feed incremental data of interest for the measure to the Consumer
Pull Scenario ($collect-data)
  • Action: A Consumer uses the $collect-data operation to gather the data-of-interest for an MRP
  • Precondition: The EHR has appropriate data to produce the data-of-interest for the measure, either based on the test data, or by walking through a simulated workflow that produces the data required
  • Success Criteria: The Producer responds to the $collect-data and the Consumer is able to successfully process the response
  • Bonus Point: Run the operation multiple times throughout a simulated measure period to collect incremental data of interest for the measure

Group Exchange Scenarios

For the Group Exchange scenarios, we focus on the need for payers to track status of members for a screening measure, a Colorectal Cancer Screening measure in this case. The scenarios are based on the case where a payer sends a request to a provider for the screening information for a group of covered patients.

These scenarios will attempt to use the Bulk Data API as specified in the FHIR specification. In particular:

  • $collect-data will use bulk data to communicate the response
  • $submit-data will use an implementation of bulk data import to communicate the payload of the request
Push Scenario ($submit-data)

In the push scenario, the provider - through some interface in their system - gathers the screening information for the set of patients and submits it:

  • Action: An EHR playing the role of the Producer uses a batch of $submit-data interactions to report the data-of-interest for the COL measure for the set of patients
  • Precondition: The EHR has appropriate data to produce the data-of-interest for the measure, either based on the test data, or by walking through a simulated workflow that produces the data required
  • Success Criteria: The Producer responds to the $collect-data batch and the Consumer is able to successfully process the responsesConsumer receives the data and successfully processes the data
  • Bonus Point: The payer provider system uses the Bulk Data format to request send the information

Reporting Scenarios

For these scenarios, a Reporter and Receiver exchange the data and results of a quality measure.

Individual Report Scenario

...

Pull Scenario ($collect-data)

In the pull scenario, the payer uses the $collect-data operation to request through the provider's API

  • Action: A Consumer uses a batch of $collect-data interactions to gather the data-of-interest for a COL measure for a set of patients
  • Precondition: The EHR has appropriate data to produce the individual MeasureReport data-of-interest for the measure, either based on the test data, or by walking through a simulated workflow that produces the data required
  • Success Criteria: The Receiver receives a completed MeasureReport and verifies that it has the correct data and the calculated result is correct for the individual
  • Bonus Point: Use a subscription to notify the Receiver when the report is ready for retrieval

...

  • Producer responds to the $collect-data batch and the Consumer is able to successfully process the responses
  • Bonus Point: The payer system uses the Bulk Data format to request the information

Reporting Scenarios

For these scenarios, a Reporter and Receiver exchange the data and results of a quality measure.

Individual Report Scenario

  • Action: A Reporter reports the data and results for a Summary an Individual MRP, COL, or VTE-1 measure to a Receiver
  • Precondition: The Reporter has appropriate data to produce the summary individual MeasureReport for the measure
  • Success Criteria: The Receiver receives a completed MeasureReport and verifies that it has the expected result correct data and the calculated result is correct for the populationindividual
  • Bonus Point: Use a subscription to notify the Receiver when the report is ready for retrieval

Calculation Scenarios

For these scenarios, a Receiver requests the calculation of a quality measure by a Reporting system.

...

Summary Report Scenario

  • Action: A Receiver uses $evaluate-measure to request the data and results for an Individual Reporter reports the results for a Summary MRP, COL, or VTE-1 measure from to a ReporterReceiver
  • Precondition: The Reporter has appropriate data to produce the individual summary MeasureReport for the measure
  • Success Criteria: The Receiver receives a completed MeasureReport and verifies that it has the correct data and the calculated result is correct expected result for the individualpopulation
  • Bonus Point: Use asynchronous result calculation (MeasureReport status = in-progress)

...

  • a subscription to notify the Receiver when the report is ready for retrieval

Calculation Scenarios

For these scenarios, a Receiver requests the calculation of a quality measure by a Reporting system.

Individual Report Scenario

  • Action: A Receiver uses $evaluate-measure to request the data and results for a Summary an Individual MRP, COL, or VTE-1 measure from a Reporter
  • Precondition: The Reporter has appropriate data to produce the summary individual MeasureReport for the measure
  • Success Criteria: The Receiver receives a completed MeasureReport and verifies that it has the expected result correct data and the calculated result is correct for the populationindividual
  • Bonus Point: Use asynchronous result calculation (MeasureReport status = in-progress)

Public Health Reporting Scenario

System Roles (PHR)

Clinical System - A system that deals with health data and potentially reportable events
Reporting Application - A system that focuses on identifying and submitting potentially reportable events
Receiving System - A system that receives potentially reportable events

Negative Chlamydia Screening Report Scenario

  • Action: A Reporting Application is notified by a Clinical System of a potentially reportable event, and uses the eRSD specification to:
    • Determine the jurisdiction of residence and jurisdiction of care for the event
    • Determine whether the event is potentially reportable, given condition-specific value sets and jurisdiction-specific criteria as specified in the rule filter portion of the eRSD specification
    • If the event is potentially reportable, submit a mock eICR to the receiving system
  • Precondition: The Clinical System has appropriate data representing a negative chlamydia screening result
  • Success Criteria: The receiving system is notified of and receives a potentially reportable eICR for a negative chlamydia screening result
  • Bonus Point: Demonstrate the pattern is effective for other conditions that exhibit similar potential to reduce the number of potentially reportable events submitted through jurisdiction-specific and condition-specific configuration

Decision Support Scenarios

System Roles (CDS)

CDS Service - A system that provides decision support guidance via the CDS Hooks API
CDS Client - A system that consumes decision support via the CDS Hooks API

...

Summary Report Scenario

  • Action: A Receiver uses $evaluate-measure to request the results for a Summary MRP, COL, or VTE-1 measure from a Reporter
  • Precondition: The Reporter has appropriate data to produce the summary MeasureReport for the measure
  • Success Criteria: The Receiver receives a completed MeasureReport and verifies that it has the expected result for the population
  • Bonus Point: Use asynchronous result calculation (MeasureReport status = in-progress)

Public Health Reporting Scenario

System Roles (PHR)

Clinical System - A system that deals with health data and potentially reportable events
Reporting Application - A system that focuses on identifying and submitting potentially reportable events
Receiving System - A system that receives potentially reportable events

Negative Chlamydia Screening Report Scenario

  • Action: A Reporting Application is notified by a Clinical System of a potentially reportable event, and uses the eRSD specification to:
    • Determine the jurisdiction of residence and jurisdiction of care for the event
    • Determine whether the event is potentially reportable, given condition-specific value sets and jurisdiction-specific criteria as specified in the rule filter portion of the eRSD specification
    • If the event is potentially reportable, submit a mock eICR to the receiving system
  • Precondition: The Clinical System has appropriate data representing a negative chlamydia screening result
  • Success Criteria: The receiving system is notified of and receives a potentially reportable eICR for a negative chlamydia screening result
  • Bonus Point: Demonstrate the pattern is effective for other conditions that exhibit similar potential to reduce the number of potentially reportable events submitted through jurisdiction-specific and condition-specific configuration

Decision Support Scenarios

System Roles (CDS)

CDS Service - A system that provides decision support guidance via the CDS Hooks API
CDS Client - A system that consumes decision support via the CDS Hooks API

Systems capable of playing these roles should implement the CDS Hooks API. Systems may also support ingestion of computable decision support content as described in the Clinical Guidelines implementation guide.

...

TestScript(s): All testing materials can be accessed from the Connectathon GitHub Repository: https://github.com/dbcg/connectathonSecurity and Privacy Considerations: The scenarios and reference implementations here run using open (i.e. unsecured) connections. Systems SHALL NOT use PHI in any form, or data derived directly from PHI.//github.com/dbcg/connectathon

Security and Privacy Considerations: The scenarios and reference implementations here run using open (i.e. unsecured) connections. Systems SHALL NOT use PHI in any form, or data derived directly from PHI.


Track Report Out

Summary: The Clinical Reasoning Track continued the testing and use of FHIR-based Quality Measures for use in Quality Measurement programs, including CMS and Clinical Decision Support (CDS) Use Cases. The Track also tested the following implementation guides (IG):

  • QI-Core IG
  • Quality Measure IG
  • Data Exchange for Quality Measures IG
  • Clinical Practice Guidelines IG

Participants

Image Added

Systems which have implemented the IG, Profile, or Resource:

  • MAT-on-FHIR: Quality Measure IG, ~80% (shareable, executable, computable for library, measure, and artifact bundles, still working on publishable for library and measure, also implements measure-specific profiles for proportion and continuous variable)
  • Commure: Quality Measure IG ~80% (continuous-variable, cohort, and proportion measures, don’t yet support ratio measures, don’t have complete stratification support or supplemental data)
  • Data Exchange for Quality Measure IG ~80% ($evaluate-measure, $data-requirements, $collect-data, successfully tested at least 50% of the connectathon tests)
  • CQF-Ruler (reference implementation): Quality Measure IG: ~80% (continuous-variable (ish), cohort, and proportion, ratio, supplemental data, don’t yet support stratification)
  • Data Exchange for Quality Measures: ~80% ($evaluate-measure, $submit-data, $collect-data, successfully tested 99% of the connectathon tests)
  • Dynamic Health IT: Progress with measure 506, integrating FHIR Server and Quality Tools with $evaluate-measure, testing focused on 506, but robust CQL implementation will likely support other measures as well

Notable Achievements: 

  • Successfully exported measure packages from MAT-on-FHIR and imported (mostly) those measures. Found and reported several issues that are currently being worked on
  • Progress with testing hospital stroke measure on PACIO data, encountering evaluation failures, but working through them
  • Progress with testing continuous variable measures, found and fixed several issues, more outstanding are still being worked
  • Gaps-in-Care end-to-end test with GIC track, used a test patient from Clinical Reasoning, got an “open gap” report, posted a submit-data with closing data and got a “closed gap” report, as well as a summary. Uncovered some issues with $collect-data and $evaluate-measure (summary), those issues are being worked on
  • Successful testing with supplemental data for multiple hospital and provider measures--124, 125, 111, 104, 108, 529

Screenshots of Implementations/Achievements:

MAT-on-FHIR Export of VTE Measure
Image Added


Gaps-in-Care End-to-End Scenario

Image Added

Image Added



Discovered Issues/Questions:
https://github.com/cqframework/clinical_quality_language/issues/564
https://github.com/DBCG/cqf-ruler/issues/250
https://github.com/DBCG/cql_engine/pull/396
https://github.com/DBCG/cqf-ruler/issues/246
https://github.com/DBCG/cqf-ruler/issues/251
https://github.com/DBCG/cql-evaluator/pull/21

  • Reference implementation of $collect-data does not include all data, only the data collection report
  • Reference implementation of summary $evaluate-measure includes too much data, should only be the aggregate result
  • Found and fixed several library resolution errors in the reference implementation, which raised the larger question about whether the use of underscores in library names should be allowed or should be added as a convention

Now What?

  • Complete continuous variable implementation
  • Stratification results
  • Multiple-population measure testing
  • Additional QI-Core Authoring support/testing
  • Composite measures


Supporting the September 2020 Virtual Connectathon...

Image Added