FHIR
Page tree
Skip to end of metadata
Go to start of metadata

Short Description

FHIR Testing at Scale: FHIR TestScript, other FHIR testing methods and lessons learned from national FHIR Testing Programs

Long Description

Explore how the FHIR Testing Framework (TestScript) is being used in national FHIR testing programs, and discuss FHIR testing tools that do not use the FHIR TestScript resource. 

The FHIR specification includes a Testing Framework that defines a Test Execution Engine and the TestScript resource for use in validating and verifying that FHIR implementations (client and/or server) are conformant to the FHIR specification. The inclusion of TestScript in the FHIR specification as a first-class resource reflects the community's experience that the management and proper use of testing are fundamental to effective, interoperable data exchange.

As the proliferation of FHIR implementations expands beyond Health IT early adopters, we are seeing a proliferation of FHIR testing tools and approaches. Although we think it would be much easier for everyone to rally around TestScript, it is more important to promote FHIR testing and interoperability between testing approaches so that FHIR implementation testing can proliferate along with the community.

  • A focus of this track will be to start defining an "interoperability framework" for FHIR testing tools.

We expect track participants to come away with a better appreciation of: 

  1. FHIR Testing and its benefits to improve FHIR implementations.
  2. Implementation and implementer experience with the FHIR Testing Framework and TestScript at scale
  3. The various FHIR testing approaches currently available.
  4. How to make FHIR testing tools interoperable–and why that is a great idea.
  5. Ideas for demonstrating FHIR testing interoperability at the next Connectathon.

Type

Educational/Brainstorming: FHIR Testing with TestScript, a first-lass FHIR resource; Testing engine interoperability; Lessons learned from FHIR testing at scale.

Submitting Work Group/Project/Accelerator/Affiliate/Implementer Group  

FHIR Implementers (FHIR-I)

Conformance

EHR

Track Lead(s)

Jeff Helman (primary), Joe Lamy

Track Lead Email(s)

jeff.helman@aegis.net; joe.lamy@aegis.net

Related Tracks

Da Vinci and Carin Blue Button are heavy users of the FHIR TestScript approach to validating implementations.

FHIR Version

FHIR R4, STU3

Specification(s) this track uses

FHIR Testing Framework: (Current Build) http://build.fhir.org/testing.html; (FHIR R4) http://hl7.org/fhir/testing.html
TestScript Resource: (Current Build) http://build.fhir.org/testscript.html; (FHIR R4) http://hl7.org/fhir/testscript.html
TestReport Resource: (Current Build) http://build.fhir.org/testreport.html; (FHIR R4) http://hl7.org/fhir/testreport.html

  • Terminology Servers this track uses? N/A
  • Specific Terminology Code Systems and Value Sets needed for this track? N/A

Artifacts of focus

FHIR Testing Framework: (Current Build) http://build.fhir.org/testing.html; (FHIR R4) http://hl7.org/fhir/testing.html
TestScript Resource: (Current Build) http://build.fhir.org/testscript.html; (FHIR R4) http://hl7.org/fhir/testscript.html
TestReport Resource: (Current Build) http://build.fhir.org/testreport.html; (FHIR R4) http://hl7.org/fhir/testreport.html

Expected participants

Expected attendees (please feel free to add your name or let us know if you will be participating):

Vendors and Implementers seeking implementation guidance on national testing programs

  • AEGIS - Touchstone tool and test scripts
  • MITRE - Inferno Framework testing tool
  • ONC - John Bender and team: Call for interoperability between testing engines.
  • Drummond - Payer and Patient Access FHIR® API Certification Program, ONC 2015 Edition Cures Update Health IT Certification  

Zulip stream

https://chat.fhir.org/#narrow/stream/274423-TestScript-Resource/topic/May.202022.20Connectathon

Track Kick Off Call

  • Fri, Apr 29, 2022 1:00 PM - 2:00 PM (EDT)

    Track kick-off recording (TO UPLOAD)

Track Details

Prerequisites

  • For all levels of testing the required prerequisite is the fundamental requirement that all FHIR implementations SHALL support the capabilities interaction.
  • Review and have a basic understanding of the FHIR TestScript Resource Type definition.
  • Review and have a basic understanding of the FHIR Testing Framework definition.

Whova Sessions

Tuesday, May 3rd: 11:00 am - 1:00 pm EDT - Session 1

  • FHIR TestScript QuickStart: From zero to hero

History of, and introduction to, the FHIR TestScript resource and the various approaches being used for FHIR implementation testing.

Tuesday, May 3rd: 2:00 pm - 4:00 pm EDT - Session 2

  • Improving FHIR TestScript: Working brainstorming session

Continuing the tradition during Connectathon of exploring how the FHIR TestScript resource can be improved, this session will dive in for a meaty discussion about interoperability between FHIR testing engines and what that means for the FHIR TestScript resource.

Wednesday, May 4th: 10:00 am - Noon EDT  - Session 3

  • Working session/hands-on/other topics

Roll up your sleeves with FHIR testing and the creation of FHIR TestScript resources.
We will explore how FHIR Shorthand (FSH) can be used for TestScript authoring, and work through other topics and questions you have.

    • Advanced preparation - View the Intro to FHIR Shorthand video about efficiently generating FHIR TestScript Resources. 

Wednesday, May 4th: 1:00 pm - 3:00 pm EDT - Session 4

  • TestScript at Scale: Lessons learned from national testing program(s)

Lessons learned from large-scale testing programs to certify (validate) FHIR implementations--and what we can take away from those lessons to improve our own work on FHIR implementations and testing.

(Schedule TBD) - Possible Session 5

  • Future of FHIR Testing - e.g. Workflow, Ecosystem, Multi-version Support; Mario Hyland (TBD)

User Roles

These are the human actors that have the requisite subject matter knowledge and expertise.

  • Test Scenario Author/SME
  • TestScript Author
  • Test Execution Results Reviewer

System roles

These are the software implementations/applications that perform the actual testing execution processes.

FHIR Testing Engine/Platform

FHIR Client (System Under Test)

  • The external FHIR Client actor system under test.

FHIR Server (System Under Test)

  • The external FHIR Server actor system under test.

Scenarios 

  • This instance of the FHIR Testing track is focusing on lessons learned, as well as ways to improve, the FHIR testing process, so there are no specific testing scenarios for attendees to exercise.
    • However, during Sessions 1 and 3 we will be exploring hands-on FHIR testing topics as a group.