Table of Contents:

Project Coordination Updates

Suggested agenda for the week:

Friday, September 29:

• 8 am Eastern: GRADE Ontology Working Group – start term development; attention to Certainty of evidence
• 9 am Eastern: Risk of Bias Terminology Working Group – review SEVCO terms; attention to Predictive Model Research Bias (4 terms open for vote)
• 10 am Eastern: Communications Working Group – presentations (MCBK for October 3), publications (Introduction to EBMonFHIR, Study design terminology)
• 12 pm Eastern: Eligibility Criteria Working Group (HL7 CDS EBMonFHIR sub-WG) – apply approved changes to Group Resource and RelativeTime Datatype; facilitate participants adapting to the newly approved changes to Group Resource

Monday, October 2:  

• 8 am Eastern: Project Management – review EBM Implementation Guide progress for January ballot
• 9 am Eastern: Setting the Scientific Record on FHIR Working Group – evaluate additional sources for conversion to FHIR (GRADEpro, then Cochrane on October 9)
• 10 am Eastern: CQL Development Working Group (HL7 CDS EBMonFHIR sub-WG) – evaluate how CQL fits with recently approved Group Resource model and FEvIR tooling
• 2 pm Eastern: Statistic Terminology Working Group – review SEVCO terms; attention to Probability Distribution Attributes terms (9 terms open for vote)

Tuesday, October 3:

• 9 am Eastern: Measuring the Rate of Scientific Knowledge Transfer Working Group – CANCELED (to allow attendance in MCBK Global meeting)
• 2 pm Eastern: StatisticsOnFHIR Working Group (HL7 CDS EBMonFHIR sub-WG) – continue ANCOVA model characteristic details in example EndpointAnalysisPlan Profile for PHUSE Lilly Redacted Protocol
• 3 pm Eastern: Ontology Management Working Group – review progress on tasks, attention to case-sensitive changes

Wednesday, October 4:

• 8 am Eastern: Funding the Knowledge Ecosystem Infrastructure Working Group – CANCELED (to allow attendance in MCBK Global meeting)

Thursday, October 5:

• 8 am Eastern: EBM Implementation Guide Working Group (HL7 CDS EBMonFHIR sub-WG) – review EBM Implementation Guide progress for January ballot
• 9 am Eastern: Computable EBM Tools Development Working Group – review FEvIR® Platform developments; Recommendation Adaptation or Summary of Net Effect related developments
• 4 pm Eastern: Project Management – prepare weekly agenda

Friday, October 6:

• 8 am Eastern: GRADE Ontology Working Group – continue term development
• 9 am Eastern: Risk of Bias Terminology Working Group – review SEVCO terms; attention to Predictive Model Research Bias
• 10 am Eastern: Communications Working Group – publications (Introduction to EBMonFHIR, Study design terminology), presentations
• 12 pm Eastern: Eligibility Criteria Working Group (HL7 CDS EBMonFHIR sub-WG) – facilitate participants adapting to the newly approved changes to Group Resource

Project Management Updates:

With 2023, the 'COVID-19 Knowledge Accelerator' was renamed Health Evidence Knowledge Accelerator (HEvKA).

The HEvKA email distribution list is used to send a short update each weekday and a longer summary update each weekend. A HEvKA weekly email distribution list is an alternative for people who want to only receive the weekly update.

There are currently 15 active working meetings weekly:

• Project Management – Mondays at 8-9 am Eastern
• Setting the Scientific Record on FHIR – Mondays at 9-10 am Eastern
• CQL Development – Mondays at 10-11 am Eastern (an HL7 CDS EBMonFHIR sub-WG)
• Statistic Terminology – Mondays at 2-3 pm Eastern
• Measuring the Rate of Scientific Knowledge Transfer – Tuesdays at 9-10 am Eastern (last Tuesday of month is Scientific Knowledge Accelerator Foundation Board of Directors meeting)
• StatisticsOnFHIR – Tuesdays at 2-3 pm Eastern (an HL7 CDS EBMonFHIR sub-WG)
• Ontology Management - Tuesdays at 3-4 pm Eastern
• Funding the Ecosystem Infrastructure– Wednesdays at 8-9 am Eastern
• Evidence Based Medicine Implementation Guide – Thursdays at 8-9 am Eastern (an HL7 CDS EBMonFHIR sub-WG)
• Computable EBM Tools Development – Thursdays at 9-10 am Eastern
• Project Management – Thursdays at 4-5 pm Eastern
• GRADE Ontology – Fridays at 8-9 am Eastern
• Risk of Bias Terminology – Fridays at 9-10 am Eastern
• Communications – Fridays at 10-11 am Eastern
• Eligibility Criteria – Fridays at 12-1 pm Eastern (an HL7 CDS EBMonFHIR sub-WG)

Project Management 2023 Quarter 1

On January 5, 2023, the Project Management Group coordinated changes across the HEvKA daily update (email message), the HEvKA Project Page on FEvIR Platform (modified to include the agenda), and the HEvKA Project Page on HL7 Confluence (and an additional HEvKA Update Summary page on HL7 Confluence). We also started the process of canceling the prior COKA meetings and creating new HEvKA meetings (for those who want meeting invites on your calendar). You may see this continue to develop over the coming week. We then developed a suggested agenda for the next 8 days.

On January 9, 2023, the Project Management Group completed sending meeting invites for HEvKA working groups. You are welcome to join any meeting and can let us know if you want to be added to a calendar invite. We also added a GuidelineCitation Profile to the Evidence Based Medicine Implementation Guide.

On January 23, the Project Management Group reviewed many changes released today with Fast Evidence Interoperability Resources (FEvIR®) Platform version 0.99.0. FEvIR®: ActivityDefinition Builder/Viewer version 0.14.0, and Computable Publishing®: ClinicalTrials.gov-to-FEvIR® Converter version 3.7.1; reviewed the initial version of the Classification Profile (in the EBM IG and on FEvIR® Platform) and the Adaptation Profile (in the EBM IG and on FEvIR® Platform); reviewed and revised Appendix C in the developing resubmission of the Introduction to EBMonFHIR manuscript; and drafted the following for proposed presentation in FHIR DevDays:

Tutorial -- Structuring Eligibility Criteria and Cohort Definitions with FHIR

This tutorial will introduce FHIR structures for characteristic expression (e.g. definitionByTypeAndValue, definitionByCombination). There are many use cases for expressing a set of characteristics to define a group -- for example, eligibility criteria, inclusion/exclusion criteria, cohort definition, phenotype, or population.
In response to requests to support expressions of characteristics without the Expression datatype, the EvidenceVariable StructureDefinition was modified to support multiple methods of expression.
Free tooling (and Profiles) in development for human-friendly viewing and data entry of characteristics, and for sharing of characteristics and related logic expressions to match patient data against characteristics, will be introduced. 
The characteristic concept is used across Group, EvidenceVariable, and Measure Resources and there is a need to make the characteristic definitions shareable in isolation from the Resources.

On February 6, the Project Management Group was pleased to announce the publication of:

• Representation of evidence-based clinical practice guideline recommendations on FHIR [Journal Article]. Contributors: Lichtner G, Alper BS, Jurth C, Spies C, Boeker M, Meerpohl JJ, von Dincklage F. In: Journal of biomedical informatics, PMID 36738871. Published February 02, 2023. Available at: https://pubmed.ncbi.nlm.nih.gov/36738871/. 
  Description: The first comprehensive implementation guide for representation of clinical practice guidelines in EBMonFHIR-created Resources

and also drafted a title for a Guidelines International Network (GIN) 2023 pre-conference course (Using technology to efficiently update and adapt guidelines) and drafted 3 aims and objectives for the course proposal:

1. Learn the basic principles used in the application of technology to facilitate guideline content modification.
2. Learn how efficient guideline content modification can be applied to workflows of updating guidelines and guideline adaptation.
3. Use open tools to update or adapt a guideline (applied to your own guideline if shared ahead in structured format)

On February 13, the Project Management Group reviewed and revised improvements to the data entry form for Identifier datatype on the FEvIR Platform, and improvements to the display of Coding datatype values which now include the code system in addition to the code and display values.

On February 16, the Project Management Group applied the ResearchSubject example to the FHIR specification.

On February 20, the Project Management Group started to apply changes to the FHIR specification including:

• breaking up research-study-classifiers CodeSystem to revert to separate code systems for each value set in ResearchStudy Resource (to match specifications needed for R5)
• FHIR-38937 - Getting issue details... STATUS – adding search parameters for the Citation.classification element

On February 27, the Project Management Group drafted a 10-minute orientation presentation for the EuroVulcan Eligibility Criteria Connectathon Track for the EUROVULCAN Conference and Connectathon in Paris March 14^th and 15^th .

On March 6, the Project Management Group reviewed the many enhancements to FEvIR®: Adaptation Builder/Viewer.

On March 13, the Project Management Group prepared for a presentation on Eligibility Criteria for the EuroVulcan Connectathon occurring tomorrow.

On March 20, the Project Management Group revised the handling of Classification Profile ArtifactAssessment Resources for Citations to FHIR Resources on the FEvIR Platform to support display under Third-Party Classifiers in the Citation Viewer.

On March 27, the Project Management Group prepared the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission to the Clinical Trials journal and continued development of creating a master index of ArtifactAssessment Resource content and Citation.citedArtifact.classification content on the FEvIR Platform. 

Project Management 2023 Quarter 2

On April 3, the Project Management Group reviewed the revised model for the HEvKA Update Summary page to group Working Group updates by quarter (3-month blocks) and create easier-to-follow sections, including a new section for Cohort Definition (Eligibility Criteria) Updates. 

On April 10, the Project Management Group revised Profiles of Evidence for the EBM Implementation Guide, including BaselineMeasureEvidence, EvidenceSynthesisEvidence, OutcomeMeasureEvidence, and SingleStudyEvidence; and wrote descriptions for two educational sessions to conduct during the HL7 2023 May FHIR Connectathon:

1. Cohort Definition Made Simple with FHIR

Whatever you call it -- cohort definitions, eligibility criteria, phenotypes, prior authorization criteria, inclusion and exclusion criteria, population definitions for quality measures – there is a very common use case for data exchange regarding the characteristics (criteria) that define whether one is or is not a member of a group. This session will introduce current (R5) and proposed (R6) FHIR structures for cohort definitions, and tooling to create them without requiring FHIR expertise.

      2. Statistics on FHIR: Representing a Statistical Analysis Plan

The Evidence Resource provides detailed structure for statistics including sample size, statistic type, attribute estimates, and model characteristics. The information model is sufficiently mature to represent an endpoint analysis plan, which is a building block for the Statistical Analysis Plan (SAP). This session will introduce the StatisticalModel Profile of Evidence Resource and the related terminology being developed in the Scientific Evidence Code System (SEVCO). Join us if you would like to be part of future projects to express SAPs in FHIR.

On April 17, the Project Management Group revised the NetEffectEstimate Profile to include Evidence.statistic.statisticType = https://fevir.net/resources/CodeSystem/27270#STATO:0000424 "Risk Difference"

On April 24, the Project Management Group adjusted the weekly agenda at https://fevir.net/resources/Project/29272 and revised the EvidenceReport Profile of Composition to add extensions for metadata for canonical resource management (versionAlgorithm, experimental, description, purpose, copyright, copyrightLabel, approvalDate, lastReviewDate, effectivePeriod). On Thursday we will create a value set to add 4 codes (author, editor, reviewer, endorser) for use with binding to Composition.attester.mode and this will complete our setup for EvidenceReport Profile to use for many other profiles of types of evidence reports.

On May 1, the Project Management Group reviewed the agenda for the HL7 Work Group Meeting May 8-12 and how this affects HEvKA meetings for the week. The following HEvKA meetings will be cancelled (Project Management, Setting the SRDR Platform on FHIR, ResearchOnFHIR, Statistics Standard and Terminology, Research Design, Knowledge Ecosystem Liaison).  The following HEvKA meeting will be determined by participants this week (Measuring the Rate of Scientific Knowledge Transfer).  The Thursday and Friday meetings (except for Project Management) will proceed as scheduled.

On May 22, the Project Management Group planned logistics for today's meetings and noted next week is Memorial Day so do not plan to meet next week.

On May 25, the Project Management Group updated the FHIR Trackers across EBMonFHIR:

On June 1, the Project Management Group noted that our efforts will be included in the 2023 Cochrane Methods Symposium on September 3 in London.

On June 5, the Project Management Group reviewed the changes to FEvIR®: Comparison Builder/Viewer, FEvIR®: EvidenceReport Builder/Viewer, and FEvIR®: Summary Of Findings Builder/Viewer.

On June 12, the Project Management Group prepared a response to the sets of responses to the Changes to Group for CohortDefinition as a GoogleSheet, and modified the FHIR specification to adjust the Evidence.statistic.modelCharacteristic elements to support the expression of an Endpoint Analysis Plan.

On June 19, the Project Management Group noted that our Cohort Definition developments may be addressed in the agenda for tomorrow's meeting of the HL7 Learning Health System (LHS) Work Group which meets at Tuesday 1 pm Eastern, updated the FHIR Trackers to note we applied FHIR-41379, and continued development of the TableConstructor Profile of Composition.

On June 22, the Project Management Group spent considerable time reviewing the overall set of working meetings instead of setting the line-item agenda for next week. We believe it is time to reset across the Working Group meetings.  Our current thinking is described below and we welcome feedback (by email or in person at tomorrow's Communication or Monday's Project Management meeting) as we work through the logistics.

There are currently 14 active working meetings weekly:

• Project Management – Mondays at 8-9 am Eastern
• Setting the SRDR Platform on FHIR – Mondays at 9-10 am Eastern
• ResearchOnFHIR – Mondays at 10-11 am Eastern
• Statistic Standard and Terminology – Mondays at 2-3 pm Eastern
• Measuring the Rate of Scientific Knowledge Transfer – Tuesdays at 9-10 am Eastern (last Tuesday of month is Scientific Knowledge Accelerator Foundation Board of Directors meeting)
• Research Design – Tuesdays at 2-3 pm Eastern
• Knowledge Ecosystem Liaison – Wednesdays at 8-9 am Eastern
• Evidence Based Medicine Implementation Guide – Thursdays at 8-9 am Eastern (an HL7 CDS EBMonFHIR sub-WG)
• Computable EBM Tools Development – Thursdays at 9-10 am Eastern
• Project Management – Thursdays at 4-5 pm Eastern
• GRADE Ontology – Fridays at 8-9 am Eastern
• Risk of Bias Terminology and Tooling – Fridays at 9-10 am Eastern
• Communications – Fridays at 10-11 am Eastern
• Eligibility Criteria – Fridays at 12-1 pm Eastern

For the 7 Working Groups underlined above, we are not recommending any changes to the scope of these working groups. (Timing may change but that is a separate logistics consideration than setting the scope of Working Group meetings.)

We considered many projects that are currently underway or desired across the HEvKA efforts and suggest as a first draft the following changes to the remaining Working Groups to adjust to current priorities:

• Project Management – retain 1 weekly meeting for overall HEvKA management; convert 1 weekly meeting to an Ontology Management Working Group with a scope of SEVCO management, system development for ontology management, and coordination with HL7 terminology systems
• ResearchOnFHIR – change this to a CQL Development Working Group as we have been deeply learning about CQL more than the original 'journal club' effort that started this group
• Statistic Standard and Terminology – change this to Statistic Terminology Working Group with a primary focus on terms and definitions
• Research Design - change this to a StatisticsOnFHIR Working Group with a primary focus on modeling statistic and analysis plan data in FHIR Resources and Profiles
• Knowledge Ecosystem Liaison – change this to a Funding the Ecosystem Infrastructure Working Group as we explore and develop grant proposals.
• Risk of Bias Terminology and Tooling – change this to a Risk of Bias Terminology Working Group with a primary focus on terms and definitions

On June 26, the Project Management Group changed the names (and scope) of the following Working Groups:

• ResearchOnFHIR – Mondays at 10-11 am Eastern – changed to CQL Development Working Group with a primary focus on learning to develop Clinical Quality Language (CQL)
• Statistic Standard and Terminology – Mondays at 2-3 pm Eastern – changed to Statistic Terminology Working Group with a primary focus on terms and definitions
• Research Design – Tuesdays at 2-3 pm Eastern - changed to StatisticsOnFHIR Working Group with a primary focus on modeling statistic and analysis plan data in FHIR Resources and Profiles
• Knowledge Ecosystem Liaison – Wednesdays at 8-9 am Eastern – changed to Funding the Ecosystem Infrastructure Working Group with a primary focus to explore and develop grant proposals
• Risk of Bias Terminology and Tooling – Fridays at 9-10 am Eastern – changed to Risk of Bias Terminology Working Group with a primary focus on terms and definitions

On June 29, the Project Management Group was notified that a Software Demo (for FEvIR®: CodeSystem Builder/Viewer and FEvIR®: My Ballot) was accepted for presentation at the International Conference on Biomedical Ontologies (ICBO) which is taking place at University Brasilia on August 30.  We learned that the software demo is to occur in person along with a poster session, and we are unable to travel for this conference at this time. If anyone is going to this conference and would like to present, please let us know.  We also sent those who signaled interest a calendar invite for the Ontology Management Working Group meetings to start on July 11 at 3-4 pm Eastern on Tuesdays.

Project Management 2023 Quarter 3

On July 6, the Project Management Group changed the Setting the SRDR Platform on FHIR Working Group to Setting the Scientific Record on FHIR Working Group to reflect a broader focus on multiple platforms and repositories (This Working Group will focus on converting or importing data to FHIR from multiple repositories (SRDR, MEDLINE, CEDAR, ClinicalTrials.gov, etc.) and facilitating searches across the dataset that represents the Computable Scientific Record.).

On July 10, the Project Management Group reviewed changes to FEvIR®: Project Builder/Viewer and FEvIR®: Classification Builder/Viewer in preparation for support of the Measuring the Rate of Scientific Knowledge Transfer Working Group, and adjusted the Classification Profile and Rating Profile in the Evidence Based Medicine Implementation Guide to allow use of workflowStatus and disposition elements.

On July 17, the Project Management Group drafted Profiles of EvidenceReport for the first 6 sections of the M11 Technical Specification – i.e. for the Interntaional Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) Clinical Electronic Structured Harmonised Protocol (CeSHarP) M11 Technical Specification – with:

On July 20, the Project Management Group submitted Lightning Talk abstracts for the MCBK Global meeting.

On July 24, the Project Management Group prepared for this week's meetings and discussed logistics for the Guidelines International Network meeting in September

On July 31, the Project Management Group reviewed FHIR change requests related to Composition Resource (Joanne Dehnbostel and Khalid Shahin are now serving as interim co-chairs for HL7 Structured Documents Work Group) and one of these included the use of Citation Resource for reference lists for Composition instances.

On August 7, the Project Management Group reviewed the DocumentBundle Profile of Bundle Resource and ClinicalDocument Profile of Composition Resource and considered using these patterns to support data exchange for "Project" Resource on the FEvIR Platform.

On August 14, the Project Management Group added the Recommendation Profile to the EBM Implementation Guide.

On August 21, the Project Management Group discussed preparations for upcoming presentations (ICBO, GIN, MCBK), added the Guideline Profile to the EBM Implementation Guide, and revised the RecommendationJustification Profile to include informationType = "container" for the codes for the top content sections.

On September 7, the Project Management Group prepared variations of the GRADE Ontology - DRAFT only for tomorrow's GRADE Ontology Working Group meeting, sent meeting cancellation notices for the next 2 weeks, and reminded us of the suggested agenda for the next 2 weeks:

On Sept 28, the Project Management Group prepared the suggested agenda for the next 8 days:

Suggested agenda:

Friday, September 29:

• 8 am Eastern: GRADE Ontology Working Group – start term development; attention to Certainty of evidence
• 9 am Eastern: Risk of Bias Terminology Working Group – review SEVCO terms; attention to Predictive Model Research Bias (4 terms open for vote)
• 10 am Eastern: Communications Working Group – presentations (MCBK for October 3), publications (Introduction to EBMonFHIR, Study design terminology)
• 12 pm Eastern: Eligibility Criteria Working Group (HL7 CDS EBMonFHIR sub-WG) – apply approved changes to Group Resource and RelativeTime Datatype; facilitate participants adapting to the newly approved changes to Group Resource

Monday, October 2:  

• 8 am Eastern: Project Management – review EBM Implementation Guide progress for January ballot
• 9 am Eastern: Setting the Scientific Record on FHIR Working Group – evaluate additional sources for conversion to FHIR (GRADEpro, then Cochrane on October 9)
• 10 am Eastern: CQL Development Working Group (HL7 CDS EBMonFHIR sub-WG) – evaluate how CQL fits with recently approved Group Resource model and FEvIR tooling
• 2 pm Eastern: Statistic Terminology Working Group – review SEVCO terms; attention to Probability Distribution Attributes terms (9 terms open for vote)

Tuesday, October 3:

• 9 am Eastern: Measuring the Rate of Scientific Knowledge Transfer Working Group – CANCELED (to allow attendance in MCBK Global meeting)
• 2 pm Eastern: StatisticsOnFHIR Working Group (HL7 CDS EBMonFHIR sub-WG) – continue ANCOVA model characteristic details in example EndpointAnalysisPlan Profile for PHUSE Lilly Redacted Protocol
• 3 pm Eastern: Ontology Management Working Group – review progress on tasks, attention to case-sensitive changes

Wednesday, October 4:

• 8 am Eastern: Funding the Knowledge Ecosystem Infrastructure Working Group – CANCELED (to allow attendance in MCBK Global meeting)

Thursday, October 5:

• 8 am Eastern: EBM Implementation Guide Working Group (HL7 CDS EBMonFHIR sub-WG) – review EBM Implementation Guide progress for January ballot
• 9 am Eastern: Computable EBM Tools Development Working Group – review FEvIR® Platform developments; Recommendation Adaptation or Summary of Net Effect related developments
• 4 pm Eastern: Project Management – prepare weekly agenda

Friday, October 6:

• 8 am Eastern: GRADE Ontology Working Group – continue term development
• 9 am Eastern: Risk of Bias Terminology Working Group – review SEVCO terms; attention to Predictive Model Research Bias
• 10 am Eastern: Communications Working Group – publications (Introduction to EBMonFHIR, Study design terminology), presentations
• 12 pm Eastern: Eligibility Criteria Working Group (HL7 CDS EBMonFHIR sub-WG) – facilitate participants adapting to the newly approved changes to Group Resource

Communications Working Group Updates:

• Introductory Pages where you can find information related to the Health Evidence Knowledge Accelerator (and EBMonFHIR and FEvIR Platform) include:
  • Health Evidence Knowledge Accelerator (HEvKA) Confluence page within the EBMonFHIR project Confluence page within the HL7 Confluence page.
  • Health Evidence Knowledge Accelerator (HEvKA) project page at https://fevir.net/resources/Project/29272 on the FEvIR Platform -- including 32 links to presentations (PPT and Video) that can serve as introductory materials.
• Projects describing sets of citations include:
  • COVID-19 Knowledge Accelerator mentions in publications
  • EBMonFHIR mentions in publications

Communications WG 2023 Quarter 1

On January 6, 2023, the Communications Working Group reviewed the HEvKA project page on FEvIR and the HEvKA Confluence pages with multiple changes to improve their usefulness.

On January 13, 2023, the Communications Working Group reviewed the feedback seeking major revisions for a JAMIA Open submission of 'Introduction to EBMonFHIR' and selected 6 conferences with open calls for presentations for additional consideration:

1. HL7 FHIR DevDays (DevDays 2023) https://www.devdays.com/devdays-2023/ June 6-9, 2023 in Amsterdam (or Hybrid online); Proposals due Feb 1
2. Sci-K (2023 ACM Web Conference) https://sci-k.github.io/2023/ April 30-May 4, 2023 in Austin, Texas; Proposals due Feb 6
3. The First International Workshop on Semantics in Dataspaces (SDS 2023) https://dbis.rwth-aachen.de/SDS23/ April 30-May 1, 2023 in Austin, Texas; Proposals due Feb 6
4. EBHC Conference 2023 https://www.ebhcconference.org/home.it-IT.html October 25-28, 2023 in Taormina, Sicily; Proposals due Feb 28
5. SciDataCon-International Data Week (SciDataCon-IDW Salzburg 2023) https://www.scidatacon.org/IDW-2023-Salzburg/submit/ October 23-26, 2023 in Salzburg (and hybrid); Proposals due Mar 31
6. Guidelines International Network (GIN 2023) https://g-i-n.net/conference_2023/welcome Sep 19-22, 2023 in Glasgow (or Hybrid online); Proposals due ??? (call for presentations opens Jan 17)

On January 18, 2023 an update of EBMonFHIR was given to the HL7 BRR meeting at the January Working Group Meeting

https://docs.google.com/presentation/d/1Ep0b9CM4LMC_7wgc9R6PE98xgFN0uTjC/edit#slide=id.p1

On January 20, the Communications Working Group reviewed changes to the Introduction to EBMonFHIR for resubmission for publication, and added "Review Intro to EBMonFHIR Walkthrough" to the agenda for 3 HEvKA meetings next week.

On January 27, the Communications Working Group reviewed the last set of changes to the Introduction to EBMonFHIR (and response to reviewers) for resubmission for publication.

On February 3, the Communications Working Group reminded authors who were present regarding 2 publications near ready for submission or resubmission (Intro to EBMonFHIR, Study Design Terminology) and noted the next presentation proposal deadline is February 13 for a Guidelines International Network pre-conference course. We are accelerating our developments of the FEvIR®: Adaptation Builder/Viewer to provide a compelling experience for updating or adapting guidelines and hope to have this developed far enough to submit a compelling pre-conference course (or coordinate with others producing related courses).

On February 10, the Communications Working Group made multiple revisions to prepare the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission to the Clinical Trials journal. 

On February 17, the Communications Working Group reviewed changes to FEvIR^®: CodeSystem Viewer support listing Authors/Editors, Reviewers and Endorsers to the SEVCO Release 1.1 (https://fevir.net/sevco) and the OSF posting of the SEVCO protocol (https://osf.io/5z84p/) in preparation for submitting the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" to the Clinical Trials journal. We also reviewed three open calls for presentation proposals that we will consider in the coming weeks:

1. EBHC Conference 2023 https://www.ebhcconference.org/home.it-IT.html October 25-28, 2023 in Taormina, Sicily; Proposals due Feb 28
2. AMIA 2023 Annual Symposium https://amia.org/education-events/amia-2023-annual-symposium November 11-15, 2023 in New Orleans; Proposals due Mar 8
3. Guidelines International Network (GIN 2023) https://g-i-n.net/conference_2023/welcome Sep 19-22, 2023 in Glasgow (or Hybrid online); Proposals due Apr 4

We also completed applying four changes to the FHIR specification:

• Adding an example to ResearchStudy https://build.fhir.org/researchstudy-example-ctgov-study-record.json.html (and to ResearchSubject https://build.fhir.org/researchsubject-example-crossover-placebo-to-drug.json.html) to resolve FHIR-39370 - Create examples illustrating use of ResearchStudy.compaarisonGroup 
• Adding EvidenceVariable.characteristic.instances[x] and EvidenceVariable.characteristic.duration[x] to resolve FHIR-37346- Add EvidenceVariable.characteristic.instances[x] 
• EvidenceVariable terminology changes to resolve FHIR-39021- Changes to EvidenceVariable resource
• Wrote explanatory text for Citation Resource to resolve FHIR-39093- Remove the Citation resource 

On February 24, the Communications Working Group reviewed open calls for proposals and discussed developments for Cochrane, AMIA, and MCBK submissions before the early March due dates.

On March 3, the Communications Working Group submitted 3 FAIRGround Demo proposals for the MCBK meeting in May, and drafted 2 proposals (1 Panel, 1 Systems Demo) for the AMIA Annual Meeting in November.

On March 10, the Communications Working Group updated the Clinical Reasoning Module - Evidence and Statistics page in the FHIR specification for R5 to cover the full set of Resource developments and link to the EBMonFHIR project page.

On March 17, the Communications Working Group updated the HEvKA 2022 Achievements and 2023 Plans with Notes as a high-level summary to be included in the HL7 Annual Review, and submitted a Panel Presentation and Systems Demo for the American Medical Informatics Association (AMIA) Annual Meeting.

On March 24, the Communications Working Group revised the SEVCO Study Design Terminology paper to be submitted for publication by Monday. We would also like to share an announcement for the upcoming GIN North America event: Bringing Guidelines to the Digital Age:  a one-day hands-on Human-Centered Design Workshop on April 21, 2023 near Chicago.

On March 31, the Communications Working Group submitted a panel presentation proposal for the Guidelines International Network (GIN) 2023 Conference titled “Making Science Computable: Guideline-relevant Developments from the Health Evidence Knowledge Accelerator”.

Communications WG 2023 Quarter 2

On April 7, the Communications Working Group submitted an eLetter response to Why ChatGPT Should Not Be Used to Write Academic Scientific Manuscripts for Publication – published at https://www.annfammed.org/content/early/2023/03/29/afm.2982/tab-e-letters#re-creating-the-language-to-speak-with-chatgpt-for-science

On April 14, the Communications Working Group discussed the following article which discusses the use of Natural Language Processing to extract information from journal articles. HEvKA members attended a JAMIA journal club which covered the article on April 13 and are considering formulating a formal response to the article. 

Tian Kang, Yingcheng Sun, Jae Hyun Kim, Casey Ta, Adler Perotte, Kayla Schiffer, Mutong Wu, Yang Zhao, Nour Moustafa-Fahmy, Yifan Peng, Chunhua Weng, EvidenceMap: a three-level knowledge representation for medical evidence computation and comprehension, Journal of the American Medical Informatics Association, 2023;, ocad036, https://doi-org.ezproxy3.library.arizona.edu/10.1093/jamia/ocad036 

The group also discussed preparing for the 2024 Amstat meeting (https://www.amstat.org/meetings/asa-biopharmaceutical-section-regulatory-industry-statistics-workshop) and the possibility of expanding the Scientific Evidence Code System (SEVCO) to meet new requirements of specifying estimands, intercurrent events in trials, and RWE studies  https://www.ich.org/news/ich-e9r1-training-material-now-available-ich-website

On April 21, the Communications Working Group discussed the presentation on 'Reporting Study Design with the Scientific Evidence Code System (SEVCO): A novel community-reviewed standard vocabulary' for the Society for Clinical Trials (SCT) meeting in May, and considered an opening example of confusion representing equivalence trials, superiority trials, and non-inferiority trials as an example of why a standard vocabular is needed.

On April 28, the Communications Working Group developed the opening concepts for the presentation on 'Reporting Study Design with the Scientific Evidence Code System (SEVCO): A novel community-reviewed standard vocabulary' for the Society for Clinical Trials (SCT) meeting on May 23. We decided to cancel the May 5 Communications Working Group meeting due to unavailability of multiple participants and will resume on May 12.

On May 12, the Communications Working Group discussed proposing a systems demo of 'FEvIR Platform for Ontology Development' for the 14th International Conference on Biomedical Ontology (ICBO 2023) and drafted "Highlights" for submitting the Study Design Terminology article to Journal of Clinical Epidemiology:

Highlights

• Scientific Evidence Code System (SEVCO) provides a standard vocabulary for research
• 75 SEVCO terms (with hierarchical relationships) represent study design
• Terms include definitions, alternative terms, and comments for application
• This supports precise, unambiguous, machine-interpretable reporting of research
• SEVCO is developed by an open, global, multidisciplinary, expert working group

On May 19, the Communications Working Group discussed necessary changes to the current draft of  the Study Design Terminology article in order to submit to Journal of Clinical Epidemiology. Potential inclusion of a graphic image and context for clinical epidemiologist interest (Why do they care?) were discussed. We also explored the tool on the Journal of Clinical Epidemiology website for creation of a competing interest statement. There was also discussion of upcoming presentations, one at the Society for Clinical Trials Meeting next Tuesday by Brian Alper in Baltimore about our SEVCO ontology and another at the GRADE working group meeting next Tuesday by Paul Whaley in Split, Croatia about creating a GRADE ontology. 

On May 23, Brian S. Alper, MD, MSPH presented 'Reporting Study Design with the Scientific Evidence Code System (SEVCO): A novel community-reviewed standard vocabulary' to about 75 people in the Society for Clinical Trials (SCT) Annual Meeting.

On May 26, the Communications Working Group discussed proposing a systems demo of 'FEvIR Platform for Ontology Development' for the 14th International Conference on Biomedical Ontology (ICBO 2023) and sent a request to conference coordinators as the submission form was not working.

On June 2, the Communications Working Group revised the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for a more powerful opening before submission to Journal of Clinical Epidemiology.

On June 9, the Communications Working Group discussed author comments and added three figures to the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)".

On June 16, the Communications Working Group made further revisions to the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" and discussed have a Second SEVCO Town Hall meeting some time in July to include (1) summary update across SEVCO developments, (2) broad-based discussion about standardizing the approach to use of capitalization across the preferred terms, and (3) open question/discussion period.

On June 23, the Communications Working Group reviewed and confirmed the suggested changes to the series of HEvKA Working Group meetings as recommended by the Project Management Group. In addition to the term-specific developments (across our current working group meetings covering statistics and risk of bias), we are planning to start a new weekly HEvKA Ontology Management Working Group meeting to manage the SEVCO efforts and systems for managing ontologies.  If you would like to join this working group, please suggest your available times at https://whenisgood.net/Ontology.

On June 30, the Communications Working Group decided to set Friday, July 21 from 9-11 am Eastern for the Second Annual SEVCO Town Hall Meeting with agenda to include progress update on SEVCO development, related publications and presentations, decision-making regarding presentation styles (i.e. capitalization) for SEVCO terms, and open question-and-answer period.

Communications WG 2023 Quarter 3

On July 7, 2023, the Communications Working Group reviewed the third round of reviewer feedback on an article "Making Science Computable: Introduction to Evidence-Based Medicine on Fast Healthcare Interoperability Resources (EBMonFHIR)" and discussed various options including separating a general informatics version from a clinical informatics version as different introductory articles for different audiences. We also started drafting Lightning Talk Abstracts for the Mobilizing Computable Biomedical Knowledge (MCBK) Global meeting in October 2023.

On July 14, 2023, the Communications Working Group suggested the Journal of Biomedical Informatics as a primary target for submitting "Making Science Computable: Introduction to Evidence-Based Medicine on Fast Healthcare Interoperability Resources (EBMonFHIR)" and 3 draft proposals for MCBK Lightning Talks were shared for feedback.

On July 28, 2023, the Communications Working Group overcame some technical challenges with the HEvKA Summary Update Confluence page. We also note that we will be presenting two sessions at the Guidelines International Network (GIN) meeting in Glasgow, Scotland:

1) pre-conference course 'Using technology to efficiently update and adapt guidelines' on September 19 from 8:30 am to 5 pm.

2) panel presentation 'Making Science Computable: Guideline-relevant Developments from the Health Evidence Knowledge Accelerator' on September 20 from 2:30 pm to 4 pm.

On August 4, 2023, the Communications Working Group started drafting the panel presentation 'Making Science Computable: Guideline-relevant Developments from the Health Evidence Knowledge Accelerator' on September 20 from 2:30 pm to 4 pm at the Guidelines International Network (GIN) meeting in Glasgow, Scotland.

On August 11, 2023, the Communications Working Group started the effort to prepare an "Introduction to EBMonFHIR" paper for submission to the Journal of Biomedical Informatics, and reviewed the plans for the panel presentation 'Making Science Computable: Guideline-relevant Developments from the Health Evidence Knowledge Accelerator' on September 20 from 2:30 pm to 4 pm at the Guidelines International Network (GIN) meeting in Glasgow, Scotland, including proposed changes to the user experience for creating a new recommendation summary on the FEvIR Platform.

On August 18, 2023, the Communications Working Group made initial revisions to an "Introduction to EBMonFHIR" paper for submission to the Journal of Biomedical Informatics, and reviewed the development of Computable Publishing®: Recommendation Authoring Tool in preparation for the panel presentation 'Making Science Computable: Guideline-relevant Developments from the Health Evidence Knowledge Accelerator' on September 20 from 2:30 pm to 4 pm at the Guidelines International Network (GIN) meeting in Glasgow, Scotland.

On August 25, 2023, the Communications Working Group made revisions to an "Introduction to EBMonFHIR" paper for submission to the Journal of Biomedical Informatics, and reviewed the development of Computable Publishing®: Guideline Authoring Tool and Computable Publishing®: Recommendation Authoring Tool in preparation for the panel presentation 'Making Science Computable: Guideline-relevant Developments from the Health Evidence Knowledge Accelerator' on September 20 from 2:30 pm to 4 pm at the Guidelines International Network (GIN) meeting in Glasgow, Scotland.

On September 1, 2023, the Communications Working Group made revisions to the exemplar Evidence Resource (Critically appraised summary of primary outcome of multi-platform RCT of anticoagulation for non-critically ill patients with COVID-19) in coordination with its detailed explanation in an "Introduction to EBMonFHIR" paper being prepared for submission to the Journal of Biomedical Informatics.

On September 8, 2023 the Communications Working Group made revisions and discussed the "Study Design Terminology in the Scientific Evidence Code System (SEVCO): A Consensus Derived Standard " paper being prepared for submission. The group also discussed the upcoming presentations and discussions that are taking place this month at the September 2023 HL7 Connectathon in Phoenix, Arizona and the upcoming Guidelines International Network Conference and GRADE Meeting in Glasgow, Scotland.

On September 19, we presented an all-day pre-conference course titled “Using Technology to Efficiently Update and Adapt Guidelines”. A big thank you to Dr. Ilkka Kunnamo who contributed his considerable expertise to the success of the class. We covered many tools on the FEvIR platform and concentrated on our newest software that is now available to create, update and adapt clinical guidelines. We had a good turnout and lively discussion with 18 students from 12 countries (Belgium, Canada, Finland, Germany, Japan, Netherlands, Norway, Philippines, Singapore, South Korea, United Kingdom, United States). Many of the participants asked great questions and made suggestions which will guide our future software development.

On September 20, we presented a panel presentation titled “Making Science Computable: Guideline-relevant Developments from the Health Evidence Knowledge Accelerator”. Speakers on the panel included Khalid Shahin who introduced making science computable, EBMonFHIR and the FEvIR platform; Joanne Dehnbostel who introduced the Health Evidence Knowledge Accelerator and the Scientific Evidence Code System; Dr. Karen Robinson who introduced the Guideline Authoring Tool and Recommendation Authoring Tool on the FEvIR platform; and Dr. Ilkka Kunnamo who introduced the Adaptation and Summary of Findings tools on the FEvIR platform.

Scientific Knowledge Accelerator Foundation Updates:

On January 31, the Scientific Knowledge Accelerator Foundation Board of Directors discussed the development of metrics to measure the pace of scientific knowledge transfer. We decided to target two concepts for initial metric development:

1. For systematic reviews, time from publication (date) to citation in 2 or more independently published guidelines, clinical reference, clinical education, or clinical decision support artifacts
2. For clinical trials, time from publication (date) to citation in 2 or more independently published systematic reviews, guidelines, clinical reference, clinical education, or clinical decision support artifacts

We decided to start the Measuring Rate of Scientific Knowledge Transfer Working Group as a weekly HEvKA meeting on Tuesdays at 9 am Eastern. The last Tuesday of each month will be used for the Board of Directors meeting.

On February 28, the Scientific Knowledge Accelerator Foundation Board of Directors reviewed the progress of the Measuring Rate of Scientific Knowledge Transfer Working Group and discussed three upcoming collaborative grant opportunities (let us know if you would like to participate in any of them):

1. National Science Foundation (NSF) Pathways to Enable Open-Source Ecosystems (POSE)
2. NIH Accelerating Data and Metadata Standards in the Environmental Health Sciences (R24 Clinical Trial Not Allowed)
3. NIH Notice of Intent to Publish a Funding Opportunity Announcement for Accelerating Behavioral and Social Science through Ontology Development and Use (U01) – for this, we considered, once the Study Design Terminology in SEVCO paper is submitted for publication, to focus development of additional Study Design terms for behavioral and social science needs.

On March 28, the Scientific Knowledge Accelerator Foundation Board of Directors reviewed grant opportunities (NIH U01 Accelerating Behavioral and Social Science through Ontology Development and Use, NIEHS Accelerating Data and Metadata Standards, NSF Pathways to Enable Open-Source Ecosystems (POSE), NLM Research Grants in Biomedical Informatics and Data Science). For the NLM opportunity, we drafted first-draft Specific Aims for a 4-year project:

• Year 1) Translate original research results (effect estimates/statistical findings and variable definitions), from studies used in pre-existing meta-analyses (at snapshot time 1), into FHIR. Validate reliable translation/transformation of original research results into computable evidence.
• Year 2) Translate search strategies, from pre-existing meta-analyses, into FHIR. Validate precision and recall of computable searches.
• Year 3) Translate statistical analysis plans for pre-existing meta-analyses into FHIR. Validate that automated processing of computable evidence in FHIR reliably reproduces the meta-analysis results.
• Year 4) Update the meta-analyses with automated support using data available at snapshot time 2. Validate reliable reproduction of the updated (cumulative) meta-analyses.

On April 25, the Scientific Knowledge Accelerator Foundation Board of Directors discussed strategies for revenue generation, including voluntary donation pathways for employees of large corporations (preparing for the fall campaign season) and grant opportunities. We discussed the potential compelling vision for accelerating scientific knowledge transfer (making science computable) to be able, as an individual facing a medical decision, to instantly know all the reasonable options and the expected outcomes (benefits and harms) of each option. For the NIEHS Accelerating Data and Metadata Standards grant opportunity, we approved providing a Letter of Support for a current proposal in progress. For the NLM Research Grants in Biomedical Informatics and Data Science grant opportunity, we drafted a Specific Aims page and are starting to work on the Research Strategy.

On May 30, the Scientific Knowledge Accelerator Foundation Board of Directors discussed strategies for revenue generation, including voluntary donation pathways for employees of large corporations (preparing for the fall campaign season) and grant opportunities. For the NLM Research Grants in Biomedical Informatics and Data Science grant opportunity, we reviewed a draft proposal that is due on Monday. We also discussed a recently posted funding opportunity for Accelerating Behavioral and Social Science through Ontology Development and Use: Research Network Projects (due September 3) and consider this a great fit for us to extend the Scientific Knowledge Evidence Code System (SEVCO) to support behavioral and/or social science research needs. Critical to this opportunity is to "Identify one or more use cases and articulate a strong justification for the proposed ontological approach, resource, or tool with an emphasis on tool and resources that are “fit for use.” The justification must include information about how the proposed semantic knowledge resources will more efficiently and effectively solve a problem/s or accelerate cumulative and integrative behavioral or social science research (BSSR)." We welcome BSSR subject matter experts to join us in developing a compelling proposal.

On June 27, the Scientific Knowledge Accelerator Foundation Board of Directors discussed multiple items, including:

• update on the progress of the Measuring the Rate of Scientific Knowledge Transfer Working Group which includes
  • completion of a walkthrough of a proposed process for recording the time from publication of clinical trial results to their use in 2 or more systematically derived reports intended to guide clinical practice

  • development of multiple improvements to the FEvIR Platform to support overall project efforts, including changes to Project Builder, Classification Builder, and MEDLINE-to-FEvIR Converter that are specifically useful for this project

  • inclusion of an intern on the project to specifically help with documentation of "Developing a reproducible process to record the observations with this metric"

• HEvKA Ontology Management Working Group to start soon

• HEvKA Funding the Ecosystem Infrastructure Working Group to start tomorrow

• GIN Developments, including acceptance of a panel presentation for the September 2023 meeting, and announcement of July 28 GIN North America Coffee Hour Chat with the theme "Which platforms and tools do others use for systematic review and guideline development?”

• review of technology-based collaborative developments, including

  • ClinicalTrials.gov-to-FHIR conversion available at https://fevir.net/ctgovconvert and soon to be available directly from ClinicalTrials.gov

  • Systematic Review Data Repository (SRDR+) data converted to FHIR and can be loaded to FEvIR Platform at https://fevir.net/srdr

  • AHRQ Center for Evidence-based Practice Improvement (CEPI) Evidence Discovery and Retrieval (CEDAR) which has FHIR Citation Resources for entries across 5 AHRQ databases (Effective Healthcare Program (EHC), Evidence-Based Practice Center (EPC), United States Preventive Services Task Force (USPSTF), SRDR, CDS Connect) – a search query can now be entered at https://fevir.net/cedar to retrieve results from CEDAR (and MEDLINE)

• Grant opportunities (across AHRQ, NIH, NSF and ARPA-H) to consider selecting one for development with a September due date

On July 25, the Scientific Knowledge Accelerator Foundation Board of Directors reviewed progress across the HEvKA Working Groups and repeated the announcement of July 28 GIN North America Coffee Hour Chat with the theme "Which platforms and tools do others use for systematic review and guideline development?”

On August 29, the Scientific Knowledge Accelerator Foundation Board of Directors reviewed progress across selected HEvKA Working Groups (Funding the Knowledge Ecosystem, Measuring the Rate of Scientific Knowledge Transfer, Eligibility Criteria) and reviewed the adaptation features to be released today for Computable Publishing®: Recommendation Authoring Tool and demonstrated in the Guidelines International Network (GIN) 2023 conference this September.

On September 26, the Scientific Knowledge Accelerator Foundation Board of Directors met and discussed progress made at multiple conferences over the last two weeks including Cochrane, HL7, GIN and GRADE.

The EBMonFHIR Implementation guide will be proposed for January ballot with HL7, which means we must meet a November 12 deadline to prepare all EBMonFHIR resources to declare our "intent to ballot".

We also need to put together a presentation to be given at the MCBK Meeting on October 3-4, 2023.

Future conferences and opportunities were then discussed including the upcoming Global Evidence Summit which will take place in Prague in September 2024 https://www.globalevidencesummit.org/ and the White house open science challenge https://www.challenge.gov/?challenge=ostp-year-of-open-science-recognition-challenge. We also received a request to present our research at the 10th International Congress meeting (occuring every 4 years) which will take place in September, 2025 in Chicago. The likely subject of the research we will present is our "measuring the rate of scientific knowledge transfer" project.

Scientific Evidence Code System (SEVCO) Updates

• Help Shape the Scientific Evidence Code System (SEVCO) – an open effort to define terms for the expression of study design, statistics, and risk of bias used across the communication of science. We are following a Scientific Evidence Code System Development Protocol.
• The SEVCO Progress Update and All-Group Discussion included a presentation by Joanne Dehnbostel and open discussion with Brian Alper on April 14, 2022.  The PowerPoint can be viewed here and the meeting recording can be viewed here.
• To join the Scientific Evidence Code System Expert Working Group to vote on the terms and definitions in this code system, just go to the Scientific Evidence Code System (SEVCO) Project Page and click the Join the Group button. There are currently 39 people in the Scientific Evidence Code System Expert Working Group.
• The code system (as it is developed) can be viewed at Scientific Evidence Code System (SEVCO) -- DRAFT ONLY (Not published for current use), and anyone is welcome to comment on any term by finding the term in the Term Detail view and clicking the Comment button
• The first version ready for release includes 75 Study Design terms and 124 Bias terms and can be viewed at https://fevir.net/sevco.
• Members of the Expert Working Group can vote on any term that is open for voting in the Term Detail view and clicking the Vote button, or by viewing all terms open for vote at FEvIR®: My Ballot.
• Current progress on term development includes:
• 76 of 76 (100%) Study Design terms approved
• 172 of 238 (72%) Bias terms approved
• 0 of 26 Rating of Bias terms approved
• 93 of 138 (67%) Statistic terms approved
• 4 of 120 Statistical Model terms approved
• 345 of 598 (58%) TOTAL terms approved

On July 21, the Second Annual SEVCO Town Hall Meeting was held with 19 participants. We reviewed the history of EBMonFHIR, HEvKA, and SEVCO and noted current progress with 57% completion across 595 terms and 100% completion of the study design terms. An active discussion touched on many areas of interest to the participants. We also discussed 4 different approaches to capitalization style within the structured data and decided unanimously to use lowercase except for proper words in the structured data, and add capitalization variants as synonyms/alternative terms. The meeting recording can be found here.

Research Design Working Group Updates:

This group submitted an article for the peer-reviewed literature to introduce the 75 Study Design terms completed for version 1 of SEVCO. This group reached 100% agreement for these 75 terms with 42 unique people from 18 countries (an average of 12 unique people contributing to an individual term).

In 2023 Quarter 2 the group started working on developing the model for an 'Endpoint Analysis Plan' for a single outcome measure, to be represented as an EndpointAnalysisPlan Profile in the Evidence-Based Medicine FHIR Implementation Guide. In 2023 Quarter 3 the Research Design WG was ended and a StatisticsOnFHIR WG was started.

Research Design WG 2023 Quarter 1

On January 24, 2023, the Research Design Working Group continued drafting an article titled “Study Design Terminology in the Scientific Evidence Code System (SEVCO)” for submission to the Clinical Trials journal. 

On January 31, 2023, the Research Design Working Group reviewed the submission guidelines for the Clinical Trials journal to prepare the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission. 

On February 7, the Research Design Working Group modified the primary Results table to prepare the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission to the Clinical Trials journal. 

On February 14, the Research Design Working Group worked to finalize the manuscript describing the creation of the study design terminology, which is the first part of the Scientific Evidence Code System (SEVCO). The group discussed the acknowledgements section of the paper and will be sending out emails to more than 50 individuals to be acknowledged in the paper.

We are very thankful to all of you that volunteered your time and helped us to build this important code system. The emails should go out in the next couple of days. Please let us know if you have been involved in this process and do not receive an email. We don’t want to omit anyone as you were all integral to this successful endeavor.   

On February 21, the Research Design Working Group continue to revise the text for the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission to the Clinical Trials journal. 

On February 28, the Research Design Working Group made adjustments to improve the Computable Publishing®: ClinicalTrials.gov-to-FEvIR Converter so it can be cited as a key example of real-world application for the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission to the Clinical Trials journal. We also were pleased to learn that our conference proposal to present on this subject was accepted for the Society for Clinical Trials (SCT) Annual Meeting in May in Baltimore.

On March 7, the Research Design Working Group adjusted the strengths section with more comments about the FAIR Guiding Principles for the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission to the Clinical Trials journal. 

On March 14, the Research Design Working Group adjusted the discussion section with "Opportunities for research and development" including coordinating with current projects (such as NIH initiatives) and technology-related areas of semantic search, crowdsourcing, machine learning, and natural language processing. We are hoping to submit the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" for submission to the Clinical Trials journal by the end of the week. 

On March 21, the Research Design Working Group prepared the final changes to the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" to prepare for submission to the Clinical Trials journal. 

On March 28, the Research Design Working Group SUBMITTED the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" to the Clinical Trials journal, and started drafting the outline for a presentation on “Reporting Study Design with the Scientific Evidence Code System (SEVCO): A novel community-reviewed standard vocabulary” to be presented May 23 in the Society for Clinical Trials (SCT) annual conference. 

Research Design WG 2023 Quarter 2

On April 4, the Research Design Working Group completed a draft outline for a presentation on “Reporting Study Design with the Scientific Evidence Code System (SEVCO): A novel community-reviewed standard vocabulary” to be presented May 23 in the Society for Clinical Trials (SCT) annual conference.

On April 11, the Research Design Working Group discussed next steps (Do we develop additional study design terms? Do we develop a data exchange model for research design, e.g. StudyProtocol Profile of ResearchStudy Resource? Do we seek Notice Of Funding Opportunity (NOFO) calls for related issues?) We reviewed a few NOFO calls and asked program officers about potential fit with SEVCO developments.

On April 18, the Research Design Working Group discussed next steps and suggested we start preparing for a long-term goal of creating a 'Research Protocol Implementation Guide' to represent a full Clinical Study Protocol (CSP) and Statistical Analysis Plan (SAP) in FHIR form, and a short-term goal of defining a Profile to represent an 'Endpoint Analysis Plan' for a single outcome measure.

On April 25, the Research Design Working Group started the effort to define a Profile to represent an 'Endpoint Analysis Plan' for a single outcome measure. We selected a relatively simple (dichotomous) outcome measure defined in a Statistical Analysis Plan (SAP) at https://clinicaltrials.gov/ProvidedDocs/94/NCT04542694/Prot_SAP_000.pdf – We created an Evidence Resource (Endpoint Analysis Plan - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694) to map it out, and a text description of the 'Endpoint Analysis Plan' for the statistic is:

This primary efficacy parameter is the percentage of patients who achieved an improvement in clinical status according to the categorical ordinal scale of clinical improvement by 2 or more categories at Visit 3.
For the analysis of this parameter, an intergroup comparison of percentages is used using

• a two-sided version of the Fischer's exact test
• (or a two-sided version of the χ2 (chi-square) test, if all the expected values in the cells of the contingency table for this analysis are 5 or more).

The percentage of patients who achieved improvement in clinical status on a categorical ordinal clinical improvement scale by 2 or more categories at Visit 3 is presented with a two-sided 95% confidence interval by treatment arm.
Hypothesis testing is performed at a 5% significance level.
The difference in proportion between treatment arms and the 95% bilateral confidence interval for the difference in proportion calculated by Newcomb-Wilson technique is presented.
The proof of the hypothesis of superiority of Areplivir film-coated tablets (PROMOMED RUS LLC, Russia) is the establishment of a statistically significant difference in the proportions of patients who achieved an improvement in clinical status according to the categorical ordinal scale of clinical improvement by 2 or more categories at Visit 3, between Areplivir arm and standard therapy arm.
Analysis of the primary efficacy parameter is performed in the ITT population (main analysis), and in the PP population (additional analysis).

The initial concept that struck the Working Group is that the specific analysis plan (using a Fischer's exact test or using a chi-square test) is conditional based on a specific condition (if all the expected values in the cells of the contingency table for this analysis are 5 or more).  We talked through multiple methods of expressing this conditional logic and our initial suggestion is to add an element within Evidence.statistic of analysisPlan 0..* BackboneElement which can include analysisPlan.condition and analysisPlan.hypothesisTest — next week we will continue to convert the above text to representation within analysisPlan element instances.

On May 2, the Research Design Working Group further developed the model for an 'Endpoint Analysis Plan' for a single outcome measure, documented in a developing Evidence Resource (Endpoint Analysis Plan - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694). To further summarize an analysisPlan within the statistic we considered a model like:

• statistic 0..* BackboneElement
  • analysisPlan 0..* BackboneElement
    • condition 0..1 CodeableConcept | Expression
    • hypothesisTest 0..1 CodeableConcept
    • inferenceMetric 0..1 CodeableConcept
    • inferenceThreshold 0..1 Quantity --- supports > or < comparator in addition to the decimal value and in rare cases a unit
    • InferenceInterval 0..1 Range

For this example, we also represented the statistic with statisticType="Risk Difference" and attributeEstimate.type="95% Confidence interval" but to support the phrase regarding how the confidence interval was calculated we considered attributeEstimate.attributeEstimate.type so the JSON for this portion looks like:

      "statisticType": {
        "coding": [
          {
            "system": "https://fevir.net/resources/CodeSystem/27270",
            "code": "STATO:0000424",
            "display": "Risk Difference"
          }
        ]
      },
      "attributeEstimate": [
        {
          "type": {
            "coding": [
              {
                "system": "https://fevir.net/resources/CodeSystem/27270",
                "code": "TBD:0000059",
                "display": "Confidence interval"
              }
            ]
          },
          "level": 0.95,
          "attributeEstimate": [
            {
              "description": "calculated by the Newcomb-Wilson technique",
              "type": {
                "text": "calculated by the Newcomb-Wilson technique"
              }
            }
          ]
        }
      ]

On May 16, the Research Design Working Group further developed the model for an 'Endpoint Analysis Plan' for a single outcome measure, documented in a developing Evidence Resource (Endpoint Analysis Plan - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694). To further summarize an analysisPlan within the statistic we considered a model like:

• statistic 0..* BackboneElement
  • analysisPlan 0..* BackboneElement
    • condition 0..1 CodeableConcept | Expression
    • hypothesisTest 0..1 CodeableConcept
    • hypothesisTestSidedness 0..1 CodeableConcept
    • hypothesisTestNull[x] 0..1 Quantity | Range
    • inferenceMetric 0..1 CodeableConcept
    • inferenceThreshold[x] 0..1 Quantity | Range
    • primaryAnalysis 0..1 BackboneElement
      • name 0..1 string
      • modelCharacteristic 0..* see modelCharacteristic
    • alternativeAnalysis 0..* BackboneElement
      • name 0..1 string
      • modelCharacteristic 0..* see modelCharacteristic

We decided that, although the statistic.modelCharacteristic element may include concepts like intention-to-treat analysis, repeating the concepts within analysisPlan is worthwhile to distinguish fully between the planned analyses and the reported analyses.  Next we will consider the naming of alternativeAnalysis (vs. secondaryAnalysis) and see if we can fully represent the example in this structure.

On May 18, the EBM Implementation Guide Working Group and Computable EBM Tools Development Working Group further developed the model for an 'Endpoint Analysis Plan' for a single outcome measure, documented in a developing Evidence Resource (Endpoint Analysis Plan - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694). The analysisPlan element is now viewable as an example in the JSON or JSON Outline. To further summarize an analysisPlan within the statistic we are now representing the proposed model (changes in red) as:

• statistic 0..* BackboneElement
  • analysisPlan 0..* BackboneElement
    • condition[x] 0..1 CodeableConcept | Expression
    • hypothesisTest 0..1 CodeableConcept
    • hypothesisTestStatistic 0..1 CodeableConcept
    • hypothesisTestSidedness 0..1 CodeableConcept
    • hypothesisTestNull[x] 0..1 Quantity | Range
    • inferenceMetric 0..1 CodeableConcept
    • inferenceThreshold[x] 0..1 Quantity | Range
    • primaryAnalysis 0..1 BackboneElement
      • name 0..1 string
      • modelCharacteristic 0..* see modelCharacteristic
    • alternativeAnalysis 0..* BackboneElement
      • name 0..1 string
      • modelCharacteristic 0..* see modelCharacteristic

On May 23, the Research Design Working Group simplified the proposed model to adapt Evidence Resource with 13 new elements to support an EndpointAnalysisPlan Profile with only 4 new elements represented as:

• statistic 0..* BackboneElement
  • modelCharacteristic 0..* BackboneElement
    • condition[x] 0..1 CodeableConcept | Expression
    • code 1..1 CodeableConcept
    • value[x] 0..1 Quantity | Range | CodeableConcept
    • intended 0..1 boolean
    • applied 0..1 boolean
    • variable 0..* BackboneElement
    • attributeEstimate 0..* BackboneElement (see attributeEstimate)
    • modelCharacteristic 0..* see modelCharacteristic

We then modified the previous example at Endpoint Analysis Plan - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694 to show the same concepts in the simpler model at Endpoint Analysis Plan in modelCharacteristic - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694

On May 30, the Research Design Working Group reviewed the simplified proposed model for changing Evidence Resource to support an EndpointAnalysisPlan Profile, and modified the Endpoint Analysis Plan in modelCharacteristic - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694 example to include intended=True elements. We then started developments in FEvIR®: Evidence Builder/Viewer to support this model, finding it easier to develop examples with the tooling rather than continue to make changes in the JSON.

On June 6, the Research Design Working Group submitted the following FHIR change request:

FHIR-41379 - Getting issue details... STATUS

and the following UTG (terminology) change request:

UP-427 - Getting issue details... STATUS

and, upon recognizing a misspelling of 'Fisher's exact test' in our example, changed the modelCharacteristic.code.text to modelCharacteristic.code.coding in Endpoint Analysis Plan in modelCharacteristic - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694.

On June 13, the Research Design Working Group reviewed author comments and figures for the manuscript "Study Design Terminology in the Scientific Evidence Code System (SEVCO)" to prepare for submission to the Journal of Clinical Epidemiology.

The group also discussed adding a capitalization rule to our protocol which would be retroactive and make the capitalization of our SEVCO terms consistent.  

On June 20, the Research Design Working Group reviewed the work-to-date on developing an EndpointAnalysisPlan Profile and an example in Endpoint Analysis Plan in modelCharacteristic - Rate of clinical status improvement in favipiravir for COVID RCT NCT04542694.

• We then started reviewing the CodeableConcept data in the modelCharacteristic element in the example in need of coding values to suggest adjustments to the Scientific Evidence Code System (SEVCO).
• The first such concept was represented with Evidence.statistic.modelCharacteristic.conditionCodeableConcept.text = "if all the expected values in the cells of the contingency table for this analysis are 5 or more"
• Specific terms and definitions will be developed further for SEVCO, but we added 3 terms (bold) and adjusted part of the SEVCO hierarchy to include within Statistical Model Characteristic:
  • Statistical model assumption
    • Data distribution assumption of normal distribution
    • Data distribution assumption of equal standard deviations
    • Data distribution assumption of asymptotic approximation
  • Statistical model assumption assessment
    • all the expected counts in the cells of the contingency table meet or exceed a threshold

On June 26, the Research Design Working Group was changed to the StatisticsOnFHIR Working Group to reflect the change in focus.

On June 27, the StatisticsOnFHIR Working Group reviewed the results of time availability for the Ontology Management Working Group (and will start that group on Tuesdays at 3-4 pm Eastern starting July 11) and identified a second example to use for creating an EndpointAnalysisPlan instance. We set up an example instance (EndpointAnalysisPlan example for combined ADAS-Cog change and CIBIC+ change) and learned this example combines 2 endpoints into a statistical test across both, so it will likely require 3 coordinated resource instances for a full representation.

Statistic Terminology Working Group Updates:

For the first half of 2023, this group was modeling the expression of statistics in the FHIR standard and setting the shape for the Evidence Resource (and Profiles of Evidence in the Evidence Based Medicine Implementation Guide), as well as defining statistics terms for SEVCO, hence the group name covering ‘Standard and Terminology’.  In the second half of 2023, this shifted to 2 different groups (StatisticsOnFHIR and Statistic Terminology). Current efforts are related to defining probability distribution attributes and there are 9 terms open for vote:

To participate you can join the Scientific Evidence Code System (SEVCO) Expert Working Group at https://fevir.net/resources/Project/27845.

Statistic Standard and Terminology WG 2023 Quarter 1

On January 11, 2023, the Statistic Standard and Terminology Working Group discussed Regression Coefficient and, before drafting a definition, needed to review the information model (within Evidence Resource) for how the terms will be used.  We mapped out a pattern using Evidence.statistic.modelCharacteristic.code for the statistical model term (e.g. Logistic Regression) and Evidence.statistic.modelCharacteristic.attributeEstimate.type for the statistic type term (e.g. Regression Coefficient). We started the development of a StatisticalModel Profile in the Evidence Based Medicine Implementation Guide – demonstrating why this working group is named “Standard and Terminology”.

On January 18, 2023, the Statistic Standard and Terminology Working Group drafted 1 additional term (Regression Coefficient). Next week we will define terms that are Measures of Dispersion to complete the “simple” set of statistic types for initial use. We will then consider changing our overall approach for the Statistic Standard and Terminology Working Group to start developing the implementation guidance for communicating statistics in computable form, and in doing so draft additional terms for statistic types and statistical model characteristics as we use them with implementation guidance.

On January 25, the Statistic Standard and Terminology Working Group approved 3 terms (Phi coefficient, Kendall Correlation Coefficient, Goodman and Kruskal’s Gamma) and drafted 1 additional term (Measure of Dispersion). 

Next week we will define terms that are Measures of Dispersion to complete the “simple” set of statistic types for initial use. We will then change our overall approach for the Statistic Standard and Terminology Working Group to start developing the implementation guidance for communicating statistics in computable form, and in doing so draft additional terms for statistic types and statistical model characteristics as we use them with implementation guidance. We will change the weekly meeting time for this group to Tuesdays 1 pm Eastern starting with the next meeting on January 31.

On January 31, the Statistic Standard and Terminology Working Group discussed multiple nuances involved in the Evidence Resource model when representing a Range along with another statistic value (e.g. mean or median) for a dataset (combination of variables). The Range data can be modeled as additional statistic types with singular quantities (minimum observed value, maximum observed value, difference between maximum and minimum observed values). However, in common practice, the Range has been reported as an "attribute estimate" (similar to the reporting of confidence intervals) while the Range is not truly an attribute of the primary statistical estimate. There is a 'convenience factor' in reporting ranges and similar measures of dispersion (interquartile range, standard deviation) as attribute estimates but it is not technically the same thing as attributes of the statistical estimate (such as p values, confidence intervals, and credible intervals). We will continue the modeling discussion next week to determine how we think this is best handled in FHIR representation of statistics, then return to definitions of Measures of Dispersion.

On February 7, the Statistic Standard and Terminology Working Group decided to change the meeting time to Mondays 2 pm Eastern to accommodate participants whose calendars changed, found 2 terms approved (Regression Coefficient, Measure of Dispersion), discussed coordination between SEVCO and STATO, and modelled the representation of Range data in FHIR (Sample Evidence with Range) ahead of defining the Range term.

On February 13, the Statistic Standard and Terminology Working Group drafted 2 statistic terms open for vote for SEVCO.  Next week we will model how to communicate then define terms that are types of Standard Deviation.

On February 20, the Statistic Standard and Terminology Working Group found 2 terms approved (Range, Interquartile range), and used the FEvIR®: Adaptation Builder/Viewer to convert the Sample Evidence with Range to Sample Evidence with Standard Deviation and discussed the potential uses of 'Standard Deviation' terms.  We will create examples of 'Standard Deviation for Population' to model next week, then draft terms based on the modeling exercise.

On February 27, the Statistic Standard and Terminology Working Group attempted to modify the Sample Evidence with Standard Deviation to include a use case for 'Standard Deviation of Population' and conceptualized this as part of the method for sample size estimation. In attempts to represent how this would be used in real-world communications of Evidence statistics, we realized we needed to modify the example from a univariate statistic (focusing on one group) to a comparative statistic (with an effect on a continuous outcome). Next week we will revise the model for this more complex and more meaningful scenario, then draft terms based on the modelling exercise.

On March 6, the Statistic Standard and Terminology Working Group modified the Sample Evidence with Standard Deviation to model a more complex and more meaningful scenario, with changes including (1) four variable definitions (Population = Patients with diabetes, Exposure = Drug A, Reference exposure = Placebo, Measured variable = Systolic blood pressure), (2) coding the statistic type as 'Difference in means', (3) coding an attribute estimate of the statistic as 'Standard error of the difference between means', (4) adding three model characteristics to the statistic (coded as 'Two sample t-test with equal variance', 'two-tailed test', and 'individual test alpha without multiple testing adjustment', and (5) adding a fourth model characteristic as a placeholder for 'The statistical analysis plan included a sample size calculation based on the goal of hypothesis testing for a difference < - 5 mm Hg using alpha 0.05 (95% confidence interval), beta 0.2, and assumptions of a population standard deviation of 15 mm Hg in the measurement of systolic blood pressure in both treatment and reference arms'. The exercise was excellent in better understanding the relations between the Evidence Resource structure and the various terms in the Scientific Evidence Code System (SEVCO). The next modeling development we will continue next week is to create a separate Evidence Resource (StatisticalModel Profile) to represent the Statistical Analysis Plan/sample size calculation data, and how this relates to the Evidence Resource with the results of the statistical analysis.

On March 13, the Statistic Standard and Terminology Working Group copied the Sample Evidence with Standard Deviation (an example Evidence Resource to model a statistical finding with a standard deviation) and created Sample StatisticalModel Profile of Evidence Resource (an example Evidence Resource to model the statistical model–endpoint analysis plan). It took a while to express the statistical model for the specific Evidence in text (unstructured) form due to the complexity of fully expressing a statistical model.  We developed the following example to subsequently model as structured data:

H0: The null hypothesis is that the absolute value of the difference in mean systolic blood pressure between a group assigned Drug A and a second group assigned Placebo is < 5 mm Hg.

HA: The alternative hypothesis is that the absolute value of the difference in mean systolic blood pressure between a group assigned Drug A and a second group assigned Placebo is >= 5 mm Hg.

The sample size calculation is based on the unpaired, equal-variance t-test statistic, which is defined as the difference in means divided by the pooled standard error of the difference, having a t-distribution with degrees of freedom equal to the total sample size minus two, with an alpha of 0.05 and a beta of 0.2, and the assumption that the population standard deviation of the measurement of systolic blood pressure is 15 mm Hg in both groups. The test further presumes that the observed data are each normally distributed and demonstrated to have the same standard deviation.

The statistics that will be reported include the mean systolic blood pressure, standard deviation of systolic blood pressure, and sample size in each group.

In the structured data portion of the Evidence.statistic element, we retained the statisticType data (Difference in means) and retained the quantity.unit data (mm Hg) and we deleted data in quantity.value, sampleSize.numberOfParticipants, and sampleSize.knownDataCount as the results data are not available at the time of the statistical/endpoint analysis plan. Next week we will continue to revise the attributeEstimate and modelCharacteristic data in the current Sample to remove extraneous concepts and add additional details to fully specify the statistical model.

On March 20, the Statistic Standard and Terminology Working Group substantially revised the Sample StatisticalModel Profile of Evidence Resource (an example Evidence Resource to model the statistical model–endpoint analysis plan). A pseudo-code structured representation of the statistical model looks like:

description: H₀: The null hypothesis is that the absolute value of the difference in mean systolic blood pressure between a group assigned Drug A and a second group assigned Placebo is <= 5 mm Hg. H_A: The alternative hypothesis is that the absolute value of the difference in mean systolic blood pressure between a group assigned Drug A and a second group assigned Placebo is > 5 mm Hg. The sample size calculation is based on the unpaired, equal-variance t-test statistic, which is defined as the difference in means divided by the pooled standard error of the difference, having a t-distribution with degrees of freedom equal to the total sample size minus two, with an alpha of 0.05 and a beta of 0.2, and the assumption that the population standard deviation of the measurement of systolic blood pressure is 15 mm Hg in both groups. The test further presumes that the observed data are each normally distributed and demonstrated to have the same standard deviation. The statistics that will be reported include the mean systolic blood pressure, standard deviation of systolic blood pressure, and sample size in each group.

statisticType: coding: {system: https://fevir.net/resources/CodeSystem/27270, code: STATO:0000457, display: Difference in means}

quantity: unit: mm Hg

attributeEstimate: {description: standard error (of difference in means) with units of measure mm Hg, type: coding: {system: https://fevir.net/resources/CodeSystem/27270, code: TBD:0000063
display: Standard error of the difference between means}, quantity: unit: mm Hg}

modelCharacteristic 1: {code: text: Hypothesis testing margin, value: 5 mm Hg}

modelCharacteristic 2: {code: {system: https://fevir.net/resources/CodeSystem/27270, code: STATO:0000303, display: Two sample t-test with equal variance}, text: unpaired t-test with homogenous variance}

modelCharacteristic 3: {code: {system: https://fevir.net/resources/CodeSystem/27270, code: STATO:0000287, display: two-tailed test}

modelCharacteristic 4: {code: {system: https://fevir.net/resources/CodeSystem/27270, code: TBD:0000085, display: individual test alpha without multiple testing adjustment}, value: 0.05}

modelCharacteristic 5: {code: {system: https://fevir.net/resources/CodeSystem/27270, code: TBD:beta, display: Beta}, value: 0.2}

modelCharacteristic 6: {code: {system: https://fevir.net/resources/CodeSystem/27270, code: TBD:sample-size, display: Sample size estimation}, text: The sample size calculation is based on the unpaired, equal-variance t-test statistic, which is defined as the difference in means divided by the pooled standard error of the difference, having a t-distribution with degrees of freedom equal to the total sample size minus two, with an alpha of 0.05 and a beta of 0.2, and the assumption that the population standard deviation of the measurement of systolic blood pressure is 15 mm Hg in both groups.}

---attributeEstimate 1: {description: alpha 0.05, type: {system: https://fevir.net/resources/CodeSystem/27270, code: TBD:0000085, display: individual test alpha without multiple testing adjustment}, quantity: 0.05}
---attributeEstimate 2: {description: beta 0.2, type: {system: https://fevir.net/resources/CodeSystem/27270, code: TBD:beta, display: Beta}, quantity: 0.2}

---attributeEstimate 3: {description: assumption that the population standard deviation of the measurement of systolic blood pressure is 15 mm Hg in both groups, type: {system: https://fevir.net/resources/CodeSystem/27270, code: TBD:0000051, display: Standard deviation for population}, quantity: 15 mm Hg}

modelCharacteristic 7: {code: text: The test further presumes that the observed data are each normally distributed.}

modelCharacteristic 8: {code: text: The test further presumes that the observed data are demonstrated to have the same standard deviation.}

On March 27, the Statistic Standard and Terminology Working Group continued to refine a representation of a statistical model (endpoint analysis plan) in an Evidence Resource with Sample StatisticalModel Profile of Evidence Resource. In doing so, the following terms were used from (or added to) the draft Scientific Evidence Code System:

• Difference in means
• Standard error of the difference between means
• Hypothesis testing margin
• Two sample t-test with equal variance
• two-tailed test
• individual test alpha without multiple testing adjustment
• Beta
• Sample size estimation
• Standard deviation for population

To complete the model representation for this example, we need to determine if any of the following 3 concepts are to be represented as terms in SEVCO, and if so where to place them:

• allocation ratio
• assumption that observed data are normally distributed
• assumption that observed data have the same standard deviation

Once these 3 concepts are addressed, we will resume drafting terms for vote – but greatly informed by how the ‘whole package’ is represented in structured data.

Statistic Standard and Terminology WG 2023 Quarter 2

On April 3, the Statistic Standard and Terminology Working Group continued to refine a representation of a statistical model (endpoint analysis plan) in an Evidence Resource with Sample StatisticalModel Profile of Evidence Resource. In doing so, the following terms were used from (or added to) the draft Scientific Evidence Code System:

• Difference in means
• Standard error of the difference between means
• Hypothesis testing margin
• Two sample t-test with equal variance
• two-tailed test
• individual test alpha without multiple testing adjustment
• Beta
• Sample size estimation
• Standard deviation for population
• Allocation ratio

To complete the model representation for this example, we need to determine if how to add the following 2 concepts are to be represented as terms in SEVCO, and where to place them:

• Assumption that observed data are normally distributed
• Assumption that observed data have the same standard deviation

We drafted a definition for Allocation ratio so there is now 1 ‘statistic’ term (listed as a Study Design Feature term) open for vote:

To participate you can join the Scientific Evidence Code System Expert Working Group at https://fevir.net/resources/Project/27845.

On April 10, the Statistic Standard and Terminology Working Group found 1 term approved (Allocation ratio) and created 4 additional terms for future drafting (Statistical Model Characteristic, Statistical model assumption, Data distribution assumption of normal distribution, Data distribution assumption of equal standard deviations), completing all the terms needed for representation of a statistical model (endpoint analysis plan) in an Evidence Resource with Sample StatisticalModel Profile of Evidence Resource. Next week we will start drafting definitions for Standard deviation terms and then the terms needed for this example.

On April 17, the Statistic Standard and Terminology Working Group drafted 2 terms that are open for vote (Standard deviation, Standard deviation for sample).

On April 24, the Statistic Standard and Terminology Working Group had extensive discussions to distinguish statistics (defined as "An information content entity that is a formalization of relationships between variables and value specification") which include 'Standard deviation for sample' from statistical model parameters (to be defined) which include 'Standard deviation for population'. We moved 'Standard deviation for population' to become 'sigma' as a Statistical model parameter and will finish its term development next week. We added a Comment for application to the Standard deviation term and there are now 2 Statistic terms that are open for vote (Standard deviation, Standard deviation for sample).

On May 1, the Statistic Standard and Terminology Working Group drafted 2 terms (Statistical Model, Statistical Model Characteristic) so there are 4 statistical terms that are now open for vote.

On May 15, the Statistic Standard and Terminology Working Group approved 2 terms (Standard deviation, Standard deviation for sample), revised 1 term (Statistical Model Characteristic), moved ‘Statistical Model Parameter’ to be separate from ‘Statistical Model Characteristic’, and drafted 1 new term (Statistical Model Component) so there were 3 statistical terms open for vote.

On May 22, the Statistic Standard and Terminology Working Group revised the definition and comment for 1 term (Statistical Model), and revised the comments for 2 terms (Statistical Model Characteristic, Statistical Model Component) so there are 3 statistical terms that open for vote (Statistical Model, Statistical Model Characteristic, Statistical Model Component).

On June 5, the Statistic Standard and Terminology Working Group approved 2 terms (Statistical Model Characteristic, Statistical Model Component), revised the comment for 1 term (Statistical Model), and drafted 3 new terms, so there were 4 statistical terms open for vote (Statistical Model, Statistical Distribution Parameter, mu, sigma).

On June 12, the Statistic Standard and Terminology Working Group approved 1 term (Statistical Model) and reviewed comments on 2 terms (mu, sigma) showing challenges in combining definitions of abstract ‘Statistical Distribution Parameter’ and contextualized ‘Population Parameter’ concepts as synonymous terms and challenges in using symbols (e.g. Greek letters) as terms where symbols may have different meanings in different contexts. We ultimately revised 1 term (Statistical Distribution Parameter), drafted 1 new term as a type of Statistical Distribution Parameter (Population Parameter), and moved the mu/sigma terms to become types of Population Parameters.  We renamed these terms (Population mean, Standard deviation for population) but will continue to revise their definitions next week, so there are 2 statistical terms that are now open for vote (Statistical Distribution Parameter, Population Parameter).

On June 19, the Statistic Standard and Terminology Working Group discussed the complexities of overlapping concepts of Statistical Distribution Parameter, Population Parameter, Population Mean as a Population Parameter for a Normal Distribution, Population Mean as might occur as a population parameter for a non-normal distribution, and Population Mean as a ‘statistic’ representing the average of a population (but not a component of a probability distribution function). We revised a term and added a subordinate term to represent 4 terms in a hierarchical approach. These 4 terms are now open for vote. Please consider the hierarchical arrangement as you provided comments:

To participate you can join the Scientific Evidence Code System Expert Working Group at https://fevir.net/resources/Project/27845. 

On June 26, the name of the Statistic Standard and Terminology Working Group changed to Statistic Terminology Working Group. We worked through several variations of how to organize terms related to 'Statistical Distribution Parameters' and ultimately made major changes to the hierarchical set of terms. We removed all these terms from 'open for vote' for now and will continue mapping this out next week.  You can go to any of the terms directly and use the Comment feature if you want to share comments.  The current draft hierarchy of terms is now:

• Probability Distribution
  • Normal Distribution
  • Log Normal Distribution
  • Exponential Family of Distributions
  • Binomial Distribution
  • Multinomial Distribution
  • Poisson Distribution
  • Negative Binomial Distribution
• Probability Distribution Attribute
  • Probability Distribution Parameter
    • Population Parameter
      • Population mean
        • Population normal-distribution mean
      • Population standard deviation

Statistic Terminology WG 2023 Quarter 3

On July 10, the Statistic Terminology Working Group continued discussions to simplify the hierarchical listing of Probability Distribution Parameters.

We started with a draft framing of:

• Probability Distribution
• Normal Distribution
• Log Normal Distribution
• Exponential Family of Distributions
• Binomial Distribution
• Multinomial Distribution
• Poisson Distribution
• Negative Binomial Distribution
• Probability Distribution Attribute
  • Population Parameter
    • Population normal-distribution mean
  • Population mean
  • Population standard deviation
• Probability Distribution Parameter

We discussed the nuance of representing the “Population” reference of Probability Distribution Parameters in the structure using the terms rather than in the terms and removed the “Population Parameter” layer of this hierarchy, so the current draft framing is:

• Probability Distribution
• Normal Distribution
• Log Normal Distribution
• Exponential Family of Distributions
• Binomial Distribution
• Multinomial Distribution
• Poisson Distribution
• Negative Binomial Distribution
• Probability Distribution Attribute
  • Population mean
  • Population normal-distribution mean
  • Population standard deviation
• Probability Distribution Parameter

Next week we will consider the framing the innermost terms to set the optimal hierarchical approach, then resume drafting terms and definitions.

On July 17, the Statistic Terminology Working Group continued discussions to simplify the hierarchical listing of Probability Distribution Parameters.

We started with a draft framing of:

• Probability Distribution
• Normal Distribution
• Log Normal Distribution
• Exponential Family of Distributions
• Binomial Distribution
• Multinomial Distribution
• Poisson Distribution
• Negative Binomial Distribution
• Probability Distribution Attribute
  • Population mean
  • Population normal-distribution mean
  • Population standard deviation
• Probability Distribution Parameter

We discussed the nuance of representing the “Population” reference of Probability Distribution Parameters in the structure using the terms rather than in the terms and removed the “Population” terms throughout, and reorganized a bit, so the current draft framing is:

• Probability Distribution Attribute
• Probability Distribution Attribute
  • Normal Distribution
  • Log Normal Distribution
  • Exponential Family of Distributions
  • Binomial Distribution
  • Multinomial Distribution
  • Poisson Distribution
  • Negative Binomial Distribution
  • Mean as normal-distribution parameter
  • Variance as normal-distribution parameter
• Probability Distribution Class
• Distribution mean
• Distribution standard deviation
• Probability Distribution Parameter

Next week we will resume drafting terms and definitions within this hierarchy.

On July 24, the Statistic Terminology Working Group began to define terms within the heading of Probability Distribution Class and decided that we will try to define all of these terms together before sending them out for vote. These terms include:

• Probability Distribution Attribute
  • Normal Distribution
  • Log Normal Distribution
  • Exponential Family of Distributions
  • Binomial Distribution
  • Multinomial Distribution
  • Poisson Distribution
  • Negative Binomial Distribution
  • Mean as normal-distribution parameter
  • Variance as normal-distribution parameter
• Probability Distribution Class
• Distribution mean
• Distribution standard deviation
• Probability Distribution Parameter

On July 31, the Statistic Terminology Working Group drafted 3 terms (normal distribution, log normal distribution, exponential distribution) for types of Probability Distribution Class.  Your feedback through voting and comments can help us shape these and subsequent terms.

On August 7, the Statistic Terminology Working Group found 1 term approved (normal distribution), revised 2 terms (log normal distribution, exponential distribution), and drafted 1 term (binomial distribution), so there are now 3 Statistic-related terms open for vote (log normal distribution, exponential distribution, binomial distribution).

On August 14, the Statistic Terminology Working Group drafted 2 terms (multinomial distribution, Poisson distribution), so there are now 5 Statistic-related terms open for vote (log normal distribution, exponential distribution, binomial distribution, multinomial distribution, Poisson distribution).

On August 21, the Statistic Terminology Working Group drafted 3 terms (probability distribution attribute, probability distribution class, negative binomial distribution), so there were 8 Statistic-related terms open for vote.

On August 28, the Statistic Terminology Working Group drafted 1 term (distribution mean), so there are now 9 Statistic-related terms open for vote:

To participate you can join the Scientific Evidence Code System (SEVCO) Expert Working Group at https://fevir.net/resources/Project/27845.

Risk of Bias Terminology Working Group Updates:

This group is defining terms for > 200 types of bias based on a dozen commonly used risk of bias assessment tools. In prior years, ‘Risk of Bias Tooling’ was developed resulting in the Computable Publishing®: Risk of Bias Assessment Tool (RoBAT) and Computable Publishing®: Risk of Bias Assessment Reader (RoBAR), which was further enhanced by Risk of Bias Assessment Tool (RoBAT) Usability Research (RoBATUR) and this was presented at the Ninth International Congress on Peer Review and Scientific Publication September 8-10, 2022 as Development and Pilot Test of Risk of Bias Assessment Tool for Use in Peer Review.  Current efforts are related to defining types of Confounding Covariate Bias and there are 5 Bias-related terms open for vote:

Risk of Bias Terminology Working Group found 3 terms approved (Confounding by adherence to intervention, Confounding by indication, Confounding by contraindication) but considered a comment suggesting the avoidance of parenthetical phrases in definitions, so revised these 3 and 2 previously approved terms. We selected ‘Predictive Model Research Bias’ as the next area of terms for consideration and identified 13 concepts noted in the PROBAST Risk of Bias tool that are not obviously mapped to one of our completed terms. We will review these terms for mapping next week. There are now 4 Bias-related terms open for vote:

To participate you can join the Scientific Evidence Code System (SEVCO) Expert Working Group at https://fevir.net/resources/Project/27845.

Risk of Bias Terminology and Tooling WG 2023 Quarter 1

On January 20, the Risk of Bias Terminology and Tooling Working Group revised 1 terms (Selective comparison reporting), dropped 1 term from the Code System (Selective analysis reporting from availability bias, considered already covered by Availability bias affecting analysis selection) and drafted 1 additional term (Selective analysis reporting from repeated analyses at multiple times).

On January 27, the Risk of Bias Terminology and Tooling Working Group approved 3 terms (Selective subgroup reporting, Selective comparison reporting, Selective analysis reporting from repeated analyses at multiple times) and drafted 2 terms open for vote for SEVCO.

On February 3, the Risk of Bias Terminology and Tooling Working Group introduced our terminology tooling to several people from Europe developing terminologies related to risk assessment of chemical exposures. We then drafted 1 term for vote (Cognitive interpretive bias in reporting) and also drafted 1 more term (Interpretation of findings not addressing risk of bias) that we will continue to discuss next week.

On February 10, the Risk of Bias Terminology and Tooling Working Group found 1 term approved (Selective analysis reporting from multiple analytic models) and revised 2 terms which were re-opened for vote for SEVCO.

On February 17, the Risk of Bias Terminology and Tooling Working Group drafted 1 new term (Interpretation of results not addressing potential for bias).

On February 24, the Risk of Bias Terminology and Tooling Working Group found 2 terms approved (Selective threshold reporting bias, Cognitive interpretive bias in reporting), added a comment to the Cognitive interpretive bias in reporting term (Cognitive interpretive biases in reporting include selective theory reporting, confirmation bias, bias of rhetoric, novelty bias, popularity bias, and positive results bias.), removed 5 of those ‘subtype’ terms from the code system, and drafted 1 new term (Confirmation bias in reporting).

On March 3, the Risk of Bias Terminology and Tooling Working Group found 1 term approved (Interpretation of results not addressing potential for bias), drafted 1 new term and moved it from Reporting Bias to Analysis Bias (Reported analysis not following pre-specified analysis plan), and moved 1 term from Reporting Bias to Rating of Factor Presence (Inadequate reporting to assess analytic strategy).

On March 10, the Risk of Bias Terminology and Tooling Working Group found 2 terms approved (Reported analysis not following pre-specified analysis plan, Confirmation bias in reporting), and drafted 3 terms within Reporting Bias (Inadequate Reporting Bias, Inadequate reporting of methods, Inadequate explanation of participant withdrawals).

On March 17, the Risk of Bias Terminology and Tooling Working Group removed one proposed term from the terminology (Inadequate Reporting Bias) as SEVCO Expert Working Group feedback highlighted that it was duplicative with “Selective Reporting Bias” and “Inadequate reporting of methods” and confusing to have a separate overlapping term. We drafted 1 additional term (Pre-final publication reporting bias).

On March 24, the Risk of Bias Terminology and Tooling Working Group drafted 1 additional term (Results emphasized based on statistical significance).

On March 31, the Risk of Bias Terminology and Tooling Working Group found 2 terms approved (Inadequate reporting of methods, Inadequate explanation of participant withdrawals) and substantially revised 1 term (Pre-final publication reporting bias is now Premature reporting bias).

Risk of Bias Terminology and Tooling WG 2023 Quarter 2

On April 7, the Risk of Bias Terminology and Tooling Working Group found 1 term approved (Results emphasized based on statistical significance) with additional Comment for application (“Results may be interpreted based on statistical significance instead of clinical significance, or results may misrepresent statistical significance and clinical significance as synonymous.”), substantially revised 1 term (Premature reporting bias is now Early dissemination bias), and drafted 1 new term (External validity bias).

On April 14, the Risk of Bias Terminology and Tooling Working Group found 2 terms approved (Early dissemination bias and External validity bias), made additions to the Comment for application for Early dissemination bias (“One form of Early dissemination bias is the reporting of results in preprints or early versions during the peer review and publication process not matching the subsequent reports. Another form of Early dissemination bias is the reporting of interim results (even if fully peer reviewed) when a study is ongoing and more data will be analyzed for the final results. This bias may result from failure to disclose that the results are preliminary or subject to change. This definition is not meant to indicate that preprints are inherently biased.“), and drafted 1 new term (Fabrication Bias).

On April 21, the Risk of Bias Terminology and Tooling Working Group found 1 term approved (Fabrication bias) and drafted 7 terms (all related to Attrition bias) open for vote for SEVCO.

On April 28, the Risk of Bias Terminology and Tooling Working Group removed the Comment for application from 2 terms (Attrition bias due to participant attrition, Attrition bias due to missing data) following SEVCO Expert Working Group feedback that the comment was not well supported, and drafted 2 terms (Inadequate random sequence generation and Confounding by indication related to Confounding Covariate Bias).

On May 12, the Risk of Bias Terminology and Tooling Working Group found 5 terms approved (Confounding by indication, Attrition bias due to participant attrition, Attrition bias due to missing data, Imbalance in missing data, Inadequate response rate) and renamed one term (from Inadequate random sequence generation to Non-random allocation) and added a Comment for application, so there are 4 Bias terms open for vote for SEVCO:

To participate you can join the Scientific Evidence Code System Expert Working Group at https://fevir.net/resources/Project/27845.

On May 19, the Risk of Bias Terminology and Tooling Working Group found 1 term approved (Attrition bias due to missing modifier data), one term with fewer than 5 votes (Non-random allocation), and 2 terms with negative votes to be reconciled (Attrition bias due to missing exposure data, Attrition bias due to missing outcome data) the definitions for these two terms were refined to reflect feedback from the expert working group. The remainder of the meeting was spent discussing potential improvements to the code system resource user interface on the FEvIR platform.

CodeSystem Builder UI suggested changes are as follows:

1. Button next to the date fields to automatically insert the current UTC time.
2. See the Votes and Comments while in edit mode.
3. A button next to the Votes header to come up with a summary of the votes if they're all "yes" votes after the specified Open for Voting time?

On May 26, the Risk of Bias Terminology and Tooling Working Group revised 1 term (Bias due to non-random allocation) and drafted 2 new terms (Confounding by time of observation, Lead time bias) so there are now 5 Bias-related terms open for vote:

To participate you can join the Scientific Evidence Code System (SEVCO) Expert Working Group at https://fevir.net/resources/Project/27845.

On June 2, the Risk of Bias Terminology and Tooling Working Group found an insufficient number of votes for the 5 terms open for voting (so please vote on these terms), and discussed two additional terms (Confounding difference associated with adherence to intervention, Post-intervention confounding different). Both of these terms are derived from specific questions in the ROBINS-I instrument (1.3. Were intervention discontinuations or switches likely to be related to factors that are prognostic for the outcome?, 1.6. Did the authors control for any post-intervention variables that could have been affected by the intervention?). We struggled with the need for and substance of specific definitions as types of Confounding difference, and we ultimately decided to review the materials behind the ROBINS-I instrument and discuss this again next week.

On June 9, the Risk of Bias Terminology and Tooling Working Group found 1 term approved (Attrition bias due to missing exposure data) with the addition of a Comment for application (If coding a bias related to the classification of exposure, misclassification of exposure may be coded as Exposure Detection Bias, but if the data is excluded from analysis it may then be coded as Attrition bias due to missing exposure data.), 4 terms revised (so they all start over for vote counts), and 1 term added (Attrition bias due to missing data about attrition), so there are now 5 Bias-related terms open for vote (Bias due to non-random allocation, Confounding by time of observation, Lead time bias, Attrition bias due to missing outcome data, Attrition bias due to missing data about attrition).

On June 16, the Risk of Bias Terminology and Tooling Working Group found 2 terms approved (Attrition bias due to missing outcome data, Attrition bias due to missing data about attrition) with a comment change to hyphenate “time-to-event” in the latter term. Upon evaluating comments for Confounding Covariate Bias terms, we discovered an error in the previously approved definition for the term Confounding Covariate Bias (between..or vs. between..and) and we added a Comment for application to that term, so re-open this term for vote. Discussions shifted to overall scope and technology related to SEVCO development, and there are now 4 Bias-related terms open for vote (Confounding Covariate Bias, Bias due to non-random allocation, Confounding by time of observation, Lead time bias).

On June 23, the Risk of Bias Terminology and Tooling Working Group made substantial revisions to all aspects of 1 term (Bias due to non-randomized allocation), and there are now 4 Bias-related terms open for vote (Confounding Covariate Bias, Bias due to non-randomized allocation, Confounding by time of observation, Lead time bias).

On June 26, the name of Risk of Bias Terminology and Tooling Working Group was changed to Risk of Bias Terminology Working Group.

On June 30, the Risk of Bias Terminology Working Group made substantial revisions to several terms, including the reclassification of multiple terms as types of Confounding Covariate Bias rather than types of Confounding difference, which re-introduces some of these terms to be voted upon again. There are now 5 Bias-related terms open for vote (Confounding Covariate Bias, Bias due to non-randomized allocation, Confounding by time of observation, Lead time bias, Confounding by indication).

Risk of Bias Terminology WG 2023 Quarter 3

On July 7, the Risk of Bias Terminology Working Group revised 1 term (Confounding by indication), added 1 term (Confounding by contraindication), drafted 1 term (Confounding by adherence to intervention), and removed 1 term from SEVCO (Confounding by cointervention – not found in any of the Risk of Bias tools but later added due to mention in Stone JC, Glass K, Clark J, Ritskes-Hoitinga M, Munn Z, Tugwell P, Doi AR. The MethodologicAl STandards for Epidemiological Research (MASTER) scale demonstrated a unified framework for bias assessment. Journal of Clinical Epidemiology (2021), doi: https://doi.org/10.1016/j.jclinepi.2021.01.012), so there are now 7 Bias-related terms open for vote (Confounding Covariate Bias, Bias due to non-randomized allocation, Confounding by time of observation, Lead time bias, Confounding by adherence to intervention, Confounding by indication, Confounding by contraindication).

On July 14, the Risk of Bias Terminology Working Group found 3 terms approved (Confounding Covariate Bias, Confounding by time of observation, Lead time bias) and revised 3 terms, so there are now 4 Bias-related terms open for vote (Bias due to non-randomized allocation, Confounding by adherence to intervention, Confounding by indication, Confounding by contraindication).

On July 28, the Risk of Bias Terminology Working Group found 1 term approved (Bias due to non-randomized allocation) and revised 3 terms (changes highlighted below), so there are now 3 Bias-related terms open for vote (Confounding by adherence to intervention, Confounding by indication, Confounding by contraindication).

On August 4, the Risk of Bias Terminology Working Group found 3 terms approved (Confounding by adherence to intervention, Confounding by indication, Confounding by contraindication) but considered a comment suggesting the avoidance of parenthetical phrases in definitions, so revised these 3 and 2 previously approved terms. We selected ‘Predictive Model Research Bias’ as the next area of terms for consideration and identified 13 concepts noted in the PROBAST Risk of Bias tool that are not obviously mapped to one of our completed terms. We will review these terms for mapping next week. There are now 5 Bias-related terms open for vote.

On August 11, the Risk of Bias Terminology Working Group revised 1 term (Confounding by adherence to intervention) and grappled with the nuanced distinction between distortion in research results (bias) due to adherence gaps (called Performance bias or Compliance bias or Intervention adherence bias) and bias due to a Confounding covariate bias in which the distorting variable (confounder) is associated with adherence. The phrase “confounding by adherence” can be interpreted to mean bias due to adherence gaps, so we need less ambiguous phrasing. We changed the preferred term to “Confounding influencing adherence to intervention” and added a third paragraph to the Comment for application.  Next week we will review ‘Predictive Model Research Bias’ and consider 13 concepts noted in the PROBAST Risk of Bias tool that are not obviously mapped to one of our completed terms.

On August 18, the Risk of Bias Terminology Working Group found 3 terms approved (Lead time bias, Confounding by indication, Confounding by contraindication) and then reviewed the first of 13 concepts noted in the PROBAST Risk of Bias tool that are not obviously mapped to one of our completed terms. We mapped the triggering question (3.6 Was the time interval between predictor assessment and outcome determination appropriate?) to a new type of Detection Bias (Inappropriate time interval between predictor assessment and outcome determination). We will map the other 12 concepts then draft the next set of definitions. There are now 2 Bias-related terms open for vote (Confounding by time of observation, Confounding influencing adherence to intervention).

On August 25, the Risk of Bias Terminology Working Group added to the Comment for application for 1 term (Confounding by time of observation) and drafted 1 term (time-varying confounding affected by past exposure), so there are now 3 Bias-related terms open for vote (Confounding by time of observation, Confounding influencing adherence to intervention, time-varying confounding affected by past exposure).

On September 1, the Risk of Bias Terminology Working Group discussed one term that received a negative vote (time-varying confounding affected by past exposure) with a comment suggesting removal due to uncommon use. The group in the meeting believes this is a common concern in certain areas of science, including observational research of treatments that are calibrated based on monitoring. In a future meeting, if the negative vote persists, we will follow the protocol to prepare for a deliberative discussion. We reviewed some of the concepts in the PROBAST risk of bias assessment tool for predictive model research and identified changes to SEVCO where the concepts were not well accounted for.  These changes included the addition of a Comment for application to 1 term (Incorporation Bias for outcome determination) and adding 2 additional terms to the list (Inappropriate weighting bias, Predictor choice bias). No additional terms were drafted for vote, so there are now 3 Bias-related terms open for vote:

To participate you can join the Scientific Evidence Code System (SEVCO) Expert Working Group at https://fevir.net/resources/Project/27845.

On September 8, 2023 the Risk of Bias Terminology Working Group drafted a definition for one term which was added to SEVCO last week (Inappropriate weighting bias). There are now 4 bias-related terms open for vote:

Cohort Definition (Eligibility Criteria) Updates

We are modeling the optimal method to express characteristics (criteria) defining whether an entity is considered part of a group of entities. Terms for this concept include cohort definition, eligibility criteria, inclusion/exclusion criteria, phenotype. Such cohort definitions are used across healthcare and biomedical research to exchange data for research cohorts, clinical trials, clinical care recommendations, clinical quality measure population definitions, and prior authorization or coverage eligibility.

Developments to support expression of structured eligibility criteria with FHIR and implementation include:

• Developments using EvidenceVariable Resource as structural base
  • The EvidenceVariable StructureDefinition
  • Eligibility Criteria specification with EvidenceVariable with examples listed in Associated Resources
  • Confluence page descriptive summary and examples of specific EvidenceVariable.characteristic elements described at https://confluence.hl7.org/display/BRR/Compact+Reference+for+Evidence+Variable
  • Eligibility Criteria via EvidenceVariable (video recording) presented at the July 2022 CodeX mCODE Community of Practice Meeting (see Monthly Meeting Minutes)
  • Eligibility Criteria Matching Software Demonstration
  • Eligibility Criteria Matching Software Library
  • Handling of eligibility criteria in ClinicalTrials.gov data with the Computable Publishing®: ClinicalTrials.gov-to-FEvIR Converter
• Developments using 'Characteristic Resource' as structural base
  • Characteristic FHIR Resource Proposal 
  • FEvIR®: Characteristic Builder/Viewer 
  • Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition
• Past projects
  • EuroVulcan Eligibility Criteria Connectathon Track for the EUROVULCAN Conference and Connectathon in Paris March 14^th and 15th, 2023.
  • On December 27, we provided an RFI response to the Office of Science and Technology Policy (OSTP)/Office of the National Coordinator (ONC) Request for Information (RFI) on Data Collection for Emergency Clinical Trials and Interoperability Pilot.
  • 2023 - 05 Cohort Definition Track for the May 2023 HL7 FHIR Connectathon
• Active projects
  • FHIR Representation of Eligibility Criteria for Clinical Trials project from the HL7 Biomedical Research & Regulation (BRR) Work Group
  • 2023 - 09 Cohort Definition Track for the September 2023 HL7 FHIR Connectathon

Cohort Definition Updates 2023 Quarter 1

The HL7® Biomedical Research and Regulation (BRR) Work Group completed the proposal for an Eligibility Criteria for Clinical Trials Implementation Guide and discussed the coordination between this proposal and the Evidence Based Medicine Implementation Guide. The group will reconvene on January 26 to discuss this coordination before formal development of another Implementation Guide.

On January 6, 2023, the Eligibility Criteria Working Group discussed modeling of EvidenceVariable Resource including characteristic.definitionByTypeAndValue.valueId usage; relatedArtifact usage for “derived-from”, “transformed-with”, and “cite-as” relationship types; useContext usage for different applications of EvidenceVariable Resource creation (e.g. consolidating different natural language processing inputs into a final adjudication form for clinical application); and relations between PlanDefinition and EvidenceVariable Resources. We demonstrated the use of Eligibility Criteria Matching Software Library and FEvIR®: SoftwareScript Builder/Viewer for the representation of a ‘Characteristic Resource’.

On January 13, 2023, the Eligibility Criteria Working Group discussed the EvidenceVariable StructureDefinition and model uses of EvidenceVariable for characteristic definitions that combine other characteristic definitions, and the need for "Characteristic Resource" and the potential for meeting this need through the StudyEligibilityCriteria Profile or another related Profile.

On January 20, the Eligibility Criteria Working Group discussed the importance of creating Profiles for eligibility criteria specification (also called cohort definitions or phenotypes) and Profiles for characteristics used in building eligibility criteria specifications and supporting libraries for sharing cohort definitions and characteristics.  The characteristic concept is used across Group, EvidenceVariable, and Measure Resources and there is a need to making the characteristic definitions shareable in isolation from specific Group, EvidenceVariable, and Measure Resources. There is a concern with doing all of this as Profiles of EvidenceVariable Resource where the naming 'EvidenceVariable' would not be expected for many of the communities using characteristics for cohort definitions. With all the developments across EBMonFHIR, CDS/CQL, CodeX, Vulcan RWD, Vulcan Phenopackets, Helios, etc., there is a great need to coordinate efforts to standardize the expression of eligibility criteria/cohort definitions/authorization criteria/phenotype definitions (whatever the phrase, it is about representing a set of characteristics to define a group) -- we will propose a Characteristic Resource so this is not "buried" under a Profile of EvidenceVariable Resource.

On January 26, the HL7® Biomedical Research and Regulation (BRR) Work Group discussed preparations for an Eligibility Criteria Track at the EuroVulcan Connectathon in March. We also discussed the desire to have a Characteristic Resource in FHIR.  There were a dozen participants in attendance and common consensus that a dedicated core FHIR Resource is preferable to a Profile buried in a different Resource Structure. Reasons discussed for a distinct Resource with patterns that differ from other FHIR Resource structures included:

1) The Characteristic Resource needs to be more flexible and less opinionated than other FHIR Resources because the use cases are not tightly defined within a specific domain of health data exchange.

2) Data exchange regarding characteristics need to support data exchange between systems not developed around FHIR and FHIR-based systems.

3) Data exchange regarding characteristics at times need to support the exchange of concepts with structured data to support clear human understanding without regard for the representation of concepts within FHIR-based datasets AND at times need to support the exchange of concepts with structured data to support executable functions interacting with FHIR datasets.

On January 27, the Eligibility Criteria Working Group discussed a current implementation that is using an EvidenceVariable Resource for each characteristic in a set of inclusion and exclusion criteria, and then referencing these characteristics to build an EvidenceVariable Resource to represent the set of eligibility criteria. We discussed that we could simplify the EvidenceVariable Resource by reducing 5 elements (characteristic, characteristic.linkId, characteristic.description, characteristic.note, characteristic.definitionId) if we used referencing to each characteristic as a separate Resource. As a start to demonstrate this approach, we created a 'possible StructureDefinition' of a Characteristic Resource at https://fevir.net/resources/Characteristic/30934 and started to create StudyEligibilityCriteria: Characteristic Set for Bariatric Surgery Randomized Trial (Diabetes Surgery Study) (https://fevir.net/resources/Characteristic/111977) to re-create StudyEligibilityCriteria: Eligibility Criteria for Bariatric Surgery Randomized Trial (Diabetes Surgery Study) (https://fevir.net/resources/EvidenceVariable/32120) in the form of Characteristic Resources.

On February 2, the HL7® Biomedical Research and Regulation (BRR) Work Group discussed the Characteristic Resource, an example at https://fevir.net/resources/Characteristic/111977, and the key reasons for supporting it as a distinct FHIR Resource:

1. Easier implementation for many who know FHIR but not a specific expression language
   1. Easier readability
   2. Easier creation of content
2. Better digital object management at the singular characteristic/expression level
   1. Digital object identification at the singular characteristic/expression level
   2. Findability at the singular characteristic/expression level – searching for the characteristic/expression itself (e.g. when building a cohort definition and wanting to find related characteristic expressions) is different than the matching function
   3. Usage metadata management (status, experimental, copyright, etc.) at the singular characteristic/expression level
3. Greater expressiveness of abstract concepts
   1. Representation of the concept in structured form independent of a specified expression language
   2. Representation of concepts that are not ‘computable’ like comments and explanatory notes and subjective interpretations
4. Potential linking or data conversion conduit between forms of expression (data input for content authors, CQL expressions, human-readable expressions)

On February 3, the Eligibility Criteria Working Group reviewed and planned coordination with discussion occurring in HL7 Biomedical Research & Regulation (BRR) and Learning Health System (LHS) Work Groups. We also discussed information models for representing a characteristic that may be defined with a codeable concept (e.g. a SNOMED-CT term) but functionally needs to be defined by a combination of characteristics to interrogate laboratory values in FHIR-based data where the codeable concept is just a term defining the combination of laboratory test result findings.  The CodeSystem resource supports a string-based definition for each term. Mapping to a characteristic-based definition could potentially occur via an addition of definitionReference element to CodeSystem.concept to reference another resource for a structured definition, by development of a CharacteristicMap concept similar to ConceptMap for mapping terminologies, or creating a Characteristic Resource with 'any-of' combination of the two different methods for defining the characteristic.

On February 7, the HL7® Biomedical Research and Regulation (BRR) Work Group discussed the Characteristic Resource, and additional use cases included documentation of specific criteria making individual patients ineligible for a study, facilitated by having unique identifiers for each characteristic.  Groups like CDISC have substantial datasets of characteristics (or terminologies for characteristics). The more use cases we get, the better the proposal for a Characteristic Resource will be.

On February 9, the HL7® Biomedical Research and Regulation (BRR) Work Group started drafting a Characteristic FHIR Resource Proposal.  Key portions so far include:

Scope of coverage:

The Characteristic Resource describes a characteristic, factor, trait, or criterion used in the definition of criteria for membership in a group of entities. 

Resource appropriateness:

Characteristics, often called eligibility criteria, are well understood in the business of healthcare for selection of patients for many actions such as prior authorization, clinical trial participation, and clinical decision support delivery. Distinct, reliable, unique ids are used in cases where an individual is reported to be excluded from a group based on a specific characteristic and in maintenance of a library of characteristics. Content developers defining eligibility criteria, cohort definitions, phenotype definitions, etc. will independently create, query and maintain characteristics for re-use within and across organizations. Data exchange of characteristics currently suffers from no standardization that is searchable for the specific characteristic (as distinct from searching patient data for matching the characteristic). The Characteristic Resource has 22 "core" data elements in addition to the metadata pattern elements for a canonical resource.

Expected implementations:

Anyone expressing eligibility criteria (also called cohort definitions or phenotypes) may be expected to implement. Expected implementations include clinical trial recruitment (including feasibility assessment for research sites, pre-screening potentially eligible patients, and trial eligibility confirmation), clinical decision support delivery, prior authorization, and cohort selection for real-world data collection (research to generate real-world evidence). Initial implementations expected include representation of eligibility criteria for studies registered in ClinicalTrials.gov and the 'Representation of evidence-based clinical practice guideline recommendations on FHIR' as introduced in J Biomed Inform. 2023 Feb 2;104305. doi: 10.1016/j.jbi.2023.104305 https://pubmed.ncbi.nlm.nih.gov/36738871/

In addition, the extension of FHIR to support Evidence-Based Medicine knowledge transfer introduces many researchers, systematic literature reviewers, and clinical practice guideline developers that will implement tools to find, create, revise, and communicate characteristics for eligibility criteria for original research studies, for systematic reviews, and for clinical practice guideline recommendations.

Many HL7 communities using characteristics for cohort definitions include BRR, CDS, CQI, LHS, and FHIR-I Work Groups; and CodeX, Vulcan, and Helios Accelerators.

On February 10, the Eligibility Criteria Working Group revised the EuroVulcan Eligibility Criteria Connectathon Track for the EUROVULCAN Conference and Connectathon in Paris March 14^th and 15th.

On February 15, the HL7® Clinical Decision Support (CDS) Work Group approved the following changes to the FHIR specification: 

FHIR-37346 - Getting issue details... STATUS - Add EvidenceVariable.characteristic.instances[x], also add EvidenceVariable.characteristic.duration[x]

On February 16, the HL7® Biomedical Research and Regulation (BRR) Work Group completed the Characteristic FHIR Resource Proposal and will bring it for vote for approval on Tuesday. 

On February 17, the Eligibility Criteria Working Group reviewed and made minor revisions to the Characteristic FHIR Resource Proposal and the EuroVulcan Eligibility Criteria Connectathon Track for the EUROVULCAN Conference and Connectathon in Paris March 14^th and 15th.

On February 21, the HL7® Biomedical Research and Regulation (BRR) Work Group approved the Characteristic FHIR Resource Proposal by unanimous (15-0-0) vote. 

On February 22, the HL7® Clinical Decision Support (CDS) Work Group initiated communication with the HL7® Biomedical Research and Regulation (BRR) Work Group and HL7® Learning Health Systems (LHS) Work Group to coordinate meeting during the May 2023 HL7 Working Group Meeting to discuss Characteristic modeling and the Eligibility Criteria/Cohort Definition Connectathon developments.

On February 23, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed examples of Characteristic Resource with the FEvIR®: Characteristic Builder/Viewer and notified FHIR Management Group (FMG) of the Characteristic FHIR Resource Proposal.

On February 24, the Eligibility Criteria Working Group reviewed the FEvIR®: Characteristic Builder/Viewer and demonstrated the use cases for Characteristic.definitionByCombination.code = 'except-subset' and for Characteristic.expression having 0..* cardinality.

On February 27, the HL7® Learning Health Systems (LHS) Work Group drafted an initial proposal for a Cohort Definition Track for the May 2023 HL7 FHIR Connectathon.  Please contact us if you expect to participate and help us shape this Connectathon Track.

On March 1, the HL7® FHIR Management Group reviewed the Characteristic FHIR Resource Proposal and suggested to use the Group Resource for this purpose and asked us to conduct an analysis to answer What will it take to make Group Resource amenable to Characteristic Profile?

On March 2, the HL7® Clinical Decision Support (CDS) Work Group revised the Cohort Definition Track for the May 2023 HL7 FHIR Connectathon to include 4 System Roles (Cohort Definition Content Creation/Editing, Cohort Definition Content Viewing, Cohort Definition Content Machine Interpretation, and Repository of 'Cohort Definition Characteristics') and 2 Testing Scenarios (Expressing Cohort Definitions and Finding Cohort Definition Characteristics).

On March 3, the Eligibility Criteria Working Group added an expected participant to the Cohort Definition Track for the May 2023 HL7 FHIR Connectathon and discussed the response to the Characteristic FHIR Resource Proposal with a suggestion from FHIR Management Group to use the Group Resource for this purpose and ask of us to conduct an analysis to answer What will it take to make Group Resource amenable to Characteristic Profile?  

We desire to demonstrate specific use cases to ground the model for Characteristic expression, then use those examples to provide such an analysis. 

We discussed three possible options for making the examples: (1) a Profile of Group Resource, (2) a Profile of EvidenceVariable Resource, or (3) a 'Characteristic Resource' developed solely for the example demonstration. The group thought it was most efficient to model the ideal state for the use cases without compromise of extensions and fitting prior models designed for other purposes.

We discussed sharing phenotype expressions for use with other systems such as HDR UK Phenotype Library. Upon demonstrating an example, we found our proposed expression 0..* Expression element lacked the metadata needed to recognize the association of an expression with a specific library or for a particular purpose. Therefore, we modified our draft model for a Characteristic Resource to include executableExpression 0..* BackboneElement which includes 2 elements:  classifier 0..* CodeableConcept and expression 1..1 Expression.  Creating extensions upon extensions for this type of modeling presented a good example for the team of implementers to support our decision to develop our model in the conceptual space of a new resource before addressing how other Resources could be profiled or extended to meet the need.

On March 6, the HL7® Learning Health Systems (LHS) Work Group posted 2023 - 05 Cohort Definition Track to the HL7 Confluence page for the January 2023 Connectathon – the May 2023 Connectathon page is not yet available. Please contact us if you expect to participate and help us shape this Connectathon Track.

On March 7, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the developing work regarding the Characteristic FHIR Resource Proposal, a partially completed example of Acute Coronary Syndrome treated with oral antiplatelets to match the Vulcan Real World Data Implementation Guide use case, and plans for the Cohort Definition Track for the May 2023 HL7 FHIR Connectathon which will soon be moved to the May 2023 Connectathon pages.

On March 9, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the latest proposed Characteristic Resource StructureDefinition:

-all elements from MetadataResource Interface
-description                     0..1 markdown    Natural language description of the Characteristic
-note                               0..* Annotation    Additional or explanatory note
-exclude                          0..1 boolean         If true, preface definition with absence of
-definitionReference       0..1 Reference(Characteristic)
-definitionCanonical       0..1 Reference(Characteristic)
-definitionCodeableConcept 0..1 CodeableConcept
-definitionByTypeAndValue   0..1 BackboneElement
----type                                        1..1 CodeableConcept   The type of characteristic
----method                                   0..* CodeableConcept   How the characteristic value was determined
----device                                     0..1 Reference(Device | DeviceMetric)  Device used for determining characteristic value
----value[x]                                   1..1 CodeableConcept | boolean | Quantity | Range | Reference() | canonical()
----offset                                      0..1 CodeableConcept   Reference point for valueQuantity or valueRange
-definitionByCombination   0..1 BackboneElement
----code                                      1..1 code      all-of | any-of | at-least | at-most | except-subset | statistical
----threshold                               0..1 positiveInt       Provides the value of n when at-least or at-most code is used
----characteristic                         2..* Reference(Characteristic)
-instances[x]                        0..1 Quantity | Range     Number of occurrences meeting the characteristic
-duration[x]                         0..1 Quantity | Range     Length of time in which the characteristic is met
-timeFromEvent                  0..* BackboneElement
----description                           0..1 markdown
----note                                      0..* Annotation
----event[x]                                0..1 CodeableConcept | Reference() | dateTime
----quantity                               0..1 Quantity
----range                                   0..1 Range
-executableExpression      0..* BackboneElement
----classifier                              0..* CodeableConcept
----expression                          1..1 Expression

We made further progress in creating an example to represent the Acute Coronary Syndrome (ACS) with oral antiplatelets (OAPs) cohort definition used in the Vulcan RWD IG, and replied to comments on the Cohort Definition Zulip stream at https://chat.fhir.org/#narrow/stream/375683-cohort-definition.

On March 9, the HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination added a few examples of cohort definition instances to the Cohort Definition Track page for the May 2023 HL7 FHIR Connectathon at https://confluence.hl7.org/display/FHIR/2023+-+05+Cohort+Definition

On March 10, the Eligibility Criteria Working Group added a CharacteristicDefinition Extension and CohortDefinition Profile to the Evidence Based Medicine Implementation Guide. We then developed CohortDefinition instances for two of our working examples: R6 StudyEligibilityCriteria: Acute Coronary Syndrome treated with oral antiplatelets and R6 StudyEligibilityCriteria: Eligibility Criteria for Bariatric Surgery Randomized Trial (Diabetes Surgery Study).

On March 16, the HL7® Biomedical Research and Regulation (BRR) Work Group and HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination reviewed examples and discussed the implementation of expressing cohort definitions with EvidenceVariable.characteristic and with a ‘Characteristic Resource’ being used for optimal modelling. Detailed discussions about expectations for the Cohort Definition Track of May 2023 FHIR Connectathon led to several people planning to follow up with developers at their institutions regarding institution-specific benefits for participating.

On March 17, the Eligibility Criteria Working Group reviewed the following with multiple participants of the Vulcan FHIR Accelerator Schedule of Activities Project:

• HEvKA 2022 Achievements and 2023 Plans
• EvidenceVariable Resource StructureDefinition
• example with StudyEligibilityCriteria: Eligibility Criteria for Bariatric Surgery Randomized Trial (Diabetes Surgery Study)
• proposed Characteristic Resource StructureDefinition, modeled to be accessed from CharacteristicDefinition Extension and CohortDefinition Profile in the Evidence Based Medicine Implementation Guide
• Vulcan Real World Data IG example in Characteristic Resource form with Acute Coronary Syndrome treated with oral antiplatelets

On March 23., the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the Acute Coronary Syndrome treated with oral antiplatelets example of ‘Characteristic Resource’ (for the Vulcan RWD IG Cohort Definition example) and revised the Acute Coronary Syndrome ICD-10 Value Set Resource referenced within this example.

On March 23., the HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination discussed coordination of the Cohort Definition Track of May 2023 FHIR Connectathon with developments of several implementers (Epic, K-Grid), the discussions on the Cohort Definition Zulip stream at https://chat.fhir.org/#narrow/stream/375683-cohort-definition, and the cross-group (HL7 BRR WG and HEvKA Eligibility Criteria WG) efforts modeling the ‘Characteristic Resource’ in preparation for analysis of what it will take to adapt the Group Resource for all the use cases for data exchange of ‘Characteristic’ and ‘CohortDefinition’ objects.

On March 24, the Eligibility Criteria Working Group completed the Acute Coronary Syndrome treated with oral antiplatelets example by creating an Oral anticoagulant value set. Because the Acute Coronary Syndrome ICD-10 Value Set created for the Initial diagnosis of Acute Coronary Syndrome Characteristic was rather complex, we discussed an alternative using a terminology-related expression language.  We created Initial diagnosis of Acute Coronary Syndrome (terminology-related expression variant) to model it, and then added Characteristic.definitionByTypeAndValue.calculatedAs to our proposed Characteristic Resource StructureDefinition:

-all elements from MetadataResource Interface
-description                     0..1 markdown    Natural language description of the Characteristic
-note                               0..* Annotation    Additional or explanatory note
-exclude                          0..1 boolean         If true, preface definition with absence of
-definitionReference       0..1 Reference(Characteristic)
-definitionCanonical       0..1 Reference(Characteristic)
-definitionCodeableConcept 0..1 CodeableConcept
-definitionByTypeAndValue   0..1 BackboneElement
----type                                        1..1 CodeableConcept   The type of characteristic
----method                                   0..* CodeableConcept   How the characteristic value was determined
----device                                     0..1 Reference(Device | DeviceMetric)  Device used for determining characteristic value

----calculatedAs                            0..1 Expression              Formula used for determining characteristic value
----value[x]                                   1..1 CodeableConcept | boolean | Quantity | Range | Reference() | canonical() | Expression

----offset                                      0..1 CodeableConcept   Reference point for valueQuantity or valueRange
-definitionByCombination   0..1 BackboneElement
----code                                      1..1 code      all-of | any-of | at-least | at-most | except-subset | statistical
----threshold                               0..1 positiveInt       Provides the value of n when at-least or at-most code is used
----characteristic                         2..* Reference(Characteristic)
-instances[x]                        0..1 Quantity | Range     Number of occurrences meeting the characteristic
-duration[x]                         0..1 Quantity | Range     Length of time in which the characteristic is met
-timeFromEvent                  0..* BackboneElement
----description                           0..1 markdown
----note                                      0..* Annotation
----event[x]                                0..1 CodeableConcept | Reference() | dateTime
----quantity                               0..1 Quantity
----range                                   0..1 Range
-executableExpression      0..* BackboneElement
----classifier                              0..* CodeableConcept
----expression                          1..1 Expression

On March 30, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the Acute Coronary Syndrome treated with oral antiplatelets example of ‘Characteristic Resource’ (for the Vulcan RWD IG Cohort Definition example) and created the Initial diagnosis of Acute Coronary Syndrome (terminology-related expression variant 2) example to demonstrate how to use an expression to define a generated value set as an alternative to creating a ValueSet Resource. In developing this model, we added Expression as a datatype to Characteristic.definitionByTypeAndValue.value[x] in the proposed Characteristic Resource StructureDefinition.

On March 30, the HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination set April 27 as the Track Kickoff time for the Cohort Definition Track of May 2023 FHIR Connectathon and discussed coordination of the cross-group projects with potential developments of creating a virtual space in the HL7 LHS WG Confluence page and/or re-organizing the HEvKA Summary Update to consolidate Cohort Development efforts distinct from other Standards Development efforts.

On March 31, the Eligibility Criteria Working Group revised the Initial diagnosis of Acute Coronary Syndrome (terminology-related expression variant 2) example to demonstrate how to use an expression to define a generated value set as an alternative to creating a ValueSet Resource. In developing this model, we added Expression as a datatype to Characteristic.definitionByTypeAndValue.value[x] in the proposed Characteristic Resource StructureDefinition. We created an example valueExpression instance with:

description: any values including the SNOMED-CT equivalents of ICD-10 I20-I25 and any descendants
language: text/ecl
expression: << 414545008 | Ischemic heart disease

We then defined a specific process for use across the multiple meetings and working groups developing the Characteristic Resource StructureDefinition as an optimal model for Cohort Definition data exchange, including:

1. Map out each element with Element name (including path), Cardinality, Datatype, Short description, Definition, Comment, Requirements, Invariants, Example(s), and Status-of-group-acceptance (in a Google Sheets for our working effort).
2. Across these working group meetings, we will review each component and revise until we have acceptance by the Working Group participants and Cohort Definition Track (of HL7 FHIR May Connectathon) participants.
3. When done, this will document the optimal model (StructureDefinition) which we will then use to provide an analysis of how 'Group' would have to change to support this model.

Cohort Definition Updates 2023 Quarter 2

On April 6, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition.

On April 6, the HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination were introduced to the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition, introduced to the re-organized HEvKA Summary Update page section summarizing these developments and added this link and links to past meeting recordings to the Cohort Definition Track of May 2023 FHIR Connectathon page. We also started preparing to summarize interested parties, their domains of interest, and use cases to be listed on an HL7 LHS Work Group Confluence page for additional coordination.

On April 7, the Eligibility Criteria Working Group filled out 3 rows of the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition, leading to substantial clarification and adjustment of examples for 3 elements:

On April 13, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition and revised and 'approved' 3 elements:

• description 0..1 markdown Natural language description of the Characteristic
  • definition: A human readable narrative summarizing the Characteristic definition
  • requirements: Need to be able to describe characteristics in natural language so that end users can understand the criteria.
  • comment: Use of this element is recommended in all cases to facilitate human inspection if questions arise regarding the structured data. Use of this element is necessary in cases with no data in any of definitionReference, definitionUri, definitionCodeableConcept, definitionByTypeAndValue, or definitionByCombination elements -- an approach that may be used for Characteristics that are unable to be represented with structured data (e.g. "no subjective criteria for which participation would introduce an undue hardship")
  • Invariants: description element must have a value if no values found in any of 5 definition elements; description element should have a value
  • Examples: "Diagnosed with T2DM at least 6 months prior to enrollment, under the active care of a doctor for at least the six months prior to enrollment, and HbA1c ≥ 8.0%." or "Expect to live or work within approximately one hour's traveling time from the study clinic for the duration of the two-year trial."
• note 0..* Annotation Additional or explanatory note
  • definition: A human readable narrative summarizing an additional consideration (such as a qualifier, modifier, or rationale) that is not part of the key description or definition of the characteristic.
  • Example: "The evidence and guideline panel decision to modify the BMI threshold for Asian Americans is based on data mostly from Asian Americans not generally including Native Hawaiians and other Pacific Islanders, so it is not explicit whether the modified thresholds apply to Native Hawaiians and other Pacific Islanders. See https://diabetesjournals.org/care/article/38/1/150/37769/BMI-Cut-Points-to-Identify-At-Risk-Asian-Americans for details"
• exclude 0..1 boolean Whether the characteristic is an inclusion criterion or exclusion criterion
  • definition: An expression of negation to be applied to the definition of the characteristic. When true, this characteristic is an exclusion criterion.
  • comment: This element value is applied to the value in a definition element. If true, this element prefaces the definition value with 'absence of'. This element does NOT apply to the value of the description element (i.e. the description element should include the negation concept when applicable.).
  • Invariants: if exclude element has a value, then there must be a value in definitionReference, definitionUri, definitionCodeableConcept, definitionByTypeAndValue, or definitionByCombination.
  • Examples: description: "not an adult", exclude: true, definitionCodeableConcept.text: "adult" --or-- description: "child (minor age)", exclude: true, definitionCodeableConcept.text: "adult" --or-- description: "abnormal systolic blood pressure", exclude: true, definitionByTypeAndValue.valueRange: {low: 120 mmHg, high: 140 mmHg}

The note of 'approval' does not mean the optimal model is set. Additional feedback is welcome.

On April 13, the HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination reviewed the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition, replaced definitionCanonical with definitionUri, and revised and 'approved' 2 elements:

• definitionReference 0..1 Reference(Characteristic) Defines the characteristic using a Reference
  • definition: Defines the characteristic using a Reference Datatype which references a FHIR Characteristic Resource. In other words, this characteristic is defined by re-using a pre-existing Characteristic Resource.
  • requirements: re-use of existing characteristic definitions
  • comment: Use this definition element when the characteristic is already defined with a Resource and the Resource is referenced by an absolute path, relative path or identifier
  • Invariants: No more than one of definitionReference, definitionUri, definitionCodeableConcept, definitionByTypeAndValue, and definitionByCombination may have a value.
  • Example: diabetes complications is one characteristic; diabetes complications in the past 2 years is a different characteristic that may be expressed as definitionReference: {diabetes complications}, timeFromEvent: {in the past 2 years}
• definitionUri 0..1 uri Defines the characteristic using a URI (typically a URL)
  • definition: Defines the characteristic using a URI (typically a URL) to an external definition.
  • requirements: re-use of existing characteristic definitions
  • comment: Use this definition element when the characteristic is already defined with an artifact that is identified by URI. This URI is an identifier and often will be the URL (absolute path).
  • Invariants: No more than one of definitionReference, definitionUri, definitionCodeableConcept, definitionByTypeAndValue, and definitionByCombination may have a value.
  • Example: https://phekb.org/phenotype/type-2-diabetes-mellitus

The note of 'approval' does not mean the optimal model is set. Additional feedback is welcome.

On April 14, the Eligibility Criteria Working Group discussed the element, definitionCodeableConcept and filled out 1 row of the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition, leading to substantial clarification and adjustment of this element:

On April 20, the Eligibility Criteria Working Group (meeting in lieu of the HL7® Biomedical Research and Regulation (BRR) Work Group) revised the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition, with changes to definitionCodeableConcept, definitionByTypeAndValue, definitionByTypeAndValue.type, definitionByTypeAndValue.method, definitionByTypeAndValue.device, and definitionByTypeAndValue.calculatedAs elements.

On April 21, the Eligibility Criteria Working Group reviewed the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition, and 'approved' 5 elements:

• definitionCodeableConcept 0..1 CodeableConcept Defines the characteristic using a single codeable concept
  • definition: Defines the characteristic using a single codeable concept, using coding(s) from a terminology and/or text.
  • requirements: re-use of existing characteristic definitions when available as codeable concepts
  • comment: This element is used when a single concept represents the entire definition of the characteristic.
  • Invariants: No more than one of definitionReference, definitionUri, definitionCodeableConcept, definitionByTypeAndValue, and definitionByCombination may have a value.
• definitionByTypeAndValue 0..1 BackboneElement Defines the characteristic with an attribute and its value
  • definition: Defines the characteristic with an attribute (as a concept) and its value (in an appropriate datatype), potentially with modifier elements.
  • requirements: Characteristic definition with an attribute-value pair is used across Group and EvidenceVariable Resources.
  • comment: This element is used when an attribute value pair is needed to express the characteristic, so one codeable concept is needed to define the attribute and a second value (codeable concept or other datatype) is needed to define the value for that attribute.
  • Invariants: No more than one of definitionReference, definitionUri, definitionCodeableConcept, definitionByTypeAndValue, and definitionByCombination may have a value.
• definitionByTypeAndValue.type 0..1 CodeableConcept The attribute in an attribute-value pair defining the characteristic
  • definition: The attribute that, coupled with the corresponding value, defines the characteristic.
  • requirements: Need a formal way of identifying the attribute of the characteristic being described, used for characteristics defined with a single attribute.
• definitionByTypeAndValue.method 0..* CodeableConcept How the characteristic value was determined
  • definition: The process, method, or system specifying how the characteristic value was determined.
  • requirements: The method of determining the attribute value may not be included in the codeable concept defining the attribute, and may be necessary to fully express the characteristic definition.
• definitionByTypeAndValue.device 0..1 Reference(DeviceDefinition | Device | DeviceMetric) Device used for determining characteristic value
  • definition: The device used for determining the characteristic value.
  • requirements: The device used for determining the attribute value may not be included in the codeable concept defining the attribute, and may be necessary to fully express the characteristic definition.
  • comment: DeviceMetric Resource is equivalent to a Device Resource (referenced from DeviceMetric) and additional parameters representing the configuration or settings for the device. The DeviceMetric references the Device so there is no need to reference both from Characteristic.

On April 27, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition and revised and 'approved' 4 elements:

• definitionByTypeAndValue.calculatedAs 0..1 Expression Formula used for determining characteristic value
  • definition: The mathematical formula or computational expression used to calculate the characteristic value.
  • requirements: The formula or expression used for determining the attribute value may not be included in the codeable concept defining the attribute, and may be necessary to fully express the characteristic definition.
  • comment: This element (calculatedAs) is not used to fully define a characteristic with an Expression; it is only used to define the expression for how the characteristic value (in the observed instance) is calculated. Within the Expression datatype, use description element for a natural language summary of the expression, reference element for a uri or link to the expression, and/or expression element for a formal specification of the expression (coupled with the language element to specify the expression language used).
• definitionByTypeAndValue.value[x] 1..1 CodeableConcept | boolean| Quantity | Range | Reference{) | canonical() | Expression The value in an attribute-value pair defining the characteristic
  • definition: The value that, coupled with the corresponding attribute, defines the characteristic.
  • requirements: Need a formal way of identifying the value of the characteristic being described, used for characteristics defined with a single attribute, and the way of identifying the value must support many different datatypes.
• definitionByTypeAndValue.offset 0..1 CodeableConcept Reference point for value defined as distance from a point
  • definition: A reference point that is used, typically in conjunction with valueQuantity or valueRange, to fully specify the value as an amount or distance from the reference point.
  • requirements: Hemoglobin concentration may have a value of 9 g/dL, or a value of 1 g/dL below the lower normal limit. The latter expression requries an 'offset' to represent the concept of the lower normal limit.
• definitionByCombination 0..1 BackboneElement Used to specify how two or more characteristics are combined
  • definition: Defines the characteristic as a combination of two or more characteristics.
  • requirements: Many cohort definitions require combinations of characteristics, including component characterstics that are defined by combinations of characteristics.
  • comment: This element (calculatedAs) is not used to fully define a characteristic with an Expression; it is only used to define the expression for how the characteristic value (in the observed instance) is calculated. Within the Expression datatype, use description element for a natural language summary of the expression, reference element for a uri or link to the expression, and/or expression element for a formal specification of the expression (coupled with the language element to specify the expression language used).
  • Invariants: No more than one of definitionReference, definitionUri, definitionCodeableConcept, definitionByTypeAndValue, and definitionByCombination may have a value.

The note of 'approval' does not mean the optimal model is set. Additional feedback is welcome.

On April 27, the HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination reviewed the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition and drafted the remaining 14 elements:

• definitionByCombination.code 1..1 code     all-of | any-of | at-least | at-most | except-subset | statistical         Used to specify the method of combination of characteristics.
• definitionByCombination.threshold 0..1 positiveInt     Provides the value of "n" when "at-least" or "at-most" codes are used
• definitionByCombination.characteristic 2..* Reference(Characteristic)     A characteristic that is being combined with one or more other characteristics.
• instances[x] 0..1 Quantity | Range   Number of occurrences meeting the characteristic.
• duration[x] 0..1 Quantity | Range   Length of time in which the characteristic is met.
• timeFromEvent 0..* BackboneElement    Timing in which the characteristic is determined.
  • timeFromEvent.description 0..1 markdown      Natural language description of the timing      A human readable narrative summarizing the timing in which the characteristic is determined.
  • timeFromEvent.note 0..* Annotation      Additional or explanatory note      A human-readable string to clarify or explain concepts about the timeFromEvent.
  • timeFromEvent.event[x] CodeableConcept | Reference() | dateTime      The event used as a base point (reference point) in time.
  • timeFromEvent.quantity 0..1 Quantity      Used to express the observation at a defined amount of time before or after the event.
  • timeFromEvent.range 0..1 Range       Used to express the observation within a period before and/or after the event.
• executableExpression 0..* BackboneElement     An expression plus metadata     An executable expression with or without classifier(s).
  • executableExpression.classifier 0..* CodeableConcept      A categorization of the executable expression.
  • executableExpression.expression 1..1 Expression      An expression that generates a value.

On April 28, the Eligibility Criteria Working Group discussed preparations for the 2023 - 05 Cohort Definition Track for the May 2023 HL7 FHIR Connectathon and noted we will not meet on May 5 due to travel to attend the Connectathon.

On May 4, the HL7® Biomedical Research and Regulation (BRR) Work Group reviewed the Google Sheet being used to develop the 'Characteristic Resource' StructureDefinition and revised and 'approved' the final 14 elements:

• definitionByCombination.code 1..1 code     all-of | any-of | at-least | at-most | except-subset | statistical         Used to specify the method of combination of characteristics.
  • Comment/Invariant: If code is "at-least" or "at-most" then threshold SHALL be used. If code is "all-of" or "any-of" then threshold SHALL NOT be used.
• definitionByCombination.threshold 0..1 positiveInt     Provides the value of "n" when "at-least" or "at-most" codes are used
  • Invariant: If code is "at-least" or "at-most" then threshold SHALL be used. If code is "all-of" or "any-of" then threshold SHALL NOT be used.
• definitionByCombination.characteristic 2..* Reference(Characteristic)     A characteristic that is being combined with one or more other characteristics.
• instances[x] 0..1 Quantity | Range   Number of occurrences meeting the characteristic. (Invariant: Units are set to 1.)
• duration[x] 0..1 Quantity | Range   Length of time in which the characteristic is met.
  • Comment: For example, a characteristic of "having at least 6 months of experiencing dialysis at least 3 months prior to enrollment" would have a durationQuantity element to represent "at least 6 months of"
  • Invariant: Units are set to measures of time.
• timeFromEvent 0..* BackboneElement    Timing in which the characteristic is determined.
  • • Requirements: The timing when the characterstic value is determined is different than the timing as frequency of occurence (instances[x]) or the duration of how long the characteristic value is present (duration[x]) or a measure of an attribute defining the characteristic (definitionByTypeAndValue). 
    • Comment: For example, a characteristic of "having at least 6 months of experiencing dialysis at least 3 months prior to enrollment" would have a timeFromEvent element to represent "at least 3 months prior to enrollment"
  • timeFromEvent.description 0..1 markdown      Natural language description of the timing      A human readable narrative summarizing the timing in which the characteristic is determined.
  • timeFromEvent.note 0..* Annotation      Additional or explanatory note      A human-readable string to clarify or explain concepts about the timeFromEvent.
  • timeFromEvent.event[x] CodeableConcept | Reference() | dateTime      The event used as a base point (reference point) in time.
    • Comment: Use dateTime when expressing timeFromEvent as a time point on the calendar. For example, a characteristic of "having at least 6 months of experiencing dialysis at least 3 months prior to enrollment" would have a timeFromEvent.eventCodeableConcept element to represent "enrollment" (dialysis may be represented in the characteristic.definitionCodeableConcept or characteristic.definitionByTypeAndValue element)
  • timeFromEvent.quantity 0..1 Quantity      Used to express the observation at a defined amount of time before or after the event.
    • Comment: Use value=0 to specify "At" the moment of the event. Positive values mean "after" and negative values mean "before" the event. For example, a characteristic of "having at least 6 months of experiencing dialysis at least 3 months prior to enrollment" would have a timeFromEvent.quantity element to represent "at least 3 months prior" with a value of "<= -3 months" (at least 3 months AFTER would be ">= 3 months"; 'at least BEFORE' requires a < comparator, i.e. -4 months is before -3 months)
    • Invariant: Units are set to measures of time. Units are unnecessary if value=0.
  • timeFromEvent.range 0..1 Range       Used to express the observation within a period before and/or after the event.
    • Comment: Use value=0 to specify "At" the moment of the event. Positive values mean "after" and negative values mean "before" the event. See additional comments under timeFromEvent.quantity 
    • Invariant: Units are set to measures of time. Units are unnecessary if value=0.
• executableExpression 0..* BackboneElement     An expression plus metadata     An executable expression with or without classifier(s).
  • • Requirements: When exchanging data for expression instances, there is a need to include metadata, such as classifiers of the expression, that are not found in the Expression datatype. 
  • executableExpression.classifier 0..* CodeableConcept      A categorization of the executable expression.
    • Comment: Examples may include use context (e.g. US-based model for race/ethnicity data exchange), technology compatibility expectations, or keywords for search support.
  • executableExpression.expression 1..1 Expression      An expression that generates a value.

The note of 'approval' does not mean the optimal model is set. Additional feedback is welcome.

On May 4, the HL7® Clinical Decision Support (CDS) Sub-Work Group for LHS Connectathon Coordination prepared an agenda for Cohort Definition Track review, review of Characteristic Resource optimal model, and next steps planning for use during the HL7 Work Group Meeting next week.

On May 6 and May 7, the HL7® FHIR® Connectathon included the Cohort Definition Track.  Our Track Report Out includes:

Cohort Definition Track Report Out

• What was the track trying to achieve?
  • Demonstration of user-friendly tools (for the non-tech user) to view and create Cohort Definitions in ‘Characteristic Resource’ FHIR structure
• List of participants (with logos if you have time and energy)
  • Brian S. Alper, CEO, Computable Publishing LLC
  • Khalid Shahin, Senior Software Engineer, Computable Publishing LLC
  • Joanne Dehnbostel, Research and Analysis Manager

We also had great contacts and interest with many of you, including:

• • Grahame Grieve
  • Robert Dieterle
  • Craig Newman
  • Mark Roche
  • Sharon Hibay
  • Floyd Eisenberg
  • Giorgio Cangioli
  • Mario Hyland
  • Sam Schifman
• Notable achievements 
  • Major enhancements to the FEvIR®: Characteristic Builder/Viewer 
    • Release 0.4.4 (May 6, 2023) creates characteristic.definitionUri content instead of characteristic.definitionCanonical content in the Characteristic Builder and displays the data from either element if present in the Characteristic Viewer.
    • Release 0.5.0 (May 6, 2023) adds the ability to edit and view Characteristic.definitionByTypeAndValue.calculatedAs and Characteristic.definitionByTypeAndValue.valueExpression elements.
    • Release 0.6.0 (May 6, 2023) adds a picklist of common choices for the Attribute concept when defining a Characteristic by type and value, and then when selecting to define the value by CodeableConcept, adds a picklist if a type-specific picklist is available.
    • Release 0.7.0 (May 7, 2023) adds the ability to create new Characteristic Resources as needed from the Characteristic Builder when defining a Characteristic by combination of Characteristic Resources.
    • Release 0.8.0 (May 7, 2023) adds a tabular display of the Inclusion and Exclusion criteria (with links to each underlying Characteristic Resource) when editing a Characteristic Resource defined by a combination of Characteristics, and collapses the existing Characteristic reference data for a simpler editing interface for adding or editing references to Characteristic Resources in the definitionByCombination element.
  • Creation and/or display with explanation to example ‘Characteristic Resources’
    • 1. Acute Coronary Syndrome treated with oral antiplatelets
    • 2. Demonstration of Characteristic Resource elements
    • 3. Initial diagnosis of Acute Coronary Syndrome (terminology-related expression variant 2)
    • 4. StudyEligibilityCriteria: Characteristic Set for Bariatric Surgery Randomized Trial (Diabetes Surgery Study)
• Screenshots and/or links to further information
  • see Cohort Definition (Eligibility Criteria) Updates
• Discovered issues

• Need to include test plans in IG development

On May 8, the HL7® Orders and Observations (OO) Work Group discussed the proposed Characteristic Resource and suggested the name 'Characteristic' may be confused with other concepts like Device Characteristics.  We considered changing the Resource name to Criterion.

On May 9, the HL7® Learning Health Systems (LHS) Work Group and HL7® Clinical Decision Support (CDS) Work Group each received a report out of the Cohort Definition Track and discussed the evolution of reporting cohort definitions through Group Resource with simple characteristics, through Group Resource with a characteristicExpression extension, through EvidenceVariable Resource, and through the proposed Characteristic Resource as an optimal model.  There is a desire to have a common approach across the FHIR specification.  Increased interest in a Cohort Definition Track for the next Connectathon was noted, and the next steps include developing an analysis to suggest what is needed for a common approach.

On May 10., the HL7® Biomedical Research and Regulation (BRR) Work Group discussed the desire for a common approach to a 'Characteristic Resource' coordinated with existing FHIR Resources (Group or EvidenceVariable). We will work on a detailed analysis of what will need to change in the Group Resource to support this as a Profile of Group.  Changes to support it are expected to include:

1) Adding "other" as an option to Group.type which is required and bound to a required list of codes.

2) Adding up to 28 elements to the Group.characteristic element to support the 'Characteristic Resource' model.

3) Adding a number of metadata pattern elements to support canonical resource management for instances of Group used for sharing Cohort Definitions.

4) Minimizing the number of extensions required for implementation, to meet the needs of a research-oriented community seeking simple approaches to implement new standards.

The BRR Work Group also discussed an alternative pathway that includes changing EvidenceVariable Resource to become Criteria Resource, and changing EvidenceVariable.handling to Criteria.statisticalHandling, and changing EvidenceVariable.category to Criteria.statisticalCategory.  This approach would avoid a new-new Resource creation and maintain the ability to support complex criteria definitions for BOTH cohort definitions and evidence variable definitions.

On May 12, the Eligibility Criteria Working Group drafted an initial analysis of what changes would be required to the FHIR Group Resource StructureDefinition to support a CohortDefinition Profile that would address all the use cases developed in the 'Characteristic Resource Optimal Model'.  We will share and further develop this analysis in HL7® Biomedical Research and Regulation (BRR) Work Group meetings next week.

On May 17, the HL7® Clinical Decision Support (CDS) Work Group reviewed the proposed changes to the FHIR Group Resource StructureDefinition to support a CohortDefinition Profile, and modified the proposal to re-use existing Group Resource elements where possible.  A short version of the proposed changes is currently:

1. Add all elements from the MetadataResource Interface to Group StructureDefinition.
2. Add multiple elements to Group.characteristic (description, note, definitionReference, definitionUri, definitionCodeableConcept, method, device, calculatedAs, offset, combinationMethod, combinationThreshold, instances[x], duration[x], timeFromEvent, executableExpression)
3. Add additional datatypes to Group.characteristic.value[x] – adding canonical() and Expression
4. Change code 1..1 and value[x] 1..1 to 0..1 and add or modify some invariants related to use of alternative definitionXXX elements when not using the code/value[x] pattern
5. Change Group.type 1..1 code binding strength from Required to Extensible.

On May 19, the Eligibility Criteria Working Group discussed the new developments in the Characteristic Resource optimal model after refinement during the HL7 Connectathon and Working Group Meeting and further developments from the May 17th CDS meeting. The group further discussed potential changes in the user interface on the FEvIR platform to encourage use. 

On May 25., the HL7® Biomedical Research and Regulation (BRR) Work Group summarized the proposed changes to Group Resource StructureDefinition to support a CohortDefinition Profile:

1. Add all the elements from the MetadataResource Interface to Group. This is necessary to support the canonical resource management of Group instances for CohortDefinition use.
2. Change Group.type 1..1 code to Extensible binding instead of Required binding.
   1. Alternatively adding code options of ‘other’ and ‘type-independent’ to bypass the required set that does not fit other circumstances, e.g. a CohortDefinition instance to define search criteria for selecting research results for a systematic review of research results, or selecting specific findings meeting a set of criteria, or ‘age 3-6 months’ that can relate to many different ‘types’.
   2. Without a simple ‘bypass’ of this requirement, extensive development of workarounds to ‘fit’ will result in finding a solution other than Group Resource.
   3.  Group.type 1..1 code needs to either become 0..1 or must support a code value of “other”.
      1. The Group.type concept assumes all Groups are groups of entities that are a defined type and are found within a defined list.
      2. Using the Group Resource to represent any characteristic as a shareable digital object is only possible if the type concept (if required) can support any type of group.
      3. For example, a characteristic of “in existence before COVID-19 was introduced to the region” could be a shareable characteristic, yet could be applied to more than one type of entity.
3. Add Group.combinationMethod 0..1 code   all-of | any-of | at-least | at-most | except-subset | statistical
   1. INVARIANT: Group.combinationMethod is required if number of Group.characteristic instances > 1, default value is ‘all-of’ if not specified
4. Add Group.combinationThreshold 0..1 positiveInt       Provides the value of n when at-least or at-most code is used
5. Change Group.characteristic.code from 1..1 to 0..1 (see invariants below)
6. Change Group.characteristic.value[x] from 1..1 to 0..1 (see invariants below)
7. Change Group.characteristic.value[x] to include canonical() | Expression among the allowed datatypes.
8. Add the following elements to Group.characteristic:
   1. description 0..1 markdown    Natural language description of the Characteristic
   2. note 0..* Annotation    Additional or explanatory note
   3. definitionReference  0..1 Reference(Group | [CohortDefinition Profile] | EvidenceVariable)   INVARIANT: If definitionReference is used, then do not use code or value[x]
   4. definitionUri 0..1 uri   INVARIANT: If definitionUri is used, then do not use code or value[x]
      
   5. definitionCodeableConcept 0..1 CodeableConcept   INVARIANT: If definitionCodeableConcept is used, then do not use code or value[x]
      
   6. method 0..* CodeableConcept   How the characteristic value was determined (when used, it is a modifier of characteristic.code)
   7. device 0..1 Reference(Device | DeviceMetric)  Device used for determining characteristic value (when used, it is a modifier of characteristic.code)
   8. calculatedAs 0..1 Expression              Formula used for determining characteristic value (when used, it is a modifier of characteristic.code)
   9. offset 0..1 CodeableConcept   Reference point for valueQuantity or valueRange (when used, it is a modifier of characteristic.value[x])
   10. instances[x]  0..1 Quantity | Range     Number of occurrences meeting the characteristic
       
   11. duration[x]  0..1 Quantity | Range     Length of time in which the characteristic is met
       
   12. timeFromEvent   0..* BackboneElement
       1. timeFromEvent.description 0..1 markdown
       2. timeFromEvent.note  0..* Annotation
       3. timeFromEvent.event[x]  0..1 CodeableConcept | Reference() | dateTime
       4. timeFromEvent.quantity  0..1 Quantity
          
       5. timeFromEvent.range 0..1 Range
   13. executableExpression  0..* BackboneElement
       1. executableExpression.classifier  0..* CodeableConcept
       2. executableExpression.expression 1..1 Expression
          
          

On May 26, the Eligibility Criteria Working Group converted the proposed changes to Group to support CohortDefinition Profile into a spreadsheet table with a Rationale for each change.

On June 1, the HL7® Biomedical Research and Regulation (BRR) Work Group sent the analysis report of changes to Group Resource to support CohortDefinition Profile to FHIR leadership. Working with multiple people across the EBMonFHIR project and the BRR, CDS, CQI, and LHS work groups we have completed the analysis that was requested. This analysis suggests many changes (49 element-level changes) to Group Resource to support the function of the proposed ‘Characteristic Resource’ as a CohortDefinition Profile of Group. The full analysis can be found at Changes to Group for CohortDefinition as a GoogleSheet.

The brief summary of the element-level changes is:

1. Add all the elements from the MetadataResource Interface to Group. (23 elements to add)
2. Change Group.type 1..1 code to Extensible binding instead of Required binding.
3. Add Group.combinationMethod 0..1 code 
4. Add Group.combinationThreshold 0..1 positiveInt 
5. Change Group.characteristic.code from 1..1 to 0..1
6. Change Group.characteristic.value[x] from 1..1 to 0..1
7. Change Group.characteristic.value[x] to include canonical() | Expression among the allowed datatypes.
8. Add 20 elements to Group.characteristic

On June 2, the Eligibility Criteria Working Group provided initial responses to ongoing discussions in the Changes to Group for CohortDefinition as a GoogleSheet.

On June 6, the Eligibility Criteria (Cohort Definition efforts) were presented in a Vulcan Accelerator: Real World Data Touchbase meeting.

On June 9, the Eligibility Criteria Working Group provided additional responses to ongoing discussions in the Changes to Group for CohortDefinition as a GoogleSheet. Concepts that can lead to an overall simplification of the underlying model include:

1) Instead of a definition[x] element with 3 datatype options and an optional code/value[x] pair, an overall simpler approach is to have code 1..1 and value[x] 1..1 where the value[x] datatype options are more expansive and the code can include a simple 'is-true' code.

2) Instead of 3 elements (method 0..* CodeableConcept, device 0..1 Reference, and calculatedAs 0..1 Expression), this can be simplified to determinedBy[x] 0..* CodeableConcept | Reference | Expression.

On June 12, the Project Management Group prepared a response to the sets of responses to the Changes to Group for CohortDefinition as a GoogleSheet.

On June 15, the HL7® Biomedical Research and Regulation (BRR) Work Group discussed the responses to the Changes to Group for CohortDefinition as a GoogleSheet, with substantial attention to the "Group Model" which requires all Group Resources to have a type and expects a type to be 1:1 matched with a FHIR Resource type.  This model is necessary for the context of using Group Resource to define computable expressions of Group characteristics to be applied to FHIR resource-type specific paths for evaluation of datasets in FHIR Resources.  We discussed the uses of the "Characteristic Model" for the context of defining computable expressions of characteristics to be applied to non-FHIR-based datasets.  In our current use cases among implementers where this was occurring, the computable expressions were not type-specific or type-dependent.  We did not have a chance to discuss how large the space of non-FHIR-based datasets is for the expected implementations of the "Characteristic Resource" and that may have a substantial influence on whether the community of implementers prefers to work within/around a "Group Model" or develop a distinct "Characteristic Model".  Discussions will continue tomorrow (HEvKA Eligibility Criteria Working Group) and next week (HL7 BRR Work Group).

On June 16, the Eligibility Criteria Working Group reviewed the developments to support an "Endpoint Analysis Plan" through the Research Design Working Group, then discussed how " how large the space of non-FHIR-based datasets is for the expected implementations" of the CohortDefinition concept (whatever we name the Resource or Profile) with a large implementer. With substantial needs to support M11 efforts (International Clinical Trial Protocols represented on FHIR) coordinated with the Vulcan Accelerator, there will a very large number of use cases globally where the Eligibility Criteria to be expressed needs to be expressed in FHIR but the datasets from which the eligibility will be determined are not based in FHIR (and when based in FHIR, are not consistently represented and have varying regional implementation maturity).

On June 20, the HL7® Learning Health Systems (LHS) Work Group received an update on Cohort Definition efforts from May 8 through June 16.

On June 21, the HL7® Clinical Decision Support (CDS) Work Group discussed setting the June 28 CDS Work Group meeting agenda to discuss the CohortDefinition efforts and adjustment of the Group model to support it.

On June 22, the HL7® Biomedical Research and Regulation (BRR) Work Group received an update on Cohort Definition efforts and notified the BRR Work Group of upcoming meetings with:

We have completed an analysis of the changes to Group Resource to support the “Characteristic Resource” model as a Profile. The simplified summary of the changes is to make Group.type an optional element and add a number of new elements to the Group Resource, but re-use existing Group elements wherever possible.

Upcoming meetings you are welcome to join (and encouraged to join if this is of specific importance to you as we meet with FHIR Management in the second meeting below) include:

Friday, June 23 at noon Eastern – Health Evidence Knowledge Accelerator (HEvKA) Eligibility Criteria Working Group meeting, using this Microsoft Teams Meeting link:

Join Microsoft Teams Meeting

Wednesday, June 28 at noon Eastern – HL7 Clinical Decision Support (CDS) Work Group, using this Zoom link:

Join Zoom Meeting https://zoom.us/j/2097018272?pwd=WC9tVWY2NnRTSmVHN21ja3gxeERqZz09

CohortDefinition Profile of Group Proposal 

On June 23, the Eligibility Criteria Working Group drafted the simplified proposal for changes to the Group Resource with:

In many areas within the domain of health data exchange, interoperability requires sharing concepts in machine-interpretable form independent of execution environments.

In many areas within the domain of science data exchange, interoperability requires sharing concepts in machine-interpretable form independent of execution environments.

The “Characteristic Resource” that was proposed, and the broad use cases for sharing cohort definitions (i.e. inclusion and exclusion criteria) across health and science, necessitates being able to support all of:

• Interoperable exchange that could support execution processing FHIR-based datasets
• Interoperable exchange that could support execution processing non-FHIR-based datasets
• Interoperable exchange that could support concept sharing without dependency on an execution environment

We propose to make the following changes to Group StructureDefinition:

1. Add most of the elements from the MetadataResource Interface to Group. (up to 23 elements to add)
2. Change Group.type 1..1 to 0..1
3. Add Group.combinationMethod 0..1 code 
4. Add Group.combinationThreshold 0..1 positiveInt 
5. Change Group.characteristic.value[x] to include uri | Expression among the allowed datatypes.
6. Add 11 elements to Group.characteristic, namely:
   • description 0..1 markdown
   • note 0..* Annotation
   • determinedBy[x] 0..* CodeableConcept | Reference(Device | DeviceMetric) | Expression
   • instances[x] 0..1 Quantity | Range
   • duration[x] 0..1 Duration | Range
   • timeFromEvent 0..* BackboneElement
     • description 0..1 markdown
     • note 0..* Annotation
     • event[x] 0..1 CodeableConcept | Reference() | dateTime
     • quantity 0..1 Quantity
     • range 0..1 Range

On June 28, the HL7® Clinical Decision Support (CDS) Work Group discussed the simplified proposal of changes to Group Resource to support the CohortDefinition Profile. There are two major issues to resolve to harmonize the 'Group for supporting execution processing with FHIR Resources' and the "Group for supporting concept sharing independent from execution processing' use cases. We resolved the first issue by a suggestion to change Group.type 1..1 to 0..1 and add a 'conceptual' code to Group.membership code choices – Group.type would be required except for Group.membership = 'conceptual'. The other major issue is how to handle the timeFromEvent element which is different in pattern to the relativeDateTime extension. We will continue the discussion next week (July 5 meeting) but one suggestion made was to create a TimeFromEvent Datatype.

On June 30, the Eligibility Criteria Working Group identified use cases for the TimeFromEvent Datatype of immediate value outside of the Cohort Definition focus, specifically for the Vulcan Accelerator Schedule of Activities (SOA) Project, and we created the following proposal to change the FHIR specification:

FHIR-41513 - Getting issue details... STATUS

Cohort Definition Updates 2023 Quarter 3

On July 5, the HL7® Clinical Decision Support Work Group discussed FHIR-41513 - Getting issue details... STATUS and proposed a disposition to create a TimeFromEvent Datatype with:

• description 0..1 markdown Human readable description
• note 0..* Annotation Footnotes or explanatory notes
• code 0..1 CodeableConcept Coded representation of the event (Binding to Evidence Variable Event)
• event[x] 0..1 Reference(Any)|canonical(Any)|dateTime The event used as a base point (reference point) in time. Reference(Any) should be a reference to an event resource and is used in patient- or instance-specific contexts. Canonical should be a reference to a canonical resource and is used in definitional contexts.
• offset[x] 0..1 Duration | Range An offset or offset range before (negative values) or after (positive values) the event. Range is limited to time-valued quantities (Durations)
• path 0..1 string Path to the element defining the point in time. Any valid FHIRPath expression

We will discuss and add comments, anticipating a vote for approval in the July 12 meeting.

On July 7, the Eligibility Criteria Working Group reviewed the progress to date regarding Cohort Definition developments (making Group.type optional and creating a TimeFromEvent Datatype), and planned the next set of expected activities include:

1) Demonstration of ability to represent current examples in the proposed Group model with TimeFromEvent Datatype (Monday 8 am Eastern HEvKA Project Management)

2) Vote to approve FHIR-41513 - Getting issue details... STATUS (Wednesday 12 pm Eastern HL7 CDS WG)

3) potentially Jira request for Group changes (Thursday 12 pm Eastern HL7 BRR WG)

On July 11, the HL7® Learning Health Systems (LHS) Work Group (meeting notes) discussed the 2023 - 09 Cohort Definition Track for the September 2023 HL7 FHIR Connectathon and drafted an OMOPonFHIR Track that can provide a related effort to express cohort definitions with Clinical Quality Language (CQL). We also discussed, for potential participants who are guideline developers, that we are providing a pre-conference course at Guidelines International Network Conference: Using technology to efficiently update and adapt guidelines. https://g-i-n.net/wp-content/uploads/2023/04/Computable-Publishing-GIN-2023.docx (or https://abbey.eventsair.com/gin-2023-hybrid-conference/gin-2023-pre-conference-course-registration/Site/Register).

On July 12, the HL7® Clinical Decision Support Work Group discussed FHIR-41513 - Getting issue details... STATUS and decided that we will (through the HL7® Biomedical Research and Regulation (BRR) Work Group meeting tomorrow) create two new FHIR change requests to the FHIR-Infrastructure Group, one to create the TimeFromEvent Datatype and the other to make the changes to Group Resource to support the Cohort Definition efforts.

On July 13, the HL7® Biomedical Research and Regulation (BRR) Work Group created 2 FHIR specification change requests (expected to be reviewed by FHIR Infrastructure Group in their meeting on Monday at 3-5 pm Eastern) with:

FHIR-41582 - Getting issue details... STATUS

FHIR-41583 - Getting issue details... STATUS

On July 14, the Eligibility Criteria Working Group reviewed the M11 Technical Specification (found at M11 Clinical electronic Structured Harmonised Protocol (CeSHarP)) to discuss how it could be represented in FHIR Resources.  We developed a model wherein each major section of the M11 Technical Specification could be used to define a Profile of Composition Resource (or more precisely a profile of EvidenceReport Profile of Composition Resource), and the entry elements for selected sections (subsections) could be mapped to reference specific profiles (e.g. Profiles of ActivityDefinition, EvidenceVariable, Evidence, etc.) and likely the CohortDefinition Profile of Group Resource for the Inclusion and Exclusion criteria sections of the Trial Population section of M11. In consideration of the larger demand across the M11 Technical Specification, and the regular use of HL7 BRR Thursday meetings to address the 'Cohort Definition' efforts, we are considering changing the Friday HEvKA Eligibility Criteria Working Group to take on a larger scope as a Clinical Study Protocol Working Group.

On July 18, the HL7® Learning Health Systems (LHS) Work Group and HL7® Biomedical Research and Regulation (BRR) Work Group discussed the request from the HL7 FHIR Infrastructure Group to document proposed structures and use cases for RelativeTiming Datatypes for both definitional (prospective) and instance (retrospective) purposes, and the desire to have an in-person meeting at the September Work Group Meeting (either Thursday Q3 or Tuesday Q3). The HL7® Biomedical Research and Regulation (BRR) Work Group also discussed the development of 15 Profiles for the M11 Technical Specification currently placed in the Evidence Based Medicine Implementation Guide and the potential to support the Friday HEvKA Clinical Study Protocol Working Group.

On July 19, the HL7® Clinical Decision Support Work Group decided to join the Thursday Q3 meeting with FHIR-I, LHS and BRR at the September HL7 Work Group Meeting.

On July 21, the Eligibility Criteria Working Group reviewed the progress of 15 Profiles for the M11 Technical Specification in the Evidence Based Medicine Implementation Guide, discussed the placement of Vulcan Schedule of Activities efforts within M11 Section 1 Protocol Summary (subsections 1.2 Trial Schema and 1.3 Schedule of Activities), and drafted initial placeholders to use Schedule of Activities Profiles for section.entry content in these subsections.

On July 27, the HL7® Biomedical Research and Regulation (BRR) Work Group discussed the dual interests in the Cohort Definition efforts (Eligibility Criteria Working Group) and the M11 support (Clinical Study Protocol Working Group) and decided to cover the M11 work in the Thursday BRR meetings and keep the Friday HEvKA Eligibility Criteria Working Group meetings mainly focused on Cohort Definition efforts. We also started to summarize the proposed RelativeTime Datatype at https://confluence.hl7.org/display/BRR/RelativeTime+Datatype.

On August 1, the HL7® Biomedical Research and Regulation (BRR) Work Group discussed changing the meeting patterns to use the Thursday noon Eastern BRR meetings for the M11 Clinical electronic Structured Harmonised Protocol (CeSHarP) developments and the Friday noon Eastern HEvKA meetings to continue the Eligibility Criteria/Cohort Definition developments with attention the proposed changes to Group Resource and creation of the RelativeTime Datatype (https://confluence.hl7.org/display/BRR/RelativeTime+Datatype).

On August 3, the HL7® Biomedical Research and Regulation (BRR) Work Group discussed and advanced descriptions for the proposed RelativeTime Datatype (https://confluence.hl7.org/display/BRR/RelativeTime+Datatype) which we will continue to discuss in the HEvKA Eligibility Criteria Working Group meeting, and determined to shift the Thursday BRR meeting agenda to focus on the study protocol, use of ResearchStudy Resource, and how the developing EndpointAnalysisPlan Profile (and other representations of the statistical analysis plan) can be included

On August 4, the Eligibility Criteria Working Group completed the documentation of how to use portion of the summary of the RelativeTime Datatype (https://confluence.hl7.org/display/BRR/RelativeTime+Datatype), specifically: