Skip to end of metadata
Go to start of metadata

This page outlines the overall requirements for the UTG project organized into the following sections:

UTG Versioning Planning and Discussions

Needed Tool Packages

Re-Use and Re-Purposing of Current Tooling

Interactions and Competing Changes

Current Strawman for Voting/Consensus Decisions on Proposals

Migration Recommendations

Rendered Pages for Published Content Page

"Blue Sky" Notions from WGMs, demos, and general 'noodling'

Requirements from TSC Demo

Needed Tool Packages

Only a few components are currently available, or can be made available easily by tweaking or re-purposing existing tools; most of the tooling to support or enable the identified components must be new.  The primary tools that are needed are:

Centralized HL7 Terminology Persistence Store

There is currently no single store for all of the HL7 terminology.  Version 2 is maintained in a Microsoft Access database, CDA terminology is maintained in Trifolia, V3 terminology is maintained in the coremif and MSAccess java-based tooling used in the publication preparation processes, and FHIR terminology is maintained in FHIR codeSystem and valueSet resources in the FHIR database maintained by Grahame Grieve.

Currently the most comprehensive set of terminology is in the FHIR database, since it has all the FHIR-specific terminology, plus the imported V2 terminology that is used, and a significant amount of the V3 and CDA terminology as well.  This is the closest thing HL7 currntly has to a single source of truth.  But it is a *copy* of the V2, CDA, and V3 terminology, and as such currently suffers from subsetting, currency, and import error issues.

A Unified Terminology Governance process will require a unified persistence store of all terminology published in HL7 Standards.

Terminology Subset Extraction for Viewing and Publishing

In order to be of use to HL7 and its community of users, terminology in the central store must be able to be viewed and extracted for excerpt inclusion in published Implementation Guides, other Standards, and Harmonization Proposals.  Currently, each product line (V2, V3, CDA, FHIR) has its own tools and mechanisms to extract terminology for publishing and for creation of artifacts for change proposals.

Tooling will be required to accomplish this against the central store.  At the current time, there are several value set editors and code system content editors in various stages of capability and usability for the FHIR database, but significant investment would have to be made to bring these up to what would be required for the other product lines.

At this time, the V2 Chapter 2C format generation is done with a large number of MSAccess and VB scripts, and run manually by Frank Oemig.  This does not produce the final publication format, which must be manually edited (a 850 page Word document) for a v2 ballot.  In addition, it is widely recognized that Word and PDF are exceptionally poor vehicles for terminology access and publication.   For Version 2, a whole new publication and accessibility format must also be designed, or it must be subsumed into one ofr the other families.

Tooling Requirement: Chapter 2C output format generation from the central Store.

Design and Tooling Requirement: Other output/accessibility format for V2 terminology


V3 ballot generation is done using RoseTree and a large collection of publishing tools run manually by Ted Klein and Lynn Laakso..  These already have bugs and issues, and can no longer be easily maintained as Woody (the original author) is no longer actively participating.  Although FHIR has tools that can import V3 content from the coremif (the format which the V3 terminology is made available in) there is no current capability to extract FHIR content into the coremif format.  In addition, there are a few V3 terminology components which are not part of the FHIR Core specification resources, and thus some extensions or some mappings are required in order to enable any lossless extraction from FHIR resources to V3 coremifs consumable by the ballot and Normative Edition generation tools.  Note there are another existing tools, such as the RMIM designer, which also need the coremif format for the terminology.

Tooling Requirement: Output of a complete and correct coremif format for V3 terminology

Design and Tooling Requirement: Ability to persist 100% of the V3 terminology content


CDA Implementation guides are largely assembled manually by the workgroups; there is no specific tooling requirement for generation of a specific format for the CDA terminology published by HL7.  However we have seen errors rife in the content, likely due to the fact that these are manually assembled.  The access to CDA terminology is through Trifolia, which is widely used in the CDA community.   At the current time, the content is manually extracted from the published implementation guides and entered into Trifolia for accessibility.

Tooling Requirement: Capability of producing an import format for Trifolia.  Note that at this time it is unknown if Trifolia has such an import capability at all.

Design and Tooling Requirement: Publication/Accessibility format for CDA terminology,


FHIR ballot generation is done by a set of tools run by Grahame Grieve and the other FHIR core team member.  This depends upon the terminology content being in FHIR valueSet and codeSystem resources.

Tooling Requirement: Capability of producing FHIR valueSet and codeSystem resources from the persistence Store.

Proposal and Endorsement Maintenance

Currently, change proposals are either Word documents or SVN or gForge tickets.  The proposed process treats all proposals for changes to HL7 Terminology as persisted workflow objects.  A database for these permits tools to be built for their management and also reports so HL7 management and WG leadership can get a better handle on changes that they are involved with.

The core of the proposed process is centered around shared asynchronous web-accessible proposals that can be viewed by anyone at any time, and comments and endorsements can be attached to them.  The way that Tickets are managed currently in gForge is a good model for this.  The Endorsements can be a portion of the format of the proposal itself, and can thus be tracked, accessed, and edited in the same way and using the same tooling.  Either the current tracker or JIRA are well suited to managing the workflow around the intended asynchronous distributed process.

Design and Tooling Requirement: Format design and tooling for managing proposals and endorsements is needed.

Proposal Viewing/Creating/Editing

Right now, a variety of tools (mostly heavy use of text editors) are used to create requests for changes to the HL7 Terminology.  A very large amount of resource is used to validate these, correct errors and incompleteness, and prepare them to be reviewed by a consensus group and able to be consumed by tooling which can apply the approved changes to the terminology.   This is a hugely manual task currently.

The proposed process intends that a new set of tooling be developed which allows a submitter to actually create the terminology that he or she wishes as a result of their proposal, rather than describing it in a way that can be thought of as a delta to the existing content.  This involves an extraction of a subset of the Terminology from the Store, editing it to produce a view of what is desired as the end state, and a save of the technical components of this as a delta that can be applied by tooling with minimal manual intervention to apply the change (if it is endorsed).  This will thus bypass the most laborious and expensive part of the current harmonization processes.

Design and Tooling Requirement: Tool to extract from the Store an editable set of content (Code System content, Value Set definitions, Concept Domains, Tables, and Bindings), a tool to edited the extract in a WYSIWYG manner, and a tool to create a delta which can be applied to the Store to instantiate the change.

Change Proposals Support

Proposal State Workflow Management

The proposed process is enabled by the central object being the Proposal, which goes through a number of states from New through Consensus Achieved and final processing.  These State transitions must be mediated by tooling as there re restrictions in the path from State to State, and some transitions are manually performed by the Terminology Curator based on documented rules

Design and Tooling Requirement: Design of State Model and the rules for State Transitions, design of the persistance for the States in the Proposal Tracking tooling, and tools to enable submitters, endorsers, commenters, and the Terminology Curator to effectively manage the State Transitions.

Terminology Curator Role

Maintenance and publishing of Terminology in a high quality way is widely recognized as a complex undertaking.  The proposed process and tools to support it was designed to minimize the need of a dedicated resource to oversee the process, address issues and problems, and manually fill in areas where tooling is insufficient or impossible to create (such as value judgements on whether or not descriptions are of high enough quality for HL7 Standards).  With sufficient tooling support, the current amount of change to the HL7 Terminology across all four product families should be able to be limited to a part-time resource with about the same time/effort requirement as consumed by the current V3/V2 Harmonization process.  HL7 has experience in depending upon strictly volunteer (non-funded) resources for terminology change control and maintenance, and the experiences were not positive.

Requirement: Support in HL7 budget for an ongoing funded support for this role.

Applying Approved Proposals

At the end of all the processes, there will be a queue of proposals whose content must be applied to the persistent Store to update the Terminology and instantiate the requested changes.  The proposed process intends to minimize the resource requirement for this by having the editing tools create as an artifact a delta which can be applied to the Store to produce the desired result.  In the ideal case, this would be completely automated and just be a matter of running a simple XML tool to apply the delta.  Real world experience has informed us that things break in unforeseen ways, so there must be an oversight and validation component to this process so that things do not go haywire.

Tooling Requirement: Tools to apply the delta created by the editing package, and to visualize the result before committing it to the Store.  Capability of rollback if things do go haywire, and support for the QA review with editing changes and reapplication of the delta is also needed.  Finally, tooling support for the recordkeeping audit trails for the changes made to the Store for full documentation are required.

Re-Use and Re-Purposing of Current Tooling

There are a number of tools being used currently in the different product families that may find a home in the new process, or may be modified to provide value in the new process.  The primary area of examination at the time of this writing is the FHIR value set editing tools, the FHIR Terminology Services and Persistence tools, and the gForge tracker system.

It is felt that the current V3 tools (RoseTree, the java tools for processing VML, the ANT scripts for processing the VMIF, the MSAccess RIM database, etc) have become too obsolete and would require modifications of too large a scope and cost to be appropriate for central use in the new processes.

Interactions and Competing Changes

Under ANSI rules, the final decisions on HL7 Standards are by the balloting process.  At the current time, terminology content is integrated within the ballot processes, and part of them.   Negative comments are made on content, and reconciliation not infrequently results in changes to previously vetted and harmonization-approved changes.  Having two independent consensus processes for content updates is a recipe for trouble; either the two processes must be coordinated in one way, or there must be only a single process.

Single Process

A single process would involve HL7 no longer placing terminology content in a balloted Standard explicitly; the terminology content would have a reference in the ballot document, but the actual codes would be accessed through a publishing/accessibility capability at the HL7 website.  This would involve viewing, searching, and downloading the content.  Note this represents a significant change to the current HL7 way of doing things.

Coordinated Processes

If the ballot process must be the final arbiter for terminology changes per ANSI requirements, then this governance process must be coordinated with the ballot process.  Preparation of a ballot would involve taking a "snapshot" of the current state of the terminology content required to be published in the ballot.  This is necessary because the suggested process is continuous, and not tied to any product line ballot calendar.  if, as part of ballot comment reconciliation, changes are required to the terminology content, then they are submitted in the same fashion as any other change to the beginning of this process.  Such proposals would be marked as resulting from ballot reconciliation, and would likely have different endorsement rules; perhaps automatically moved to the CONSENSUS: PASS state.  This would the easiest and cheapest means of coordinating the ballot processes with this unified terminology governance process.

Current Strawman for Voting/Consensus Decisions on Proposals

There is a VERY ROUGH draft of the initial ideas for the voting/decision consensus process laid out in the past.   It is shown in this section for review and additional enhancement.  The basic notion is that it is a point system, where voters have their votes weighted by being worth one or more points.  Once a decisions is made (up or down) by the necessary quorum being met plus the deciding number of affirmative or negative points, the endorsement period closes.  The period may also close at the expiry of the time period allotted when quorum has not been reached.  At the expiry time, if no yes/no decision is reached due to controversial votes by the controversy number rules, then the proposal endorsement period ends with a status of controversial.

Additional Consensus Process Voting Design Details

Principles

  • Some votes have more weight (by constituency and by reputation)
  • Some votes are associated with specific communities
  • The Open voting process means that all voters who are not specially recognized in roles defined below have an equal unweighted vote.
  • Avoid preponderance of interest effects from any group
  • Ensure broad participation where necessary
  • Open

Details and Voting Levels

Items in red font require additional discussion. 

  • On the 3/14/2017 call, we changed the levels to be lower, because looking at example Use Case, it seemed that the levels originally posited in earlier versions of this document were too high, and would likely result in failure to attain the minimum necessary votes on most proposals.   It was also noted that many of the FHIR voting have huge numbers of voters and commenters (eg on Zulip) whereas for V3, CDA, and v2 voting items, and current harmonization, there is a struggle to get more than a dozen participants.  More discussion is needed here, and perhaps a survey to see how many folks could be expected in the new process.   The numbers below have been adjusted to these lower levels.
  • Oversight groups:
    • Vocabulary = Vocab work group
    • Product line: MnM for V3, FMG for FHIR, InM for v2, SD for CDA, CIMI for CIMI
    • International: International Council
  • Proposals must all be vetted NB: does this mean review and vote/comment?  Or must the process have some preliminary vetting like what is currently done by tech review for Harmonizatoin?  This is not currently part of the process; a tool-driven technical validation followed by a brief review by the Terminology Curator is all that is done for a submission before it is released to the community for consensus acquisition. by voters from Vocabulary, at least one product line (plus all other relevant product lines NB: how to implement this?  Must submitters identifiy the product lines involved?  What if they are wrong, or intentionally leave out those that they know will push back?) and International unless realm-specific
  • Oversight groups must designate individuals who may make votes for their “category” What is a 'category'?  
    • Oversight groups may designate individuals with significant experience/expertise (generally 5+ years experience in vetting proposals) as super-voters, having double-weight votes.  We need to defined the process for identifying such super-voters, and approving them by the oversight groups, and who they vote on behalf of?
    • Affiliate members are automatically members for their respective countries.  The IC may designate a process to recognize “international” voters from non-affiliates and non-designated affiliate voters for the specific reason that they are expected to have a cross-country perspective. How will be track such folks?  Will the tooling need a separate lookup with all the relevant UTG metatdata from designated individuals, where anyone not in that list gets a default set derived from their membership database and list serve subscription status?
  • Each oversight group needs 3 net points, Controversial = more than 30% of total votes are negative. what if there are just a couple positive, and the rest abstentions, where there are more than 60% say abstentions?   ie are there any other criteria for controversial other than a lot of negatives?  Or should that be the defining criteria for controversial?
    • Proposals may be designated by the terminology curator as “pro forma” for one or more of the oversight groups which reduces total votes needed to 2
    • Proposals may be designated by the terminology curator as “realm specific” where weighting is changed for the international and US-specific votes NB how to change?  Remove Intl as an oversight group?  other changes?
  • Each proposal needs a net of 9 non-abstention points from non-oversight groups, plus the points from the oversight groups (18 for most proposals) (number of up/down votes will determine consensus result).  Pro-forma proposals may be reduced to 5 votes
    • Consensus: approval = 70% or more of non-abstentions positive; rejection = 70% or more negative; controversial otherwise (over 30% negative but less than 70%);
    • Consensus: controversial should also be judged if too many folks abstain; ie if the time expires with insufficient non-abstention votes, then rather than rejection, should it go into the controversial queue, or should there be some other process to cover the fact that the lack of synchronous discussion may mean too few folks feel they have enough familiarity to cast any vote.
  • First time voters (those who have not voted for longer than 1 month or on fewer than 3 proposals) can only make up 30% of the total votes.  If their percentage of the tally is larger, their votes will be prorated down to only 30% of the total
  • Multiple voters from the same organization will have their total votes prorated down to a maximum of 2.
  • New additional idea: when a person logs into the system to review a proposal and cast a vote, they log in with the objective to cast a vote on behalf of a particular group.  The system validates that they are a member of that group (subscribed to the mailing list for a certain minimum period of time?).  So each vote rather than being one person/one vote is one person-group/one vote.  We need to discuss how this would affect this algorithm.  Does this imply that the 'category' above is the WG identified here?  

Migration Recommendations

There will be a need to migrate from the current tooling and processes used for Terminology Governance to this new process.   It is unlikely to be impossible and is inadvisable to replace all product family processes wholesale in a single update event.  Therefore a migration strategy from what is done currently to arrive eventually at where we wish to be is a critical planning component for this process recommendation.   Details of the migration, including specific tool recommendations, will have to await subsequent projects that compile gap analysis and requirements evaluations for suggested tools and solutions.

  • No labels