WORK IN PROGRESS
(this does not perform on the actual resource instance e.g. [base]/Patient/01/$merge)
The target is the remaining patient resource (survivor), the source will be marked as inactive (or in some systems deleted)
The source-patient resource will be updated to add a new link reference to the target-patient resource (link-type=replaced-by), and also update the status to inactive (unless the resource was deleted)
The target-patient resource will be updated to add a new link reference to the source-patient resource (link-type=replaces) - and must be included in the result-patient parameter if used (if the source patient is deleted, then this link is not required)
|source-patient||0..1||ResourceReference||A direct resource reference to the source patient resource (this may include an identifier)|
The Identifier(s) of the source patient resource; all of these identifiers MUST be present in the located resource (in addition to the one included in the resource reference above - if included)
(The purpose of this property is to ensure that the correct source patient resource is selected)
A direct resource reference to the target patient resource
This is the surviving patient resource, the target for the merge
|target-patient-identifier||0..*||Identifier||The Identifier(s) of the target patient resource; all of these identifiers MUST be present in the located resource|
(The purpose of this property is to ensure that the correct source patient resource is selected)
The details of the Patient resource that is expected to be updated to complete with and must have the same patient.id and provided identifiers included.
This resource MUST have the link property included referencing the source patient resource
It will be used to perform an update on the target patient resource
The absence of this parameter the servers should copy all identifiers from the source patient into the target patient, and include the link property (as shown in the example below)
This is often used when properties from the source patient are desired to be included in the target resource
The receiving system may also apply other internal business rules onto the merge which may make the resource different from what is provided here.
If this is set to true then the merge will not be actually performed; an OperationOutcome will be returned in the Parameters response that will indicate that no merge has occurred and may include other diagnostic info if desired, such as the scale of the merge.
e.g. Issue.details.text "Preview only Patient merge - no issues detected"
e.g. Issue.diagnostics "Merge would update: 10 years of content or 120 resources"
The resulting target patient resource will also be returned in the result
The status of the response will be one of:
The Parameters resource will include:
There must be at least 1 source patient/patient-identifier parameter and at least 1 target patient/patient-identifier parameter
The result-patient.id must be the same as the target patient reference (if the patient reference is provided as an input parameter)
If a client needs the server to create a new patient merged from the 2 patient resources, the client should create a new patient record and then call the merge operation to merge each old patient resource into the newly created patient resource.
A server May decide to delete the source record, but this is not defined by the standard merge operation, and if this occurs then the target patient's link property will remain unchanged.
The merge operation will have multiple stages, and some of these may take additional time for processing and thus be done asynchronously
This is a call to the operation (with preview=true) that simply checks for potential errors and warnings, without committing any changes.
This may not be able to capture all possible causes of errors that could be encountered during the processing of the data patching.
The returned Patient resource is a preview only and has not been committed. Hence the version number and last_modified date would be different.
This stage processes the input parameters checking for errors/warnings and begins the changes to the patient resources.
If the system is able to complete the processing of all reference data to the target patient, then it may be complete and no task is required. Otherwise a task for tracking would be created and monitor the progress of the merge.
The rest operation may have returned, and processing is ongoing to patch any other resource that references the source patient to reference the target patient.
This may take a considerable period of time in some systems where the volume of records being updated is large.
|Completed (or failed)||All data processing is complete, and the Task is marked as completed (maybe with errors)|
During the Data Processing stage the patient resource and resources referencing the source patient may be indeterminate until the merge processing operation completes.
Note: Some servers may also update the inactive source patient resource to remove most of the data to make it more clear that the resource should not be used, and the replaced-by link is the key information. Even to the extent of clearing the name and contact details etc.
If the result patient is provided in the parameters to the operation, then it is assumed that the caller has correctly included all the required identifiers desired to be in the target patient (though must include the identifiers specified in the input parameters).
If the result patient is not provided, then the values provided in the request parameters (source-patient.identifier and all source-patient-identifiers) will be copied into the target resource and marked as old.
The server may also migrate other identifiers (and properties) at its discretion, and choose to mark these as old or not.
Note: If an identifier value is masked, then the server will migrate the identifier value correctly, regardless of the masking.
Updating Data that References the source Patient
The merge data processing SHALL update all references that refer to the source patient to reference the target patient.
While updating resources that reference the source patient, ensure that the target patient link value isn't also accidentally updated. (don't make it point at itself)
A Provenance resource SHOULD be created that references the source and target patient resources (or just the target-resource if the source was deleted, or the source patient's old identifier) indicating that the merge has occurred. (example below)
A Provenance resource MAY be created to link all of the resources that referenced the source patient that could then provide information to a potential un-merge operation.
Note: Some resources that have been updated as a result of the merge, such as AuditEvent, Provenance, may have been digitally signed, and this change would invalidate the signature. There may be other reasons impacting the updates that should be considered, further feedback on this specific use case is required.
Post Merge Expectations
Once the patient resources have been merged:
A GET on the old Patient resource ID return: e.g. GET [base]/Patient/pat01
- 200 OK and returns the old Patient which is now marked as inactive, and has the link (replaced-by) populated with the new Patient ID
- 404 not found (when the merge system deleted the resource)
Note: If the system "knows" that the resource was there it would be preferable to return a stub patient 202 as described above.
When performing a SEARCH by the old Patient Resource ID return: e.g. GET [base]/Patient?_id=pat01%active=true (often used as a substitute for direct GET when doing _include for the managing org/general practitioner)
- 200 Ok Bundle with the inactive patient which is marked as inactive and has the link (replaced-by) populated in it (that you'll need to follow to get any further data)
Note: This will have to be checking the ID with active=false/null too for it to success
- 200 Ok Bundle with no patient resource (case where the old patient was deleted)
- 200 Ok Bundle with the target patient resource (with the link to the one from the search) and not include the old patient resource
- 200 Ok Bundle with both the target and old patient resources
Accessing the Patient $everything operation on the source patient resource (now marked as inactive) will return an OperationOutcome and http status of 400 Bad Request. The error message should inform that the patient has been merged and should follow the Patient link to access the $everything content.
Searching content (e.g. Observations) based on patient ID:
- Observation?patient=Patient/pat01 would return a 200 Ok Bundle with no results (as all have been moved to Patient/pat02), an OperationOutcome may be included indicating that the patient was merged into patient xxx
- Observation?patient=Patient/pat02 would return all the data that is referencing pat02, and all the data that was referencing pat01 (which was updated by the merge operation to reference pat02)
Use Case - Polling using search on Observations to get updates:
- Observation?patient=Patient/pat02 (intial client call at start of year, gets all patient obs)
- Observation?patient=Patient/pat02&date=ge2019-03 (client calls at start of march to detect any new patient obs)
- (Patient pat01 is merged into pat02 during April)
- Observation?patient=Patient/pat02&date=ge2019-06 (client calls at start of june to detect any new patient obs, but misses all the pat01 observations prior to June)
- Client needs to check for a provenance record of the merge having taken place to determine that they need to refresh the local content to see the older data
In this case need to detect the patient merge, and perform a fresh retrieval of all content, there is no way for the server to return any error codes in this use case, and may also need to consider notification mechanisms too.
Creating/Updating content (e.g. Observations) that reference the old Patient ID: (feedback on this required)
- 422 Unprocessible Entity with an OperationOutcome indicating that the patient referenced was merged into patient xxx
- 412 Pre-condition failed with an OperationOutcome indicating that the patient referenced was merged into patient xxx (if the client provided an IfMatch header)
- 400 Bad Request with an OperationOutcome indicating that the patient reference did not resolve (existing behavior if the patient resource was deleted)
Merge Notification Mechanisms
The indication that a merge has been completed can be notified through several ways:
- Using FHIR Messaging to invoke the same operation
- An integration engine sending HL7v2 A40 merge message (A18 may also be applicable in backward compatibility modes)
- Directly calling the $merge operation on the dependent systems (requires the system to have both patient resources)
- Client data refresh notification (to be defined, could be triggered by merge, security changes, system migrations, consent changes, etc...)
- Using Subscriptions to detect the merge operation has occurred
- Polling the Merge Provenance resource
- (other non standard notification channels)
These notifications can be sent to other downstream systems, partners, or other applications (including EMPIs). An EMPI could expose the merge operation, and therefore be a notification sender.
Consideration should be taken to ensure that the correct data is acted on.
The downstream systems may not have all identifiers that the notifying system has, the notifier may be configured to know what "types" of identifiers should be propagated to which systems.
When using the identifier parameters (rather than id) you should be using the same assigner (which in the example above would be the PAS or clinical system), this may be configured in the sending notification system, such as an EMPI based on local business rules.
Impact on Subscriptions
Subscriptions on merges are most likely to be used by applications connecting directly to the system. Many use cases could consider using FHIR Messaging (or other messaging e.g. v2 messages) to communicate the merge occurred.
There is a whole discussion on the impact on subscriptions, so won't try and replicate all of that, but instead summarize the issues
- What can be used as triggers for the subscription
- Patient update with new link values
- Provenance(s) as an event
- operation itself as an event (the Task resource, although that may not exist, so just a pre-defined topic)
- AuditEvent as a trigger?
- Will all the data that is patched over to the target patient ID be notified
Additional work needs to be done on this as the Subscription v2 resource matures, we should check with FHIR Infrastructure and other implementers working on the new content (Michael Donnelly, Jenni Syed)
Mapping HL7v2 Merge to FHIR
In the case where there aren't 2 records in the clinical system, then a simple update could be used.
this case is only in the notification handling, not the merge request.
Further consideration on the identifier assigning authority needs to be done.
In HL7v2, the trigger message only reports the results of the operation, and has PID and merged from PID
The HL7v3 in particular looks like they link content, and send a duplicates resolved message interaction.
Q & A
Question: Should support for a simple query parameter form of the operation be permitted? (using the form used in search url|value for identifiers, and the resource references will just include the reference value (no identifiers)
Answer: As this operation is not idempotent it is only defined for POST.
------ up to here ------
Question: How should privacy related meta tags be handled? When reconciling 2 patient resources to be merged, any security or privacy related tags should be included in the resulting target resource with the most sensitive values surviving.
Question: Should a Task resource be included to track things that might take longer to process? Yes.
Question: Should we be explicit about how the identifiers from the source resource should be copied into the target resource (and if they should be marked with use=old), the system will not be expected to make these decisions in the operation, the caller of the operation will make those decisions, which are defined in the result-patient resource. For a case where this was an old MRN, we recommend that these also remain in the target resource so that if content comes in (such as a lab result) with the old MRN on it, then it can be attached to the appropriate.
Question: What should happen if the 2 patient resources are already merged, an error code?
Question: Should the merge operation update/correct/change all the resources that reference the patient to link to the resulting patient resource. E.g. Any Observations referencing Patient/01 will be changed to Patient/02
Question: Provenance/AuditEvent actions - any security issues here?
After some discussion, the Provenance is the preferred resource to track this content, as will likely remain with the clinical data retention, however the AuditEvents may be cleansed periodically.
Question: Will the spec also describe an $unmerge style operation, or will that be essentially a manual activity on the API using the Provenance/AuditEvents to determine context.
Question: Should the updating process be able to be performed synchronously, or async? - Prefer to support Async given the potential duration the processing could take for some long term records
Note: The preferAsync header is intended for large data results, not long data processing results.
Question: Should there be a mode parameter to get an estimate on what would be impacted by the merge - an aggregate on how many resources on each side etc.
Question: Should we be able to merge 3 patients into 1? Suggest keep it simple at 1 into 1 for now.
Question: During the processing of data for a merge operation (which can take take some time), any query for a resource that references either source or target patient could be returning partial content, and operation outcome could be returned in the search bundle indicating this - it's really a warning or information case. Resources like medications or allergies could be significant in this situation.
Question: What should happen to the source patient post merge? Should it be deleted, marked as inactive, hidden from searches? What impact on subscriptions and lists?
Question: Should there be a pre-merge or dry run merge operation to check the scale of the operation, this could be useful to check the scale. If the results were to be reported, it can be an estimate, and doesn't really need to be broken down into resource types, its just to help advise age of the record. This preview parameter satisfies this need.
Question: Should a search no the old MRN identifier return both the old patient and target patient resources, or just the target patient resource? How do we filter out the old resource? Rely on the inactive flag?
Question: Should the merge itself mark the source as inactive? Suggest that this is a good idea.
Question: Should other resources, such as Patient Accounts, be merged too? Or is that a supplementary exercise? Could they be tagged with Task resources for review?
Other considerations could be claims to be withdrawn, communications in process to be cancelled.
Question: Should anything happen with SMART App Launch tokens that have the Patient ID in them? If they were revoked, that would be the cleanest, as apps need to handle this and would just challenge for a new token, which would get the new value
Question: If the 2 patient resources had links to each other, should those be removed?
Question: How would a merge be reflected in subscriptions?
Question: Should the merge operation also be reflected in the Person Resource - expect that would just update as per any other resource type with a reference to the patient resource.
Question: How can we merge multiple patients into 1 record? This is typically done using multiple calls to the operation. e.g. merge: A→C, merge: B→C
Question: How do Patient (or other users) accounts with access be changed (these aren't fhir resources)
Question: What happens in a Rollback scenario? When can that occur?
Question: Should searches on other resources include an OperationOutcome in the bundle that indicates that there is a processing action in progress that could effect the results.
Question: (During join PC/PA session) Amit Popat questions the optional nature of the source-patient and target-patient resource references, and encourages always requiring the actual resource.id be specified. The reason for the additional identifiers is to ensure that the correct resource is selected, as with v2 we included additional demographic data to ensure a safe match.
Question: Post merge when committing (PUT/POST) a resource (e.g. Observation) that references the old inactive patient, should it succeed, re-target the resource, or return a failure?
Question: Post merge when searching for an observation using the old patient ID, should it be found? - I think not, but possibly return an op outcome, but this needs discussion
Question: Need to have some mechanism for notifying that a client needs to "refresh" its view of a patient record, as something has significantly changed, e.g. Merge occurred, security permissions changed, policy updated, historic data imported, backup restored etc. End result is that the client needs to discard its local cache, and re-read it all.
Example $merge operation parameters resource
Any errors will be reported with an OperationOutcome resource and could include
|err: Same resource||The Source and Target matching resulted in the same FHIR Patient resource, likely already merged.||Bad Request|
|err: Missing Source Parameters||There are no source patient parameters, please include either a source-patient, source-patient-identifier parameter (or both)||Bad Request|
|err: Missing Target Parameters||There are no target patient parameters, please include either a target-patient, target-patient-identifier parameter (or both)||Bad Request|
|err: Target Patient Id mismatch|
The target patient id does not match the patient id in the result-patient resource
(should we permit this to create a new resource?)
|err: Source Patient not found||The source patient was not found based on the provided parameters||Bad Request|
|err: Target Patient not found||The target patient was not found based on the provided parameters||Bad Request|
|err: Target/Source not duplicates|
Attempt to merge 2 records that are known to not be duplicates of each other.
(Previous manual marking of the resources was done, and will need to be removed before retrying)
|422 Unprocessable Entity|
|err: Target patient already merged||The Target patient resource was previously merged into another patient record||422 Unprocessable Entity|
|err: Target patient inactive||The Target patient resource is marked as inactive||Bad Request|
|info: Target Patient updated||Additional notes on what happened to the target patient resource on update, such as if fields weren't updated as requested due to internal business rules etc||422 Unprocessable Entity|
|info: Update summary||Other notes that are included reporting on what changed, such as how many resources were/may be effected|
|warn: Recommend reverse merge||The source resource is much larger than the target resource (in terms of resources that reference it) and recommend that the merge occur in the other direction|
The FHIR Request Message should be a Bundle with:
The Messaging header
The focus of the message will be the Parameters resource
|Parameters||1..1||The same Parameters object that would be passed to the $merge operation|
Brian: This is not required, as it is already in the parameters resource, however should note that this is required in the parameters object when used in messaging
Source Patient resource (may not be complete, but should have enough to be able to identify the source record)
This is the details of the patient resource that will be marked as inactive after the merge
Target Patient resource (may not be complete, but should have enough to be able to identify the target record)
This is the details patient resource that will remain active after the merge operation is complete
Note: If the target patient resource has an id, then the resulting patient.id value must be either blank, or the same resource Id.
Operationally, on the receiving system, the end result will be that the id on the patient result and target patient are the same.
The FHIR Response Message should be a bundle with:
The Messaging header
The focus of the message will be the Parameters resource
|Parameters||1..1||The parameters resource that was included in the request|
|OperationOutcome||1..1||The results of the merge operation|
The resulting patient resource from the merge operation
(required when the result was a successful operation)
|AuditEvent||0..1||An operation event that includes the full details of the operation, including references to all the resources that were updated as a result of the merge|