MOSAIQ Real-World Testing Results
2023 Performance Year
Download results or visit MOSAIQ Real-World Testing
Performance Year: 2023
Plan Report ID Number: For DRUMMOND use
Developer Name: Elekta
Product Name(s): MOSAIQ
Version Number(s): 2.86
Certified Health IT Product List (CHPL) Product Number(s):
| MOSAIQ 2.86 | 15.04.04.1420.MOSA.02.05.1.221220 |
Developer Real World Testing Page URL: https://www.elekta.com/products/oncology-informatics/mosaiq-real-world-testing/
Developer Real World Testing Results Report Page URL: https://www.elekta.com/products/oncology-informatics/mosaiq-real-world-testing-results-2023/
Changes to Original Plan
| Summary of Change | Reason | Impact |
|---|---|---|
| For Criterion §170.315(f)(1), the methodology used to collect data was initially planned to be the analysis of interface logs. After development work was planned, it was decided the best mechanism to collect this data was through SQL queries. | This decision was made as it would be the most efficient and accurate way to access the data for the sites selected. Utilizing SQL queries provided insight into the individual sites use. | Minimal impact as the data was still collected. |
| Elekta intended to begin conducting real world testing on July 1, 2023. Elekta began conducting real world testing on Nov 15, 2023. | Due to resource constraints and difficulty confirming dates/ times with participating sites, Elekta was unable to begin real world testing within timeframe initially specified. | Seemingly no impact, as Elekta was still able to conduct sessions of real- world testing with participating sites and completed them prior to January 31, 2024. |
Summary of Testing Methods and Key Findings
To conduct real world testing, Elekta utilized multiple mechanisms, including the following:
- Running Existing Reports in MOSAIQ : Elekta asked participating sites to run an existing report, “Promoting Interoperability Report”, on their local version of MOSAIQ for all clinic providers.
- SQL Queries : Elekta asked participating clinics for access to their SQL servers to run multiple queries for various data extracts.
- Visual Inspection : Elekta imported CCDA’s with privacy flags and visually checked those users with privacy viewing capabilities were able to view a CCDA, and those without privacy viewing capabilities were not.
- Interface logs : Elekta inspected numerous interface logs to analyze the rate of use for various certified functionalities.
Elekta worked with participating sites to collect the appropriate data. Elekta selected a three- month timeframe, October 1st to December 31st, 2023 for data analysis. The SQL queries, Reports, and Interface logs analyzed this three- month timeframe only to avoid any data outliers.
Care Settings(s)
Elekta conducted four sessions of real- world testing. All sessions were conducted with practices in radiation oncology and medical oncology settings. Elekta successfully tested the following versions of MOSAIQ v2.86
Metrics and Outcomes
Criterion: §170.315(h)(1) Receive health summary using DIRECT
| Relied Upon Software | Secure Exchange Solutions SES Direct |
|---|---|
| Measurement/ Metric | Log files were used to track the frequency of DIRECT to receive health information |
| Expected Outcome(s) | Demonstrates real-world use of DIRECT messaging to receive clinical summaries sent by referring providers. |
| Outcomes | The average number of received clinical summaries via DIRECT Messaging was ~195 messages. This number represents the average number of clinical summaries received for all participating sites during the period of October 1st-December 31st of 2023. Elekta believes the DIRECT Messaging functionality demonstrated fully compliant real- world use. |
Criterion: §170.315(b)(2) Create a single reconciled list of medications, medication allergies, or problems from received summary and active patient record
| Measurement/ Metric | Reports were used to calculate the number of times that medications, problems, and allergies were incorporated into a medical record compared to the number of referrals into a clinic |
|---|---|
| Expected Outcome | It is expected that a high percentage of new patients and incoming referrals with clinical summaries are incorporated into the new patient records. The test will report the frequency of use of MOSAIQ to receive, reconcile, and incorporate clinical summaries. |
| Outcomes | On average, providers had reconciled lists of medications, medication allergies, or problems from received summary and active patient records zero times. Because we did not have any active users of this functionality, we validated this functionality internally using three test patients in Elekta’s test environment. All three tests were successful with no errors. |
| Challenges Encountered (if applicable) | Elekta utilized the Promoting Interoperability measure report within MOSAIQ. Providers were not asked during which 90- day period they collected their Promoting Interoperability data. Elekta assumes the 90- day period selected for all providers to run the report, October 1st – December 31st, was not the same period providers used in submission of their PI data to CMS. Alternatively, the providers surveyed may typically not use MOSAIQ for their referral workflow and may instead lean on additional software. Given this, Elekta believes the lack of data or reconciliation efforts is not representative of the reliability of the functionality but instead the use level. |
Criterion: §170.315(e)(1) View, download and transmit to a third party
| Relied Upon Software | Medfusion by Medfusion, Inc. |
|---|---|
| Measurement/ Metric | SQL Queries were used to track the frequency of patient views, downloads, and transmissions to third parties |
| Expected Outcome | Number of patient views, downloads, and transmissions |
| Outcomes | Elekta calculated the average rates, given the available sites, for view, download, and transmit to a third party during the period of October 1st to December 31st. The averages for the participating sites were as follows
|
Criterion: §170.315(b)(8) Security tags - summary of care – receive
| Measurement/ Metric | SQL queries were used to ascertain the frequency of clinical summaries received with a privacy indicator. A test CCDA with a privacy indicator was imported by an a) unauthorized user and b) an authorized user. |
|---|---|
| Expected Outcome | Numbers that reflect the frequency of received clinical summaries with a privacy indicator and a positive test of restriction of viewing rights of a summary with a privacy indicator to authorized users. |
| Outcomes | Among the surveyed sites, the average number of CCDA’s received with a security tag was zero, and therefore the number of positive tests of restriction of viewing rights was zero. |
| Challenges Encountered (if applicable) | Elekta believes this may not be a functionality commonly used by providers within MOSAIQ hence the lack of available data. Elekta believes the lack of data is not representative of the reliability of the functionality but instead the use level. |
Criterion: §170.315(b)(3) Electronic Prescribing
| Relied Upon Software | First DataBank Medknowledge Framework 4.2 Surescripts ePrescribing Dr. First EPCS Gold |
|---|---|
| Measurement/ Metric | Reports were used to determine the frequency of ePrescribing messages and the number of errors during transmission. |
| Expected Outcome | Clinicians will be able to manage outpatient prescriptions with local pharmacies with few errors. Less than 1% of errors are expected. |
| Outcomes | The e-prescription rate of success for transmission was 100%. On average providers transmitted ~1756 e-prescriptions, for which there were no errors. Therefore, the rate of failure of transmission was 0. |
Criterion: §170.315(f)(1) Transmission to Immunization Registries
| Measurement/ Metric | SQL queries were used to determine the use of the immunization interface message transmissions. |
|---|---|
| Expected Outcome | It is expected that clinics can transmit their immunization information successfully. Error rates will be tracked. |
| Outcomes | Providers on average did not utilize the immunization interface. The average number of immunizations sent was 0. |
| Challenges Encountered (if applicable) | Elekta believes that the immunization interface is not being used because Elekta users are predominantly focused on oncology care and do not typically send immunizations. Elekta validated successfully this functionality to send immunization for five different patients in a test environment. This test was validated once as part of Elekta’s formal validation process during release. There were no errors reported. No additional validation tests were executed. Elekta believes the lack of data is not representative of the reliability of the functionality but instead the use level. |
Criterion: §170.315(g)(7) Application Access- Patient Selection
| Measurement/ Metric | Interface logs were reviewed to determine the number of API applications registered and the number of patient selection API transactions over time. |
|---|---|
| Expected Outcome | API applications will be able to receive properly-authorized patient ID’s. |
| Outcomes | Currently, there are no applications utilizing Elekta’s available FHIR API. Therefore, there have been no applications able to receive properly authorized patient ID’s. |
| Challenges Encountered (if applicable) | While Elekta’s FHIR API is available for use to any vendor, there has been little interest from application developers. Elekta believes this to be because Elekta is an oncology specific EHR and does not have a large margin of market share. To date, no applications have progressed beyond Elekta’s sandbox environment. However, Elekta verified this functionality as part of internal testing against one sample API application. This test was validated once as part of Elekta’s formal validation process during release. There were no errors reported. No additional validation tests were executed. Elekta believes that while the API is not being used, it is still functioning compliantly. |
Criterion: §170.315(g)(8) Application access- data category request
| Relied Upon Software | Firely |
|---|---|
| Measurement/ Metric | Interface logs were reviewed to measure the volume of patient laboratory results retrievals. |
| Expected Outcome | The patient lab results will be retrieved in full and without error. |
| Outcomes | No patient lab results were retrieved. Currently there are no applications utilizing our available API. Therefore, there have been no data calls to Elekta’s API. |
| Challenges Encountered (if applicable) | As mentioned above, there are currently no applications making use of Elekta’s FHIR API. Elekta believes this is because Elekta is an oncology specific EHR and does not have a large margin of market share. Elekta believes that while the API is not being used, it is still functioning compliantly. Applicable criterion for the surveyed sites was changed to §170.315(g)(10)- Standardized API for patient and population services during the selected time frame, however, the Measurement, outcomes and challenges remained same. |
Criterion: §170.315(h)(1) Send health summary using DIRECT
| Relied Upon Software | Secure Exchange Solutions SES Direct |
|---|---|
| Measurement/ Metric | Log files were used to track the frequency of DIRECT to send health information. |
| Expected Outcome | It is expected that health care providers will receive clinical summaries using the DIRECT protocol. |
| Outcomes | The average number of clinical summaries sent via DIRECT Messaging was 11 messages. This number represents the average number of clinical summaries sent for all participating sites during the period of October 1st-December 31st of 2023. Elekta believes health care providers are compliantly receiving clinical summaries using the DIRECT protocol. |
Criterion: §170.315(b)(1) Transitions of Care
| Measurement/ Metric | SQL queries were used to determine the number of clinical summaries sent to providers. |
|---|---|
| Expected Outcome | It is expected that a high percentage of oncology treatment results in clinical summaries sent to referring providers. |
| Outcomes | Elekta utilized a SQL query and looked at data from October 1st – December 31st. During this three- month period, an average of 195 CCDAs were sent to a direct mail address. Elekta believes MOSAIQ is functioning compliantly given the general use of the functionality. |
Criterion: §170.315(g)(9) Application Access- all data request
| Relied Upon Software | Firely |
|---|---|
| Measurement/ Metric | Logs were used to measure the volume of all data requests |
| Expected Outcome | API applications will be able to request and receive patient’s properly authorized clinical summary data. Error rates will be tracked. |
| Outcomes | Currently, Elekta has no applications utilizing the available API. Therefore, no API applications were able to request and receive patient’s properly authorized clinical summary data. |
| Challenges Encountered (if applicable) | As mentioned above, there are currently no applications making use of Elekta’s API. Elekta believes this is because Elekta is an oncology specific EHR and does not have a large margin of market share. Elekta validated this functionality for three patients as part of internal testing. This test was validated once as part of Elekta’s formal validation process during release. There were no errors reported. No additional validation tests were executed. Elekta believes that while the API is not being used, it is still functioning compliantly. |
Criterion: §170.315(b)(7) Security tags- summary of care- send
| Measurement/ Metric | Test CCDA’s with a privacy indicator were created by a) an unauthorized user and b) an authorized user and visually analyzed. |
|---|---|
| Expected Outcome | All CCDA’s designated as “private” by the user will be constrained with the Confidentiality Code in accordance with the standard as specified in DS4P R1. |
| Outcomes | For all participating sites, CCDA’s with a privacy indicator were imported into the site’s MOSAIQ. Users without privacy viewing rights were unable to see the contents of the CCDA. Users with privacy viewing rights were able to see and import the contents of the CCDA into the patient chart. |
Criterion: §170.315(b)(6) Data export
| Measurement/ Metric | SQL queries were used to measure the number of batch export instances and CCDA’s per batch. |
|---|---|
| Expected Outcome | Users will be able to batch export clinical summaries for multiple patients for use by external systems. Error rates will be tracked. |
| Outcomes | Among the surveyed sites, the average number of batch export instances was ~ 389. The average number of batch export errors was ~12 |
| Challenges Encountered (if applicable) | For one site, we were unable to report on their SQL data for this specific measure. The data produced contained errors. Elekta assumes the data for the available sites to be free from error and therefore reliable. |
Key Milestones
| Key Milestone | Care Setting | Date/ Timeframe |
|---|---|---|
| Finalization of SQL Queries for Real World Testing | June 19, 2023 | |
| Real World Testing Conducted with Site #1 | Radiation Oncology and Medical Oncology | Nov 15, 2023 |
| Real World Testing Conducted with Site #2 | Radiation Oncology and Medical Oncology | Dec 5, 2023 |
| Real World Testing Conducted with Site #3 | Radiation Oncology and Medical Oncology | Jan 10, 2024 |
| Real World Testing Conducted with Site #4 | Radiation Oncology and Medical Oncology | Jan 15, 2024 |
| Final Results collected and aggregated | Jan 15, 2024 |
Attestation
The Real World Testing Results above are complete with all required elements. All information included in these results is up to date and fully addresses the health IT developer’s Real World Testing requirements.
Authorized Representative: Ken Hotz
Authorized Representative Email: ken.hotz@elekta.com
Authorized Representative Phone: +1 408 306 7744
Date: January 19, 2024