Kaiku Health
Real-World Testing Results
2024 Performance Year
Download results or visit Kaiku Real-World Testing
General Information
Performance Year: 2024
Plan Report ID Number: For DRUMMOND use
Developer Name: Elekta Inc.
Product Name: Kaiku Health
Version Number: 8
CHPL Product Number: 15.04.04.1420.Kaik.08.01.1.240530
Developer RWT Page URL: https://www.elekta.com/products/oncology-informatics/kaiku-real-world-testing
Changes to Original Plan
| Summary of change | Reason | Impact |
|---|---|---|
| Real World Test Plan 2024 for Kaiku Health indicated version 7, CHPL Product Number 15.04.04.1420.Kaik.07.00.1.211015. Real World Test Results 2024 contain data from both Kaiku Health versions 7 and 8, and fully apply to currently certified version 8. | Kaiku Health version 8 superseded Kaiku Health version 7 during 2024. Version number increase was required due to regulatory changes in the European Union (transition from MDD to MDR regulation). All customers were updated to Kaiku Health version 8 on 2024-03-05. | No impact; certified functionalities are identical between Kaiku Health versions 7 and 8, and there is no reason to believe that data collected from both versions are not comparable. |
Summary of Testing Methods and Key Findings
Kaiku Health is a cloud-hosted health information system that is currently sold to Radiation Oncology and Medical Oncology care settings. Cloud hosting enables telemetry connectivity for all sold instances of the health information system. This provides remote event tracking capability which was be used for real world testing of the criterion which is in scope of these results. This ensures testing results accurately reflect the real-world performance of the health information system.
The tested criterion requires that patients can view, download and transmit their health information. Remote tracking therefore focused on patients’ attempts to view, download, and transmit, their health information using the health information system. Additionally, patients need to be able to view the activity log, showing how their health information was accessed for view, download and transmit. Real-world error rates were established for each of those functionalities.
Measuring real world use of the certified functionality by remote event tracking was a successful approach. It enables data collection across all instances of Kaiku Health and gives an accurate picture of how frequently which functionality is used.
The results show that view, download & transmit functionalities of patient records are used unevenly. The “View” functionality is by far the most frequently used functionality (66.3% of events in tracking period), followed by “View Activity Log” (22.5% of events in tracking period). “Download” accounted for 10.0% of events in the tracking period. “Transmit” functionalities are rarely used, with “unencrypted transmit” accounting for 1.1% of events in the tracking period, and “secure transmit” accounting for 0.1% of events in the tracking period. This usage pattern is very similar to the Real World Test Results from 2023.
Observed error rates for View, Download and View Activity Log were very low. In comparison, observed error rates for Transmit functionalities in comparison were higher (around 10-20%), but on the basis of low or very low overall event numbers. The transmit functionalities performed as intended in Elekta-internal testing on production-equivalent software. The most likely source of the observed errors is an invalid receiver email address entered by the user. In those situations, the product informs the user that the email address is invalid and allows them to correct the email address and continue with transmission.
In summary, the results demonstrate that Kaiku Health is used in the real world for functionalities certified under §170.315(e)(1): View, Download and Transmit to 3rd Party.
Standard Updates
There have been no updates to certified standards for Kaiku Health in 2024.
Care Settings
Elekta markets its products in the Radiation Oncology and Medical Oncology care settings. The results reported in this document are aggregated across all instances of Kaiku Health in the United States. This includes both care settings.
Metrics and Outcomes
Criterion: §170.315(e)(1): View, Download and Transmit to 3rd Party
| Measurement/Metric | Error rate for View |
|---|---|
| Expected Outcome | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate. |
| Outcomes | Baseline error rate: 0% errors out of 2755 viewing events Tracking error rate: >0.1% errors out of 18421 viewing events Outcome is as expected. |
| Relied Upon Software | MOSAIQ 2.86 (or higher) |
| Measurement/Metric | Error rate for Download |
|---|---|
| Expected Outcome | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate. |
| Outcomes | Baseline error rate: 0% errors out of 651 download events Tracking error rate: 1.6% errors out of 2771 download events Outcome is largely as expected. It is notable that all the observed errors during the tracking period occurred in March 2024, with no errors observed thereafter. Errors can be traced to an individual bug that was corrected as part of normal maintenance. |
| Relied Upon Software | MOSAIQ 2.86 (or higher) |
| Measurement/Metric | Error rate for Transmit (unencrypted email) |
|---|---|
| Expected Outcome | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate. |
| Outcomes | Baseline error rate: 10.7% errors out of 122 transmit (unencrypted)
events Tracking error rate: 8.6% errors out of 314 transmit (unencrypted) events Baseline error rate is lower than tracking error rate from 2023, which was 15.7%. The observed rate of about 10% still means that roughly 1 out of 10 attempts to transmit a record fails. The transmit (unencrypted) functionality performed as intended in Elekta-internal testing. Most likely observed errors arise from an invalid receiver email address entered by the user. Note that in those cases the user interface informs the user that the entered email address is invalid, allowing them to correct the email address and continue with transmission. Tracking error rate is lower than baseline error rate, as expected. The data shows that the transmit (unencrypted) functionality is rarely used, since transmit (unencrypted) events account for only 1.1% of overall tracked events. |
| Relied Upon Software | MOSAIQ 2.86 (or higher) |
| Measurement/Metric | Error rate for Transmit (secure) |
|---|---|
| Expected Outcome | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate. |
| Outcomes | Baseline error rate: 10.0% errors out of 10 transmit (secure)
events Tracking error rate: 23.5% errors out of 17 transmit (secure) events Tracking error rate is higher than expected, although it is considerably lower than the tracking error rate from 2023, which was 66.7%. It must be noted that the overall sample size is very low. The transmit (secure) functionality performed as intended in Elekta-internal testing. Most likely observed errors arise from an invalid receiver email address entered by the user. Note that in those cases the user interface informs the user that the entered email address is invalid, allowing them to correct the email address and continue with transmission. Tracking error rate is higher than the baseline error rate, but this is most likely due to variation in a very small sample size. The data shows that the transmit (secure) functionality is very rarely used, since transmit (secure) events account for only 0.1% of overall tracked events. |
| Relied Upon Software | MOSAIQ 2.86 (or higher) |
| Measurement/Metric | Error rate for View Activity Log |
|---|---|
| Expected Outcome | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate. |
| Outcomes | Baseline error rate: 0% errors out of 1493 activity log viewed
events Tracking error rate: 0% errors out of 6249 activity log viewed events Outcome is as expected. |
| Relied Upon Software | MOSAIQ 2.86 (or higher) |
Key Milestones
| Milestone | Care Setting | Timeframe |
|---|---|---|
| Baseline remote event and error rate collection period | Radiation Oncology Medical Oncology | January 1 – February 28, 2024 |
| Error rate tracking period | Radiation Oncology Medical Oncology | March 1 – December 31, 2024 |
| Analysis and report creation | Radiation Oncology Medical Oncology | January 2025 |
| Submit Real World Testing report to ACB per instructions | Radiation Oncology Medical Oncology | January 22, 2025 |
Attestation
The Real World Testing Results above are complete with all required elements, including measures that address all certification criteria and care settings. All information included in these results areup to date and fully addresses the health IT developer’s Real World Testing requirements.
Authorized Representative Name: Martin Deppe
Authorized Representative Email: martin.deppe@elekta.com
Authorized Representative Phone: +358 40 661 7170
Date: 2025-01-22