Introduction
The innovative medical device startup, ComeBack Mobility is tackling the challenge of bringing more science into post-surgical weight-bearing. What is now mostly theory (how much weight should one bear), becomes measurable and evidence-based with “Smart Crutch Tips” tech. By monitoring patient activity and accurately measuring weight-bearing on the injured limb, Smart Crutch Tips aims to provide compliance and help patients get rid of crutches faster. Though the idea may seem simple, it involved extensive testing, improvement, and compliance procedures before reaching the stage of clinical study with a Tier 1 New York Hospital.
Navigating Compliance Procedures: The path from idea to product readiness presented numerous compliance challenges, including HIPAA, FCC, CGMP regulation, ISO 9001:1994, ISO 13485, and ISO/IEC 27001. The company conducted database penetration tests, stress tests, data collection accuracy tests (firmware => patient mob app => backend => doctor mob app), and data segregation tests. Additionally, the servers were relocated to the United States to meet regulatory requirements.
Developing a Weight-Bearing Tracking System: ComeBack Mobility, the company behind Smart Crutch Tips, has developed a comprehensive weight-bearing tracking system. This includes mobile apps for both doctors and patients (available on iOS and Android platforms) and a specialized device that accurately measures weight-bearing during walking with crutches or a cane.
Methodology: what is our secret?
To ensure the reliability and functionality of Smart Crutch Tips, a meticulous methodology was followed. Nine iterations were conducted, simulating various scenarios and usage conditions. These tests encompassed average daily usage, scenarios without internet or Bluetooth connection, and even extreme weather conditions (-5°C/23°F). The collection of valuable data from diverse situations enables the company to pivot and initiate product sales, guaranteeing patient safety and the sustainability of Smart Crutch Tips.
Role Assignments and Testing: Based on different patient situations and conditions, specific roles were assigned for testing purposes. These included:
- The Outdoor,
- The Non-Internet,
- The Poor Connections,
- The No Connections,
- The Discharged Cane,
- The Doctor,
- The Normal Patient Control group.
By assessing the device’s performance in these scenarios, Smart Crutch Tips could address specific user needs effectively.
Settings
During the testing and development phase of Smart Crutch Tips, the following settings and parameters were established to ensure the comprehensive evaluation and accurate data collection:
Platform Usage: The Smart Crutch Tips application was utilized on both iOS and Android platforms, ensuring compatibility and accessibility for a wider range of users.
Time Zone Variations: Testing was conducted in different time zones, to account for any potential variations in performance or data synchronization across different geographical locations.
Diverse Weight-Bearing (WB) Programs: Participants in the testing phase were assigned different WB programs based on their specific rehabilitation protocols. These programs followed the guidelines and protocols prevalent in the United States, featuring a gradual progression from 20% to 80% of the individual’s body mass. The WB program evolved and adjusted as the rehabilitation process advanced.
Step Count Commitment: Each team member involved in the testing phase committed to taking a daily average of 500 to 3000 steps. This range approximates the typical step count made by patients with crutches. To accurately track the step count, team members utilized clickers, manually recording each step taken. The objective was to ensure that every movement was correctly displayed in both the Doctor’s and Patient’s Apps, providing accurate data for analysis.
By implementing these settings, the testing process aimed to capture data from various platforms, time zones, and rehabilitation scenarios. This comprehensive approach enabled the evaluation of Smart Crutch Tips’ performance under different conditions, ensuring its reliability and effectiveness across diverse user profiles.
Testing Runs: Achieving 99% Accuracy
Throughout the testing phase, our goal was to reach 99% accuracy in step detection and data unloading to the Doctor’s and Patient’s Apps. We conducted multiple testing runs, each aimed at improving the performance and reliability of Smart Crutch Tips. Let’s explore the journey to attaining our target accuracy:
The First Run: The initial run focused on comparing and verifying the accuracy of step detection in the Mobile apps. While we accomplished our goal four days later, we encountered numerous bugs during this phase. Issues such as incorrect timestamps, delayed voice messages, and false steps emerged. Although challenging, these bugs were identified and addressed, allowing us to proceed.
During the Next Three Runs: Subsequent testing runs involved refining the system and fixing bugs. We continually worked on polishing the software, ensuring the accuracy of data collection, and refining the step detection algorithm. Calibrating the weight gauge proved to be particularly demanding, as we aimed for only a 1% error margin in data collection. We focused on rectifying false steps and enhancing overall accuracy.
False steps – the step-like movement that a person makes unintentionally, e.g., lean on crutches, body turns, crouching, or while walking on stairs.
The Fifth Run: False steps were successfully resolved, leading to improved accuracy. However, there were still remaining bugs that required attention. We expanded the scope to include testing scenarios specific to Cane mode and made focused on indications.
The Sixth Run: This run involved subjecting ourselves to extreme conditions without an internet or Bluetooth connection to the Smart Crutch Tips for ten days. We relied solely on the indication prompts on the tips. Our goal was to ensure that all steps were correctly unloaded to the Doctor’s and Patient’s Apps and the server. While we faced challenges with synchronization and encountered a few indication bugs, we made progress and prepared for future runs.
The Seventh Run: This trial had to be halted due to firmware issues. Despite achieving a step detection rate of 98% and 100% data unloading to apps and servers, the firmware required improvements.
Meanwhile, we had signed an agreement with an NYU Langone Health, and researchers were eagerly awaiting our devices. This added to our sense of urgency as we prepared to dispatch 50 pairs of Smart Crutch Tips. With no time to waste, we dived into bug fixing.
The Eighth Run: Ilya, CEO personally double-checked all 50 pairs of Smart Crutch Tips, taking 50 steps on each pair. We achieved an overall accuracy of 99.64%. The barrier has been overcome, but for assurance, it was now necessary to conduct long-term testing on 12 pairs of devices with, simulating all conditions under which errors once occurred, or data could have been recorded incorrectly.
The Ninth Run: This final run before dispatching the devices involved expanding our testing group to nearly twice its size. Twelve individuals were selected to walk daily under various conditions, cases, and WB programs. We meticulously tracked every move, ensuring there were no false steps, incorrect timestamps, or missed days. Nine days later, the moment of truth arrived, finally, we had reached our desired accuracy level!
The Ninth Run summary revealed a total of 65,000 steps with an astounding accuracy of 99.52%.
What’s Next?
Despite the challenging circumstances during the device assembly and testing, we are thrilled to announce that we have successfully prepared 200 pairs of Smart Crutch Tips. 50 of them will now be utilized by a prestigious Tier 1 Hospital in Manhattan, NYC, as part of a comprehensive study. The study aims to examine the impact of using Smart Crutch Tips on compliance with prescribed weight-bearing protocols compared to traditional methods. Here’s what this means for us:
- The scientific validation of our technology through evident differences in compliance will serve as a solid foundation for showcasing the significant patient outcomes we can deliver.
- The existence of this scientific proof will pave the way for strategic partnerships and commercialization opportunities with hospitals that prioritize patient compliance and care.
- Furthermore, the publication of this study will provide us with the necessary credibility to seek reimbursement for the use of our devices by Medicare and Medicaid.
- As we continue to make strides in the healthcare industry, the impact on our company valuation is bound to be nothing short of magical.
We express our heartfelt gratitude to Dr. Kenneth A. Egol, Matthew Sgaglione, and the research team from NYU Langone Health for believing in us and our medical device. Their support and collaboration have been instrumental in bringing our innovation to the forefront of patient rehabilitation.
