On a balmy mid-April afternoon, amidst the cacophony of passionate deliberations ringing the walls of the Paul Brest Hall on Stanford University’s sylvan green campus, a picture began to emerge. It was the annual mHealth Connect conference and most of the talks by academic stalwarts, healthcare entrepreneurs and venture capitalists were harping on a common thesis: The growing nexus between mobile sensors and AI (Artificial Intelligence) in the healthcare domain.
Amidst ardent calls for silicon-valley styled disruption of the ailing healthcare sector, it became ostensibly clear that three facets of personal healthcare were going to see important changes driven by the emergence of the sensors-laden smartphone as the centerpiece of democratization of healthcare-hardware.
In the next few paragraphs, you will be exposed to the following three domains earmarked to be recipients of the disruption:
- Gait and Posture related disorders.
- Geriatric Care
- Treatment of neurodegenerative disorders
On first glance, it might seem as if ushering these advancements demands dramatic popularization of highly specialized and potential expensive medical sensors such as the Photo-Plethysmo-Gram (PPG) heart rate monitor sensor found thus far only on high-end phones such as Samsung S8. Far from it, we will see that many solutions catering to a wide array of challenges in all three of the above-mentioned domains, will, in fact, be fueled by data emanating from the humble triad of Inertial Measurement Unit-IMU-sensors, or motion sensors, that is accelerometers, gyroscopes and magnetometers. With this background, let us dive deeper into the specific domains and the success stories that have emerged.
Gait & Posture Related Disorders
Paleoanthropologists believe that approximately 4 to 2.8 million years ago, the now extinct hominin Australopithecus afarensis, had finally mastered the art of performing a series of complicated, cooperative and coordinated manipulation of the musculoskeletal and locomotor controller systems, resulting in a physical activity that we now take for granted: The upright human bipedal walk!
The repetitive pattern spanning two successive steps or one complete stride constitutes what is formally studied as a Gait cycle. A gait cycle begins when the heel of one foot strikes the ground and culminates the instant when the same foot contacts the ground again. As per the RLA (Rancho Los Amigos) system, a single gait cycle consists of eight temporal phases, which include: Initial Contact, Loading Response, Mid-stance, Terminal Stance, Pre-swing, Initial Swing, Mid Swing and Late Swing phase.
Fine grained analysis of gait reveals an astonishingly wide array of clues about the person’s state of general health, both from the orthopedic perspective as well as the neuromuscular perspective. From an orthopedic viewpoint, study of gait kinematic deviations from the normative model is an integral part of diagnosis of conditions such as developmental dysplasia of the hip (DDH) , Hip fracture and chronic ankle instability. The Stanford Motion and gait-analysis laboratory lists eight basic pathological gaits that can be attributed to neurological conditions: hemiplegic, spastic diplegic, neuropathic, myopathic, Parkinsonian, choreiform, ataxic (cerebellar) and sensory.
A typical gait analysis study entails a Physical exam, followed by video analysis and possibly Electromyographic (EMG) measurements of muscle activity during movement. Enter the scene, motion sensors. Modern day smart-phone sensors such as Accelerometers and Gyroscopes that allow sampling rates of as high as 400 samples/sec facilitate highly precise measurements of the gait cycle that can be done by the patient just placing the phone in his/her pocket and walking while at the comfort of one’s home.
In Figure 1, we see a visualization of gait cycle time-series signals measured from the accelerometer of a smartphone carried by a volunteer at UnifyID labs.
As for the question of these measurements meeting the rigor of a proper medical examination, there exists a plethora of peer-reviewed published research that showcases that it is indeed the case. Nearly eight years ago, Rigoberto et al [1] presented a study that showcased how they had successfully used an off-the-shelf commercial smartphone as a tool to evaluate anticipatory postural adjustments (APA) before the beginning of normal gait in healthy subjects.
In the same year, Lemoyne et al [2] demonstrated an implementation of an iPhone as a wireless accelerometer for rigorously quantifying gait characteristics. Then in 2012, Nishiguchi et al [3] studied in detail the reliability and validity of gait analysis by the Xperia SO-01B Android smartphone on 30 subjects and concluded that the smartphone-based measurements indeed had the capacity to quantify gait parameters with a comparable degree of accuracy that was attained using a dedicated medical grade tri-axial accelerometeric sensor. In 2016, we saw the first usage of a smartphone as a wireless gyroscopic platform for quantifying reduced arm swing in hemiplegie gait [4]. More recently, researchers from Toho university [5] have also developed the Gait-Kun system consisting of tri-axial accelerometers and shown promising levels of accuracy in measuring ataxia due to spinocerebellar degeneration.
Geriatric Care
A crucial component of delivering Geriatric care is continuous monitoring. In this regard, the motion sensor laden smartphone emerges as a key tool that can be used in lieu of or in conjunction with video monitoring systems. One oft-ignored facet of geriatric care is Fall detection and response. As stated in this study on smartphone-based fall detection systems, ‘A fall has also dramatic psychological consequences, since it drastically reduces the self-confidence and independence of affected people. This may contribute to future falls with more serious outcomes or it may lead to a decline in health’.
In 2012, He et al [6] showcased how fall detection using smartphone motion sensors can be performed rather accurately and how this technology can be a crucial piece in dramatically reducing the response time of the healthcare professionals in the event of a fall. In the same year, Fontecha et al [7] demonstrated a system that could support physicians to construct a highly granular and centralized elderly frailty diagnosis, by munging activity data from an accelerometer-enabled mobile phone with clinical data emanating from lab tests. Besides this, the Timed-Up-and-Go test (TUG test) which is also heavily used in geriatric care and for assessing the quality of recovery from hip fractures and Cerebrovascular accidents (CVAs) can be performed with high accuracy using the motion sensors on a smartphone.
In 2012. Mellone et al [8] published a detailed study on the validity of a smartphone-based instrumented Timed Up and Go. This was followed by another highly cited work by Milosevic et al [9] published in 2013 where they introduced a smartphone application called sTUG that could completely automate the TUG test, so it could be performed by patients from the comfort of their homes.
Treatment of neurodegenerative disorders: Parkinson’s, Multiple Sclerosis, Alzheimer’s, and Huntington Disease
To begin with, it is important to note that the advancements in the TUG test explored above is also applicable for neurodegenerative disorders. The TUG test was initially designed for elderly persons but is today also prescribed for patients with neurodegenerative disorders such as Parkinson’s disease, Multiple Sclerosis, Alzheimer’s disease and the Huntington Disease.
Besides this, there exists a vast body of work where researchers have used motion sensors to help diagnose and assess neurodegenerative disorders. The first mobile gait analysis for early diagnosis and therapy monitoring in Parkinson's disease was published way back in 2011 by a group of German researchers at University Hospital of Erlangen [10]. Then in 2012, Chung at al [11] showcased how their stride detection algorithm when applied to accelerometric data from commercial smartphones could be used to perform early onset detection of Alzheimer's Disease (AD). This is based on the crucial observation that AD patients exhibit a significantly shorter mean stride length and slower mean gait speed than those of the healthy controls, both of which metrics could be precisely estimated using the commercially available accelerometers.
Then in 2014, a Taiwanese team led by Kun-Chan Lan [12] showcased how their Pedestrian Dead Reckoning (PDR) based gait monitoring system could be used for early Diagnosis of Parkinson’s Disease. This was followed by Kostikis et al’s work [13] on building a practical smartphone-based tool to accurately assess upper limb tremor in Parkinson's disease (PD) patients.
Associated with this tremor analysis trend has been the slew of studies that have looked at replacing expensive Electromyogram(EMG) readings with smartphone accelerometric readings. Two years back, a team of researchers at University of Coimbra [14] showed that measurements from off-the-shelf apps such as iSeismometer could be used as a reliable alternative to EMG for tremor frequency assessment when it comes to care of patients with a diagnosis of PD, ET (Essential tremor) and Holmes’ tremor. This was followed by another study published last year [15] where accelerometric readings from an iPhone inserted into a patient’s socks could help detect Orthostatic tremor (OT), which is indeed a strong vindication of this approach as OT was hitherto considered to be one of the few tremor conditions that did require an EMG lab assessment for definitive diagnosis given that leg tremor might not be visible to the naked eye as such.
Conclusion
On the concluding note, one can say it is particularly impressive to note that unlike similarly promising nexuses between gene editing and AI, for example, there is a far lesser degree of uncertainty and speculation here with noteworthy advancements already achieved that now await careful productization. The other two aspects that only enhance the prospects of this impending revolution of sorts are:
- Emergence of Deep Learning (DL) as the computational framework of choice: Many of the results above have been achieved using simple (shallow) Machine Learning and traditional model-based signal processing approaches. As seen in the domains of speech, vision and text, DL has shattered the state-of-the-art metrics by a fair distance and given that adoption of DL based approaches has been slow in the sensor domain, we will surely see the accuracies of these tests only go higher.
- Sampling rates of sensors on phone: As things stand, the motion sensor chips that are currently available on smartphones are severely underutilized in terms of the sampling rate they can provide. This is done mainly to extend battery-life as well as because of a few security concerns. However, with the right security policies in place and use of specialized and well monitored SDKs (such as Samsung health SDK), one can unlock the hidden potential of these nifty little workhorses to open avenues for more granular and sensitive measurements at much higher sampling rates, that will in turn propel the way to more medical applications.
References:
[1] Rigoberto MM, Toshiyo T, Masaki S, Smart phone as a tool for measuring anticipatory postural adjustments in healthy subjects, a step toward more personalized healthcare; Conf Proc IEEE Eng Med Biol Soc. 2010; 2010():82-5.
[2] LeMoyne, Robert, et al. "Implementation of an iPhone as a wireless accelerometer for quantifying gait characteristics." Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE. IEEE, 2010.
[3] Nishiguchi, Shu, et al. "Reliability and validity of gait analysis by android-based smartphone." Telemedicine and e-Health 18.4 (2012): 292-296.
[4] LeMoyne, Robert, and Timothy Mastroianni. "Implementation of a smartphone as a wireless gyroscope platform for quantifying reduced arm swing in hemiplegie gait with machine learning classification by multilayer perceptron neural network." Engineering in Medicine and Biology Society (EMBC), 2016 IEEE 38th Annual International Conference of the. IEEE, 2016
[5] Sakakibara, R., et al. "Wearable gait sensors to measure degenerative cerebellar ataxia." Journal of the Neurological Sciences 381 (2017): 56-57.
[6] He, Yi, Ye Li, and Shu-Di Bao. "Fall detection by built-in tri-accelerometer of smartphone." Biomedical and Health Informatics (BHI), 2012 IEEE-EMBS International Conference on. IEEE, 2012.
[7] Fontecha, Jesús, et al. "Elderly frailty detection by using accelerometer-enabled smartphones and clinical information records." Personal and ubiquitous computing 17.6 (2013): 1073-1083.
[8] Mellone, Sabato, Carlo Tacconi, and Lorenzo Chiari. "Validity of a Smartphone-based instrumented Timed Up and Go." Gait & posture 36.1 (2012): 163-165.
[9] Milosevic, Mladen, Emil Jovanov, and Aleksandar Milenković. "Quantifying Timed-Up-and-Go test: A smartphone implementation." Body Sensor Networks (BSN), 2013 IEEE International Conference on. IEEE, 2013.
[10] Barth, Jens, et al. "Biometric and mobile gait analysis for early diagnosis and therapy monitoring in Parkinson's disease." Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE. IEEE, 2011.
[11] Chung, Pau-Choo, et al. "Gait analysis for patients with Alzheimer's disease using a triaxial accelerometer." Circuits and Systems (ISCAS), 2012 IEEE International Symposium on. IEEE, 2012.
[12] Lan, Kun-Chan, and Wen-Yuah Shih. "Early Diagnosis of Parkinson's Disease Using a Smartphone." Procedia Computer Science 34 (2014): 305-312.
[13] Kostikis, N., et al. "A smartphone-based tool for assessing parkinsonian hand tremor." IEEE journal of biomedical and health informatics 19.6 (2015): 1835-1842.
[14] Araújo, Rui, et al. "Tremor frequency assessment by iPhone® applications: correlation with EMG analysis." Journal of Parkinson's disease 6.4 (2016): 717-721.
[15] Balachandar, Arjun, and Alfonso Fasano. "Characterizing orthostatic tremor using a smartphone application." Tremor and Other Hyperkinetic Movements (2017).
About the author
Vinay Prabhu is the Principal Machine Learning Scientist at UnifyID Inc, a post-Series-A security startup based in San Francisco. His current research lies at the intersection of deep learning, security and smartphone sensors. He has published over 20 peer-reviewed publications in various international conferences and journals and holds a Ph.D. in Electrical and Computer Engineering from Carnegie Mellon University.