class: center, middle, inverse, title-slide # Unsupervised Behavior Change Detection ## Using Passive Sensor Systems in the Homes of Older Adults ### Garrick Aden-Buie, C. VandeWeerd, A. Yalcin --- class: middle exclude: true <!-- LINKS [isg-2018]: https://www.sfu.ca/fc/ISG2018/ [xaringan]: https://github.com/yihui/xaringan --> ## Outline 1. Introduction - CREATE Health - HomeSense 1. Activity Profiles 1. Behavior Change Detection --- exclude: true ## Overview 1. Intro - Our study focuses on addressing X (one slide) - Study participants (one slide) - Research objective 1. System - Sensors, placement, etc. 1. Activity Profiles - Day-event sequences and bag-of-event *n*-grams - How are APs compared? - Occupant identification task: experiment and results 1. Behavior Change Detection --- background-image: url(images/create_health.png) background-size: 100% background-position: 50% 50% name: create-health class: bottom, center ??? Who are We? Collaborative for Research & Education<br> in Aging, Tech & Health - Community Partners, Funders, Academics, Students from a variety of disciplines - supported and connected by USF - with a focus on improving health and qol of life in late life --- exclude: true <!-- My CREATE Health Slide --> .big[ ![](images/createhealth-team.jpg)<br><br> `usf.edu/engineering/create-health`<br><br> ] ??? CREATE Health - is an **interdisciplinary team** - of ENG & PUB HEALTH students & researchers - dedicated to improving the health and quality of life of aging populations - through the use of technology CH's research focuses on - alleviating the burdens, financial and physical, of chronic disease treatment - by examining how technology can improve healthcare delivery and support, - in particular at the intersection of community and organizational health system components. --- layout: false name: villages class: bottom, center, bg-full, no-pad, no-slide-number <!-- background-size: cover --> <!-- background-image: url(images/villages-flyover.png) --> <video loop muted autoplay poster="images/villages-flyover.png" class="bg-video"> <source src="images/villages-flyover2.mp4" type="video/mp4"> </video> .box-center-bottom.box-light[ <img src="images/thevillages-logo.svg" width="80%"> <div style="text-align: left; padding-left: 60px"> <ul> <li>The largest retirement community in US</li> <li>75 miles north of Tampa, FL</li> <li>115,000 residents in 50,000 homes</li> <li>Average age: 62 (M) and 60 (F)</li> <ul> </div> ] <!-- <div style="padding: 15px"> </div> --> ??? A majory community partner in CREATE Health's work is The Villages... .notes-list[ - 32 sq miles - 3 town squares - 63 rec centers - legend has it a Villager can play 18 holes of gold for 30 straight days without repeating ] --- class: bottom center no-slide-number background-image: url(images/participant-seated.jpg) background-size: cover layout: true --- count: false ??? - I'd like to introduce you to one of the participants in our study, who I will call **Dorothy**. - Dorothy is between 75 and 80 years old, she's divorced, and she lives alone in her home in The Villages. - Overall, she's in great health. - As a typical retiree living in the Sunshine State, she leads a highly active life involving - an assortment of clubs and groups, - dance classes at the recreation center, and - frequent lunches and dinners in the town square with friends. - She lives alone, but she talks to her daughter regularly, even though they live on opposite sides of the US. --- count: false ??? - About a year or so ago, though, Dorothy's **recurrent hip pain** worsened and she and her doctor decided for **hip replacement surgery**. - She arrived at the hospital on the day of the procedure only to find out that due to an insurance issue, she would have to wait another three months. - In the mean time, - her mobility declined, - she spent more time in bed, and - she couldn't make it to many of the activities she loved. - Her daughter had planned to visit during this time, but due to a family emergency she had to cancel her plans. - In the end surgery went well, Dorothy recovered quickly and a few months later she was back to living the life she enjoyed. --- layout: false .pull-left[ ### Aging in Place - Americans 65+ will **double** from 2014 to **98M** in 2060<sup>1</sup> - 80% manage one or more<br>chronic conditions<sup>2</sup> - Decrease in available traditional family caregivers<sup>3</sup> - Strong preference to<br>**age in place** ] .pull-right[ <img src="images/prb--aging-us-pop-bulletin--plots.png" height="450px" style="margin-top: 40px"> ] .footnote.small[ 1: Mather, Jacobssen, and Pollard (2015) <br> 2: Pearson, Bhat-Schelbert, and Probst (2012) <br> 3: Demiris and Hensel (2008) ] ??? .notes-list[ I'm sharing Dorothy's story for __2 reasons__: The first because... __her experience highlights the challenges__ we face as a society in how we approach aging and manage health care for older adults. 1. She is part of a __growing population of seniors__, one that is expected to double to 98M over the next 40 years 2. __Life expectancy has increased by 3 mon/year since 1840<sup>*</sup>__ - and while 75% of seniors report being in __good to excellent health__, - __a majority are managing chronic health conditions__. 3. More older adults are divorced or live alone than ever before, which in conjunction with demogrph. changes leads to decreased availability of traditional family caregivers 4. Additionally, there is a strong preference to age in place, __like Dorothy__ -- remain in their own homes and communities, for as long as possible (75% want to stay, vast majority 96% live in community) ] <!-- .footnote.small[<sup>*</sup>Wouters 2017] --> --- layout: false background-image: url(images/participant-pointing-sensor.jpg) background-size: cover class: right middle no-slide-number .box-right.box-light[ # HomeSense ] ??? The second reason I've shared Dorothy's story is because, - as a participant in our study, - she had a home sensor system installed in her home - and that system allowed Dorothy's daugher to stay in touch with her mother's daily activities while they waited 4 hip replace The goal of the system that we call __HomeSense__ - is to support seniors who want to age in place safely and independently - by monitoring daily activities - in particular focusing on the activities that signal or influence health outcomes --- ### The HomeSense System .pull-left.center[ ![](images/sensors.png) <img src="images/server-rack.jpg" height="155px"> ] <!-- .bigger.center[ <i class="fas fa-bolt" style="color: #FFC107"></i><br> <i class="fas fa-database" style="color: #28A745"></i> <i class="fas fa-table" style="color: #17A2B8"></i> <i class="fas fa-bell" style="color: #DC3545"></i> ] --> .pull-right[ <img src="images/wilson-admin-01-front-page.png" style="border: 2px #000 solid; border-radius: 2px"> <img src="images/homesense.jpg" width="100%" style="border: 2px #FFF solid"> ] ??? The main compenents of the system are - wireless sensors installed throughout the home - **presence**: PIR motion sensors in all commonly used areas - **magnetic contact sensors**: doors, cabinets, fridge access - **other**: electricity, temp, luminance, etc. - the data is collected through raspberry pi in home<br> ...then collected, warehoused and processed on our servers - A suite of tools for internal use such as - a custom inventory management system and - a number of dashboards - A web interface for users and family/friends<br> can review summarized and detailed activity data --- class: bottom center no-slide-number background-image: url(images/participant-smiling-computer.jpg) background-size: cover .box-bottom.box-light.big[_I like the idea that my kids feel really secure, knowing that I’m on the other side of the country._] ??? When asked about her experience with HomeSense, Dorothy said... --- class: bottom center bg-full no-slide-number background-image: url(images/participant-computer-pointing.jpg) count:false .box-bottom.box-light.big[_This is a great non-intrusive system<br>that makes me at ease, makes my family and <br>I feel more safe and connected._] ??? This really summarizes the potential impact of a system like HomeSense in terms of the sense of security and reassurance that it provides to older adults and their caregivers. --- ### Lifestyle Reassurance - Alleviate burden of chronic illness with supportive technology<sup>3</sup> - Provide safety and security around changes in health and routine ??? This need for safe aging in place is reflected in our design goal of supporting **lifestyle reassurance**. Our objective is to develop supporting technologies such as HomeSense to alleviate the burden of chronic illness on OA & CGs A primary goal of such a system is to support safety and security by noticing changes in behavior and routine and alerting OA/CGs when these occur -- ### Unsupervised Behavior Change Detection - Labelled activity data is difficult and expensive to collect<sup>4</sup> - Requires solutions without explicit, higher-level activity labels - Reflects real-world, natural, smart home activity data .footnote.small[ 3: Demiris and Hensel (2008)<br> 4: Szewcyzk, Dwan, Minor, et al. (2009) ] ??? <hr> OTOH, because labelled activity data is difficult and expensive to collect –<br> especially natural patterns of Daily Living –<br> we need an approach that does not require explicit, high-level activity labels and that can effectively make use of real-world, natural SH act. data --- name: research ### Research Objective .big[ >_Can we effectively use passively-recorded unlabelled sensor data to characterize lifestyle activity patterns and enable detection of changes in routine?_ ] ??? To summarize, our research objective is centered on the question... We will address this question from two angles: 1. Defining *lifestyle activity pattern* 2. Using the AP for behavior change detection Over the next few slides, we'll touch briefly on<br> the sensor system and the data that we collected<br> Then I'll explain our definition of *activity profiles* and present the results of an experiment to demonstrate that these activity profiles capture the unique behavior of the occupant. Finally, we'll discuss the method and results of the algorithm for behavior change detection. --- layout: true ### Activity Profiles ??? *Faux-animated slides* - Explain how sensor events are used, divided, processed - Given example of sensor events - String representation simply encodes a series of actions - To get bag of n-grams, take sliding window across event series - Gather and count each n-gram --- - Concise representation of sensor activities - Allow comparison between activities at different time points or occupants -- <br> Represent sensor events as daily *event sequences* 1. Use only **active** sensor firings grouped by day 1. Day begins at first 5 active events after 4 a.m. 1. Insert *pause event* when no activity in 15 minutes 1. Summarize event sequence as **bag-of-event _n_-grams**<sup>5</sup> .footnote.small[ 5: Hamid, Johnson, Batta, et al. (2005) ] --- .ngram-example[ <code> IiIAEFfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] --- count: false .ngram-example[ <code> .hl-text[I]iIAEFfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> </p> --- count: false .ngram-example[ <code> .hl-text[Ii]IAEFfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-bath" style="color: #006484"></i> </p> --- count: false .ngram-example[ <code> .hl-text[IiI]AEFfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-bath" style="color: #006484"></i> <i class="fas fa-bed" style="color: #29AFCE"></i> </p> --- count: false .ngram-example[ <code> .hl-text[IiIA]EFfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-bath" style="color: #006484"></i> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-couch" style="color: #DBE442"></i> </p> --- count: false .ngram-example[ <code> .hl-text[IiIAE]FfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-bath" style="color: #006484"></i> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-couch" style="color: #DBE442"></i> <i class="fas fa-utensils" style="color: #9CCB3B"></i> </p> --- count: false .ngram-example[ <code> .hl-text[IiIAEFf]Ff0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-bath" style="color: #006484"></i> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-couch" style="color: #DBE442"></i> <i class="fas fa-utensils" style="color: #9CCB3B"></i> <br> <i class="fas fa-sign-out-alt" style="color: #009374"></i> <i class="fas fa-sign-in-alt" style="color: #a97a7a"></i> </p> --- count: false .ngram-example[ <code> .hl-text[IiIAEFfFf]0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-bath" style="color: #006484"></i> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-couch" style="color: #DBE442"></i> <i class="fas fa-utensils" style="color: #9CCB3B"></i> <br> <i class="fas fa-sign-out-alt" style="color: #009374"></i> <i class="fas fa-sign-in-alt" style="color: #a97a7a"></i> <i class="fas fa-sign-out-alt" style="color: #009374"></i> <i class="fas fa-sign-in-alt" style="color: #a97a7a"></i> </p> --- count: false .ngram-example[ <code> .hl-text[IiIAEFfFf0]AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] <p class="huge" style="line-height: 1.5em"> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-bath" style="color: #006484"></i> <i class="fas fa-bed" style="color: #29AFCE"></i> <i class="fas fa-couch" style="color: #DBE442"></i> <i class="fas fa-utensils" style="color: #9CCB3B"></i> <br> <i class="fas fa-sign-out-alt" style="color: #009374"></i> <i class="fas fa-sign-in-alt" style="color: #a97a7a"></i> <i class="fas fa-sign-out-alt" style="color: #009374"></i> <i class="fas fa-sign-in-alt" style="color: #a97a7a"></i> <i class="fas fa-pause-circle" style="color: #466069"></i> </p> --- .ngram-example[ <code> IiIAEFfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0 </code> ] --- count: false .ngram-example[ <code>.word.hl-text.hl[IiIAE]FfFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0</code> ] -- .big[⬇] -- #### Bag of event n-grams .word[IiIAE] --- count: false .ngram-example[ <code>I.word.hl-text.hl[iIAEF]fFf0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0</code> ] .big[⬇] #### Bag of event n-grams .word[IiIAE] .word[iIAEF] --- count: false .ngram-example[ <code>Ii.word.hl-text.hl[IAEFf]Ff0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0</code> ] .big[⬇] #### Bag of event n-grams .word[IiIAE] .word[iIAEF] .word[IAEFf] --- count: false .ngram-example[ <code>IiI.word.hl-text.hl[AEFfF]f0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0</code> ] .big[⬇] #### Bag of event n-grams .word[IiIAE] .word[iIAEF] .word[IAEFf] .word[AEFfF] --- count: false .ngram-example[ <code>IiIA.word.hl-text.hl[EFfFf]0AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0</code> ] .big[⬇] #### Bag of event n-grams .word[IiIAE] .word[iIAEF] .word[IAEFf] .word[AEFfF] .word[EFfFf] --- count: false .ngram-example[ <code>IiIAE.word.hl-text.hl[FfFf0]AE0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0</code> ] .big[⬇] #### Bag of event n-grams .word[IiIAE] .word[iIAEF] .word[IAEFf] .word[AEFfF] .word[EFfFf] .word[FfFf0] --- count: false .ngram-example[ <code>IiIAEF.word.hl-text.hl[fFf0A]E0AIiiOIiIA0AEGg0GAgEIiIA0AEAA0A0AGgE0A0AEAEA0AFfIiIIEAFGgAf0FfAIiIA0AE0AEIiII0IiI0</code> ] .big[⬇] #### Bag of event n-grams .word[IiIAE] .word[iIAEF] .word[IAEFf] .word[AEFfF] .word[EFfFf] .word[FfFf0] .word[fFf0A] --- .pull-left[ **Bag-of-event n-grams** - Repeat process for `\(n\)` from 2 to 6 - Calculate relative frequency `\(p\)` of each `\(n\)`-gram - Can be used to summarize `\(m\)` days **Compare two profiles** - Using Kullback-Leibler Symmetric Distance - `\(D_{KL} (P \Vert Q) = 0.331\)` ] .pull-right[ <table> <thead> <tr> <th style="text-align:center;"> n-gram </th> <th style="text-align:center;"> Count </th> <th style="text-align:center;"> Rank </th> <th style="text-align:right;"> Freq. </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;"> <span class='word'>0A</span> </td> <td style="text-align:center;"> 11 </td> <td style="text-align:center;"> 1 </td> <td style="text-align:right;"> 0.0242 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>AE</span> </td> <td style="text-align:center;"> 8 </td> <td style="text-align:center;"> 2 </td> <td style="text-align:right;"> 0.0176 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>A0</span> </td> <td style="text-align:center;"> 7 </td> <td style="text-align:center;"> 3 </td> <td style="text-align:right;"> 0.0154 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>A0A</span> </td> <td style="text-align:center;"> 7 </td> <td style="text-align:center;"> 4 </td> <td style="text-align:right;"> 0.0154 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>Ii</span> </td> <td style="text-align:center;"> 7 </td> <td style="text-align:center;"> 5 </td> <td style="text-align:right;"> 0.0154 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>iI</span> </td> <td style="text-align:center;"> 7 </td> <td style="text-align:center;"> 6 </td> <td style="text-align:right;"> 0.0154 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>0AE</span> </td> <td style="text-align:center;"> 6 </td> <td style="text-align:center;"> 7 </td> <td style="text-align:right;"> 0.0132 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>IiI</span> </td> <td style="text-align:center;"> 6 </td> <td style="text-align:center;"> 8 </td> <td style="text-align:right;"> 0.0132 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>A0AE</span> </td> <td style="text-align:center;"> 4 </td> <td style="text-align:center;"> 9 </td> <td style="text-align:right;"> 0.0088 </td> </tr> <tr> <td style="text-align:center;"> <span class='word'>EA</span> </td> <td style="text-align:center;"> 4 </td> <td style="text-align:center;"> 10 </td> <td style="text-align:right;"> 0.0088 </td> </tr> </tbody> </table> ] ??? *** KL Distance provides a distance measure between probability distributions. We use KL dist to compare frequencies across b-o-e n-grams (activity profiles) `\(D_{KL} (P \Vert Q) = \sum_{i} \left( p_i - q_i \right) \log \frac{p_i}{q_i}\)` --- layout: true ### Evaluating Activity Profiles --- *Do activity profiles capture the occupant's unique patterns?* -- #### Reframe as supervised learning *Can we identify the correct occupant for a de-identified day of activity?* ??? Also: If we're creating a baseline profile, how many days are needed? --- .pull-left[ #### Data Collection **HomeSense** was installed in homes of - Community-dwelling older adults in The Villages, FL - Who live alone and were of<br>good health Bi-weekly phone survey tracks<br>health changes, significant life events ] ??? Read slides... 14 participants, aged 68+ -- .pull-right[ #### Data Selection for Activity Profiles Selected 6 participants: - Homes with similar floorplan and sensor layout - Who completed at least 196 days<br> (Excluding vacation, maintenance visits, guests & visitors) ] ??? Because sensor installations varied according to floorplan we selected a subset of participants with longterm participation and standard sensor layouts. --- background-image: url(images/simple-exp-covered.png) background-size: 96% background-position: 20% 50% exclude: true --- layout: false ### Floorplan and Sensor Layout <img src="images/simple-exp-covered.png" height="430px"> ??? Included in the sensor network - motion sensors for the generalized detection of presence throughout the home - magnetic contact sensors for detection of interaction with entrance and exit doors and routinely used objects - wireless, z-wave --- ### Evaluating Activity Profiles #### Experimental Setup - Create **activity profile** for each occupant from *N* days - *N* — 14 days to 3 months - Label test day with occupant of best activity profile match - 7-fold cross validation - Test on each of **28 testing days** - Profiles drawn from remaining **168 training days** ??? Note, because profiles drawn from 168 training days, I repeatedly draw from training pool so that all folds have 12 trial total (12 iters, 1 rep -- 2 iters, 6 reps) --- layout: false ### Results: Activity Profiles <table id="table-results"> <thead> <tr> <th style="text-align:left;"> </th> <th style="text-align:right;"> 14<br>Days </th> <th style="text-align:right;"> 28<br>Days </th> <th style="text-align:right;"> 42<br>Days </th> <th style="text-align:right;"> 56<br>Days </th> <th style="text-align:right;"> 70<br>Days </th> <th style="text-align:right;"> 84<br>Days </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Accuracy </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">48</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">60</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">61</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">60</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">58</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">54</span> </td> </tr> <tr> <td style="text-align:left;"> Precision </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">52</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">62</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">63</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">62</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">60</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">56</span> </td> </tr> <tr> <td style="text-align:left;"> Recall </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">48</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">60</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">61</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">60</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">58</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">54</span> </td> </tr> <tr> <td style="text-align:left;"> F-Measure </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">48</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">59</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">61</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">59</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">58</span> </td> <td style="text-align:right;"> <span style="color: #555;">0.9</span><span style="font-weight: normal">53</span> </td> </tr> </tbody> </table> <br> <img src="gab-gerontechnology-2018_files/figure-html/results-ap-1.png" width="720" /> ??? Note 42d (6w) maximizes performance across the board Plotting **F Measure** (F1): balance of precision and recall (balanced measure of binary classification) 28-42 days ok (4, 6 wk) --- count: false exclude: true <img src="gab-gerontechnology-2018_files/figure-html/results-ap-ungrouped-1.png" width="720" /> --- count: false exclude: true <img src="gab-gerontechnology-2018_files/figure-html/results-ap-sel-1.png" width="720" /> --- layout: true ### Behavior Change Detection - Compare occupant's recent activity to previous behavior - 4 participants with 1+ years data - Using all binary, active sensors - Permutation-based methods for change detection<sup><sup>6</sup></sup> .footnote.small[ 6: Sprint, Cook, and Schmitter-Edgecombe (2016) ] ??? - Having validated that the activity profiles are capable<br> of summarizing the activity patterns of an occupant,<br> we now turn to the task of behavior change detection - compare occupant's recent activity to previous behavior<br> previous behavior is either a baseline period of normalcy<br> or sliding windows - Given that KL-dist is a scale-less distance metric,<br> need to establish bounds guidelines for expected similarity *What is normal variation vs. anomaly/change?* - Overall, our goal is to bring abnormalities to the attention of caregiver<br> human review and intervention is expected<br> and system issues are equally requiring of attention --- -- <img src="gab-gerontechnology-2018_files/figure-html/permutation-plots-1.png" width="720" /> --- <img src="gab-gerontechnology-2018_files/figure-html/permutation-plots-2.png" width="720" /> --- layout: true ### Permutation-Based Comparison --- .center[ <img src="images/permutations4.gif" height="450px"> ] ??? Each shuffling of --- .center[ <img src="images/permutations4-boxplot.png" height="450px"> ] --- layout: false ### Results: Behavior Change <img src="gab-gerontechnology-2018_files/figure-html/behavior-change-house013-1.png" width="720" /> ??? - **June** (05-31 -- 06-27) - June 20XX was an odd month, due to company in home - **Late June, Early July** (06-19 -- 07-16) - Back to normalcy - **August** 08-01 -- 08-28 - System wide change in data collection - 10-21 -- 11-17 - ?? - **December** (12-04 -- 01-02) - Company in home all of December - **January** (01-03 -- 02-20) - Reduction in mobility leading up to surgery - **Late February** (03-24 -- 04-20) - Post surgery - **Late April** (04-21 -- 05-18) - Post recovery - **November** (11-09 -- 12-06) - Holidays --- .pull-left[ ### Looking Forward - Early alerting of caregivers or admin through dashboard or notifications - Couple behavior change with comparison to **baseline** - Use permutations to<br>**explain differences**<sup>7</sup> ] .pull-right.center[ <img src="images/chdash-05-overview-by-hour.png" height="225px" style="margin-top: 25px; border: 2px #000 solid; border-radius: 2px;"> <img src="images/notification-email.png" height="225px" style="margin-top: 25px"> ] .footnote.small[ 7: Ribeiro, Singh, and Guestrin (2016) ] ??? .notes-list[ - This work can be integrated into a dashboard or notification system so that caregivers or admin can be alerted early as behavior patterns shift, triggering more careful monitoring - The method for behavior change identification works best on longer-term time horizons -- i.e. previous month was *unusual* -- but can be coupled with a comparison to a baseline period, leading to more fine-grained comparison of daily behaviors. - The permutations-based method will be extended to provide more information about the differences between windows, by looking at the contribution of individual days, hours, or even event sequences over the various permutations. **Local Interpretable Model-Agnostic Explanations (LIME)** ] --- name: closing background-image: url(images/usf-closing.jpg) class: middle right white no-slide-number .bigger[Thank You<br>Questions?] <div style="font-family: Merriweather; font-size: 1.25em; font-weight: 300; padding-top: 75px">Garrick Aden-Buie<br>garrick@adenbuie.com</div> --- class: title-slide center middle inverse-text exclude: true # Thank You ## Questions?<br>usf.edu/engineering/create-health ### Garrick Aden-Buie <br>garrick@adenbuie.com ### University of South Florida --- name: refs class: refs-size ### References 1. Mather, M., Jacobssen, L., & Pollard, K. M. (2015). *Aging in the United States*. 2. Pearson, W. S., Bhat-Schelbert, K., & Probst, J. C. (2012). Multiple chronic conditions and the aging of America: Challenge for primary care physicians. *Journal of Primary Care & Community Health*, *3*(1), 51–56. doi:[10.1177/2150131911414577](https://doi.org/10.1177/2150131911414577) 3. Demiris, G., & Hensel, B. K. (2008). Technologies for an aging society: A systematic review of "smart home" applications. *Yearbook of Medical Informatics*, 33–40. pubmed:[18660873](http://www.ncbi.nlm.nih.gov/pubmed/18660873) 4. Szewcyzk, S., Dwan, K., Minor, B., Swedlove, B., & Cook, D. (2009). Annotating smart environment sensor data for activity learning. *Technology and Health Care*, *17*(3), 161–169. doi:[10.3233/THC-2009-0546](https://doi.org/10.3233/THC-2009-0546) 5. Hamid, R., Johnson, A., Batta, S., Bobick, A., Isbell, C., & Coleman, G. (2005). Detection and explanation of anomalous activities: Representing activities as bags of event n-grams. *Proceedings - 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005*, *I*, 1031–1038. doi:[10.1109/CVPR.2005.127](https://doi.org/10.1109/CVPR.2005.127) 6. Sprint, G., Cook, D. J., & Schmitter-Edgecombe, M. (2016). Unsupervised detection and analysis of changes in everyday physical activity data. *Journal of Biomedical Informatics*, doi:[10.1016/j.jbi.2016.07.020](https://doi.org/10.1016/j.jbi.2016.07.020) 7. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. arXiv:[1602.04938](http://arxiv.org/abs/1602.04938) --- exclude: true ## Extra stuff - bag of event *n*-grams method was first validated as a supervised classification problem - activity profiles were used to identify occupants from 6 homes with identical layouts - Activity profiles based on 4 and 6 weeks of activity led to correct identification of a given occupant for unlabelled days of activity with high accuracy - unsupervised behavior change was applied to the activity data from four participants - over a period of one year (one participant) or two years (three participants) - who reported health changes ranging from acute episodes of illness to mobility restrictions leading to major surgery - Comparison of activity profiles over time windows of 1 to 4 weeks reliably detects major shifts in behavior ??? Points pulled from abstract