Memento

Imagine being told that within the next ten years, you'll be at risk of forgetting the memories that make you, you. Someone is faced with this life changing realization every 70 seconds. Our lives are filled with defining moments. At any given point, we have the ability to recall these moments and reflect upon them, reaffirming our sense of self and giving meaning to the present.

Those faced with Alzheimer's disease may lose access to neural pathways that store and recall those memories, making the act of memory recall nearly impossible. By understanding how the brain stores and recalls memories using the aid of external stimuli such as music, photos, and video, we can begin to craft new therapy experiences that are grounded in a patient's personal history, assist in memory maintenance, and provide valuable benchmarking for caregivers.


What is Alzheimer's disease? 

  • One in eight older Americans has Alzheimer's disease.
  • Alzheimer's disease is a type of dementia that causes problems with memory, thinking, and behavior
  • In mid to late stages of the disease, one may not remember familiar people, places, or things, caused by progressive damage to brain cells and dimmed neural pathways.
  • Alzheimer's disease can also affect the areas of the brain that facilitate speech and motor skills.
  • In addition to the neurological and physical symptoms, many dealing with Alzheimer's Disease also suffer from severe depression stemming from greatly reduced social interactions, loss of sense of self, and a feeling of low self-worth.

How does Alzheimer's Disease affect memory?

The brain has two primary systems of storing and recalling memories:

Audio source: Ed Ferrigan, "Implicit and Explicit Memory", https://www.youtube.com/watch?v=o-W6TDYi0Cw

Edited and animated by Joe Bradford under Fair Use of the source material.

Audio source: Ed Ferrigan, "Implicit and Explicit Memory", https://www.youtube.com/watch?v=o-W6TDYi0Cw

Edited and animated by Joe Bradford under Fair Use of the source material.

Takeaways:


what kind of therapies are currently in place for those suffering from Alzheimer's Disease?

There are no cures for Alzheimer's disease, however there are a variety of therapy options tailored to slow the progression of certain symptoms for those suffering from Alzheimer's disease. Here are three of the most commonly administered therapies:

  • LSVT Big and Loud -  Lee Silverman Voice Treatment is a treatment for speech and movement disorders associated with Parkinson's disease and is often used for those suffering from late stage Alzheimer's disease. The therapy is designed to increase the speech and motor skill faculties in the brain by administering a series of vocal and physical exercises, administered by a speech therapist and physical therapist, given four times a week over four weeks. The therapy is supplemented with family support and guidance on diet.
  • Musical Intonation Therapy (MIT) - a targeted therapy that utilizes melody and rhythm to improve speech output, often used for those suffering from aphasia (loss or severe impairment of speech faculties). 
  • General music therapy - Usually held as group classes with varying degrees of musical participation (active listening, playing along with simple percussive instruments, singing, and dancing) is shown to improve memory recall, greatly reduce anxiety and depression, and promote vocal fluency among other positive emotional changes.

LSVT Big and Loud

During my research, I had an opportunity to shadow two patients at a local rehabilitation hospital. Below are summaries of the two parts of the LSVT Big and Loud program.

Music Therapy

Sensory-based cognitive processing, like listening to (and singing along with) a song from your past can create high levels of bihemispheric stimulation, enabling the brain to route around dead areas and access alternative neural pathways to parts of the brain previously inaccessible. Below is a clip from the documentary Alive Inside, which shows the transformative power of music on those suffering from symptoms like aphasia (loss of speech) and depression.

Above: Alive Inside clip of Henry

Alive Inside documents the neurological healing potential that music has on those with late stage Alzheimer’s.

Above is an example of the emotive and physical responses our bodies have to music. Each response activates a different part of our brain.

Music increases brain activity

"Researchers in Finland using magnetic resonance imaging (fMRI) found that music listening recruits not only the auditory areas of the brain, but also employs large-scale neural networks. For instance, they discovered that the processing of musical pulse recruits motor areas in the brain, supporting the idea that music and movement are closely intertwined. Limbic areas of the brain, known to be associated with emotions, were found to be involved in rhythm and tonality processing. Processing of timbre was associated with activations in the so called default mode network, which is assumed to be associated with mind-wandering and creativity. Music also engages areas of the brain involved with paying attention, making predictions and updating events in memory. Findings suggest that music can help the brain organize incoming information. Recent research also showed that listening to music releases dopamine (a neurotransmitter) in the brain sending pleasure signals to the rest of the body." - Dementia care, Alzheimer's Association

The past which is not recoverable in any other way is embedded, as if in amber, in the music, and people can regain a sense of identity
— Oliver Sacks, M.D., Author, Neurologist

Takeaways:

  • In many cases therapies are a patient's only form of socializing, as those suffering from late stage Alzheimer's may not feel as though they can relate or contribute to many conversations due to their inability to recall the memories that form their sense of self -- these people may close off emotionally and suffer from depression, which may negatively affect their speech and movement leading to further complications like aphasia.
  • Music therapy can have impressive results regarding memory recollection, emotional wellbeing, speech, and movement, and has a measurable output through PET scans.
  • Programs like LSVT are effective, yet impersonal forms of therapy hyper focused on speech and movement.
  • Currently there are no therapies that exist with the intention of helping a patient recall specific events or memories throughout the progression of the disease, though families and caregivers are often encouraged to evoke memories by playing music, showing photos to gently remind the person of important people and places.

How does the brain process external stimuli, like music?


Can we route around dimmed neural pathways with aide from external stimuli?

Memento Content v2-05.png

How can we synthesize this data to inform the design of a product or system? 

With lots of new information gathered about Alzheimer's disease, memory, and existing therapies, we can surmise the following:

  • Our sense of self stems from the memories we store in the explicit memory system of our brain.
  • Alzheimer's disease affects our ability to recall explicit memories due to certain areas of the brain suffering from dimmed neural pathways caused by plaques formed in the neural networks of our brain.
  • By leveraging the neuroplasticity of the brain via repetitive bihemispheric stimulus (repetition priming), we have the ability to take memories from the explicit memory system and build new neural pathways encapsulating them in our implicit memory system
  • Music, and other memory-evoking objects can produce high levels of bihemispheric stimulation, providing new neural networks to route around dimmed areas of the brain. 
Brain-17.jpg

How can we create a personalized therapy, based on one's own memories, that encourages neuroplastic growth in the brain through the act of practicing memory recollection?

What is Memento?

Memento is a digitally integrated memory bank which affords a targeted therapy based in one's own memories, designed to slow the progression of Alzheimer's disease by encouraging neuroplastic growth in the brain.

Memento is comprised of a system of objects: RFID picture frames (on magnetic wall mounts), a centralized RFID reader hub, a television or tablet, and a structured session of interaction with the system of objects.

Each Memento picture frame displays a photo representing a specific event or memory. Additional media that reminds one of that event, such as video, additional photographs, music, images of people that are relevant to that memory, etc, are digitally linked to the photo frame via tagging the embedded RFID chip through an app.

When a Memento frame is placed on the RFID hub, a ring of LEDs illuminates, signifying that the memory is being accessed, and a simple interface is displayed on a designated display (television, monitor, or tablet). The user may then access the media easily, centralizing all of the digital assets that remind us of a specific event, and creating a meaningful memento.

To encourage movement, a Microsoft Kinect may be introduced to the system to enable navigation through gross body movement. The user's hand is tracked by the connect, and they simply need to extend/stretch their arm in the direction of the media they would like to interact with.

What does a therapy session with memento look like?

Those with Alzheimer's can set aside time each week to practice the recollection of specific memories via Memento. With the aide of a trained therapist, caregiver, or family member, a structured therapy can be administered which involves the following steps:

  • Before interacting with Memento, a patient is asked to try to recall an event or memory and describe specific details about that event or memory, such as trying to remember who was present, the location, year, and season the event took place, how the event or memory came to happen, and why it is important to that person. A caregiver will note the level of detail that the patient is able to achieve in their recollection.
  • Patients and caregivers then spend time interacting with Memento in an unstructured format, where the patient is encouraged to experience the media associated with the memory in a free and creative way (active and passive music interaction, playing 'games' like trying to name the person on the screen or location seen, singing and dancing to music that may be associated with the event, etc), while the caregiver listens, prompts for additional details, and notes the level of detail the patient is able to recall.
  • After the caregiver will ask the same questions as before the session, noting if the level of detail matched or surpassed that of the prompts before the Memento session.
  • Sessions can be compared week over week, month over month, for insight into the memory faculties. In the process, the act of repetitive priming of cognitive stimulus may promote neuroplastic growth and strengthen alternative pathways for the brain to access these memories outside of the Memento session. 

Memento is a therapy which: 

  • Collects and centralizes digital assets like video, photos, and music that reminds us of an event or memory, providing a point of reference to the self
  • Assists in memory maintenance and strengthening of neural pathways in the brain
  • Can help track the progression of the disease by affording systematic benchmarking to caregivers

 

In this works-like prototype, RFID chips are embedded into handmade picture frames. When a frame is placed on the Memento RFID smart hub, a command is wirelessly transmitted to a linked computer which pulls up a custom coded web page on the TV. The user may then access the media through body movement tracked on a Microsoft Kinect. In this early code, the right hand was tracked as the cursor and raising the left hand was tracked as a click.

Prototype screens:

Example of a memory/event that may be selected to be stored on a Memento frame. Here we have an event: a special vacation, location: The White Mountains, a year: 1964, music that is associated with the memory, and specific people that can be used as prompts during a Memento therapy session.


User Interaction


Exploring iterations of the interface

Working through early form and user interaction with a rough demo in Keynote. A goal was to -- trying to see how to incorporate both movement and speech into the system of objects, as both movement and speech are important components to the therapy program LSVT Big and Loud.

Splash Screen exploration in Illustrator.

Process booklet

Memento Content v2-09.png