1. Wenn-Chieh Tsai, PhD Candidate, Graduate Institute Of Networking And Multimedia, National Taiwan University
Advisors: Jane Yung-jen Hsu PhD, Rung-Huei Liang PhD 2014.05.20
Technology Mediated Reminiscence
Designing Interactions with Digital Mementos

2. What is reminiscence?
2

3. 3
What is reminiscence?

4. 4
What is reminiscence?

5. Physical vs. Digital Mementos 5

6. Traditional/Tangible Memory Aids 6

7. 7
What if digital media are more than just a reflection?

8. Initial Questions
✤ How do we design a reminiscence aid with digital materials?
✤ How do we mediate reminiscence through digital artifacts?
✤ How do we extract significant mementos from personal collections?
✤ What is a preferred state of technology-mediated reminiscence?
8

9. Figure 1. An illustration of the pathways and deliverables between and among Interaction Design Researchers and
other HCI Researchers. The model emphasizes the production of artifacts as vehicles for embodying what “ought to
be” and that influence both the research and practice communities.
CHI 2007 Proceedings • Design Theory April 28-May 3, 2007 • San Jose, CA, USA
“how”
“true”
“real”
Zimmerman, J., Forlizzi, J., and Evenson, S. Research through design as a method for interaction design research in HCI. Proc. CHI 2007, 493–502.
Research for Design Research through Design
9
Our Approach

10. Expected Contributions
✤ Knowledge answering the question:
“How would you design a [reminiscence aid]?”
✤ A series of artifacts
✤ A concrete problem framing
✤ Articulation of the preferred state
✤ Documentation of the design process
Cross, N. Designerly Ways of Knowing. Springer, 2006.
10

11. Digital Memory Aids
✤ Utility-oriented Design
✤ “Memory Prosthesis”
✤ “Memory Augmentation”
Sparrow, B., Liu, J., and Wegner, D.M. Google effects on memory: cognitive consequences of having information at our fingertips. Science 333, 6043 (2011), 776–8.
11

12. Living Memory Box
Stevens, M.M. et al. Getting into the Living Memory Box: family archives & holistic design. Personal and Ubiquitous Computing 7, 3-4 (2003), 210–216.
authentic
12

13. Lifelogging
Sellen, A.J., Fogg, A., Aitken, M., Hodges, S., Rother, C., and Wood, K. Do life-logging technologies support memory for the past? Proc. CHI 2007, 81–90.
recollecting
13

14. CIRCA
Alm, N., Dye, R., Gowans, G., Campbell, J., Astell, A., and Ellis, M. A communication support system for older people with dementia. Computer 40, 5 (2007), 35–41.
objective facts
14

15. CaraClock
Uriu, D., Shiratori, N., Hashimoto, S., Ishibashi, S., and Okude, N. CaraClock. Proc. CHI EA 2009, 3205–3210.
indexical
15

16. FM Radio
Petrelli, D., Villar, N., Dib, L., and St, P. FM Radio : family interplay with sonic mementos. Proc. CHI 2010, 2371–2380.
evocative
16

17. Audiophotography
Frohlich, D. and Fennell, J. Sound, paper and memorabilia: resources for a simpler digital photography. Personal and Ubiquitous Computing 11, 2 (2006), 107–116.
reconstructive
17

18. Pensieve
Peesapati, S.T., Schwanda, V., Schultz, J., Lepage, M., Jeong, S., and Cosley, D. Pensieve: supporting everyday reminiscence. Proc. CHI 2010, 2027–2036.
dialogical
18

19. Definition
Memory Aid Reminiscence Aid
Interaction Attribute
Indexical Evocative
Interaction Attribute Monological DialogicalInteraction Attribute
Authentic Reconstructive
Output/Product Facts (Objective) Meaning (Subjective)
Human Experience Recollecting Interpreting/Reflecting
Research Approach Cognitive Psychology Ethnomethodology
19

20. A Preferred State
20
The Reflexive Printer

21. 21
It prints and deletes one digital photo from a personal archive everyday.

22. Users can have the thermal printout participating in their live,
22

23. 23
Restore its original back to the digital archive,

24. 23
Restore its original back to the digital archive,

25. 23
Restore its original back to the digital archive,

26. 24
Or, only keep the tangible embodiment in their lifeworld.

27. 3 Framing Artifacts, 3 Lessons
✤ How to initiate a reminiscence dialog? (The Retro Jukebox)
✤ Experiential resonance for being evocative
✤ How to intervene this dialog? (The Oblivescence Board)
✤ Self-presentation for being dialogical
✤ What to expect from this dialog? (The SoundTag)
✤ Alternative perspective for being reconstructive
25
Ylirisku, S., Halttunen, V., Nuojua, J., and Juustila, A. Framing design in the third paradigm. Proc. CHI 2009, 1131–1140.

28. 26
Reminiscence
Mode
interpersonal interpersonal intrapersonal
Digital Media music photo sound
Context hospital dorm room in the wild
Participant
14 inpatients
(> 65 y/o),
1 research nurse
two 22 y/o female
close friends
10 young adults
Method
unstructured
weekly meeting
experimental
research
semi-structured
interview
Evaluation
usability testing,
user feedback
memory retention
event records,
user experience
HCI Paradigm pragmatic cognitive situated
Focus evocative dialogical reconstructive
The Retro Jukebox The Oblivescence Board The SoundTag

29. NSC 99-2218-E-002-002 Research Report 2008-2011
The Retro Jukebox
Music Listening in a Ward
27

30. How might we evoke a reminiscence dialog
through digital media?
28
The Retro Jukebox is a conversation aid used in bedside activity for elder inpatients and their care givers. In addition
to a music playing function, three cognitive (one categorization and two memorization) games are integrated in the
interaction design to maintain inpatients’ mental fitness.

31. Usability Testing
29

32. 30

33. Findings & Reflection
✤ Empathic resonance
✤ “I’m not feeling well!” (Patient)
✤ “I feel like a saleswoman every time I step into the ward with the
device.” (Research Nurse)
✤ Situated usage in wards
✤ impromptu, appropriated
✤ Music-mediated experiential resonance
✤ embodied rapport
31

34. CHI 2013
The Oblivescence Board
Photo Sharing in a Close Relationship
32

35. How might we shift responsibility from digital
to personal memory in a reminiscence dialog?
33
The Oblivescence Board is a digital memory board for people to remember more. The photos shared on it will fade
over time according to a forgetting curve as human memory. The fading photos can be revived through users’ selective
viewing behavior. Our hypothesis is that a “forgetful” and self-presentative artifact would result in more users’
cognitive participation and retention of their valuable memories.

36. Transactive Memory System
Processor 1 Processor 2
Dir 1
Memory 1
Dir 2 Dir 2
Memory 2
Dir 1
Wegner, D.M.A computer network model of human transactive memory. Social Cognition 13, 3 (1995), 319–339.

37. Our Model
35

38. Ebbinghaus Forgetting Curve
(Min.)
(%)
self-presentation of forgetting
being reminded
Ebbinghaus, H. Memory: a contribution to experimental psychology.Teachers College, Columbia University, 1913.

39. General Hypothesis
• If the limitation of memory of a system is
embodied in a reminiscence dialogue,
transactive memory would be formed and
encourage users to remember more
37

40. Field Experiment
38

41. Findings
• Keeping memories on fading photos
no significant memory decay (Z=-1.890, p=.059>.05)
• Losing memories on constant photos
significant memory decay (Z=-2.271, p=.023<.05)
• Tapping more on fading photos
t=-12.248, p=.000<.01
• Viewing correlates with event richness
r=.468, p=.037<.05
39

42. Reflection
✤ Limitations in our experimental approach
✤ ownership of the system
✤ fixed dialogical pattern
✤ Require a situated approach to unfold the phenomenon and felt
experiences in one’s life
40

43. DIS 2012, IASDR 2013
The SoundTag
Soundscape Association in Everyday Life
41

44. How might we provoke self-reflection and
reinterpretation on personal memories?
42
The SoundTag, a self-associative artifact, is used to explore the design space of using sounds as tags to annotate and
associate event records. It intends to provide alternative perspectives for personal reminiscence in everyday life. We
utilize timbre as the metric to determine the similarity between soundscapes of events and implement a proof-of-
concept on smartphones.

45. What if we tag the world with sound?
43

46. Human Artifact
Event Cluster
Reminiscence Dialog
44
Brown, N.R. and Schopflocher, D. Event clusters: an organization of personal events in autobiographical memory. Psychological Science 9, 6 (1998), 470–475.

47. Human Artifact
Event Cluster
Auditory-Association
Reminiscence Dialog
44
Brown, N.R. and Schopflocher, D. Event clusters: an organization of personal events in autobiographical memory. Psychological Science 9, 6 (1998), 470–475.

48. Human Artifact
Event Cluster
Reminiscence Dialog
45

49. The SoundTag
a self-associative artifact 46

50. 47
user-subjective
Deterministic
system-subjective
Random
intersubjective
Self-Associative

51. Findings & Reflection
✤ Encoded personal meaning in sound tags
✤ System limitation increase sensibility in situ
✤ Internalization of self-association
✤ Personal value on digital mementos
48

52. Technological Imagination?
"…cultivating and shaping the technological imagination is a cultural imperative of the
highest order" (Balsamo, 2011)

53. MoreThan Physical Form
50
Vallgårda, A. Giving form to computational things: developing a practice of interaction design. Personal and Ubiquitous Computing 18, 3 (2013), 577–592.

54. The Design Space
Interaction
Attribute
Evocative Dialogical Reconstructive
Intended
Outcome
Self-initiation
Responsibility-
Shifting
Reinterpretation
Human
Value
Empathy Subjectivity Reflexivity
Technological
Mediation
Experiential
Resonance
Self-presentation
Alternative
Perspective
Design
Quality
Familiarity,
Serendipity
Extemporaneity,
Negotiability
Ambiguity,
Perceived
Drawbacks
51

55. DIS 2014
The Reflexive Printer
Perceived Drawbacks in Technology-Mediated Reminiscence
52

56. 53

57. 3Themes (Norms)
✤ The normality of digital photo consumption:
✤ Powerful Artifact and Feeble User
✤ Fast Consumption and Slow Rumination
✤ Simple Materiality and Monological Performance
54

58. 3 Designed
Drawbacks
✤ One-bit Halftone
✤ memory recollection
✤ Slow Rumination
✤ memory storage
✤ Material Decay
✤ memory retention
55

59. Prototype for Probing
✤ 4 Users
✤ 1 Week
✤ Initial & Final Interviews
✤ Diary Method
56

60. 57

61. 58

62. 59

63. Lessons Learned
60
✤ Just Enough Information
✤ Feeling of Being Accompanied
✤ Biomorphic Form

64. 61

65. 62
Field Study
✤ Two (one male and one female) young adults
✤ Digital photo natives
✤ Retrospective interviews (elicited by diaries)

66. ParticipantYT
2013.12.10
installation
in YT’s room
12.12 12.20
1st week interview
& app reinstall
3rd week
interview
12.25
double
prints
2014.1.3
(print at 10:00pm) (print at 9:00am)
!
63
initial
interview

67. ParticipantYJ
initial interview
(pilot study)
2013.11.14
installation
in YJ’s room
12.25
2nd week
interview
2014.1.6
pilot final
interview
11.26
(print at 8:00am)
64

68. Findings
Humble form and function initiate
intersubjective dialog
! 65

69. ✤ “The photo was printed when I woke up, and a series of characters were printed:
‘UUUUUU.’ It feels like it was talking to me by saying ‘you you you you you’, I
was like, are you talking to me?” (YT)
✤ “It is not a robot, and it is for sure not a printer. Oh, yes, it should have a
‘wanting to communicate with me about something’ kind of setting.” (YJ)
66

70. Findings
Minimal information encourages active and
personal interpretation
67

71. ✤ “Although it is in black and white, it enables me to recall the colors and sound
from the memory. [...] It is easier for me to recall the feeling of that moment when
the photo is blurry. [...] That feeling is like, there is no word to explain or be
written, maybe, it is just different from the real photo. Maybe it is the
medium.” (YT)
68

72. Findings
Materiality enriches the context for
reminiscence
69

73. ✤ “I’ll bring it with me. I was thinking that maybe when I am free I could just
conveniently take it out [of my pocket], because if I put it on the shelf, it could too
easy get blown away by the wind.” (YJ)
✤ “I could simply take it from my pocket, but then I find out that it is not in these
pants, so I have to go back and look for it” (YJ).
70

74. Findings
Embodiment anchors expectations in the
routine grounds of everyday activities
!
71

75. ✤ “I imagine it as something that is always there. It gives me a sort of expectation
and curiosity to find out which photo will be printed. And it could become
something that I can interact with when I am free, or put aside when I am
busy.” (YJ)
✤ “In the morning, I thought I heard the printing sound from the machine. I felt
excited, but I continued my sleep. When I woke up, I found no printed photos—it
was like I was dreaming” (YT)
72

76. Contributions
✤ 3 framing artifacts and 1 design example
✤ The design space of technology-mediated reminiscence
✤ 1 experiential design quality
✤ Design implications for reminiscence aids
73

77. DemoVideo
The Preferred State

78. https://docs.google.com/file/d/0B-rOc4S9n-4yeFVVR0RZZVRiaVE/edit

79. The Design Space
Evocative Dialogical Reconstructive
Intended
Outcome
Self-initiation
Responsibility-
Shifting
Reinterpretation
Human
Value
Empathy Subjectivity Reflexivity
Technological
Mediation
Experiential
Resonance
Self-presentation
Alternative
Perspective
Design
Quality
Familiarity,
Serendipity
Extemporaneity,
Negotiability
Ambiguity,
Perceived
Drawbacks
76

80. Related Publications
77
✤ Tsai, W.-C. (2014). Technology-mediated reminiscence: designing interactions with digital mementos
(Doctoral dissertation, National Taiwan University, Taiwan).
✤ Tsai, W.-C., Wang, P.-H., Lee, H.-C., Liang, R.-H., and Hsu, J.Y. (2014, accepted). The Reflexive Printer:
toward making sense of perceived drawbacks in technology-mediated reminiscence. In proceedings of the
10th ACM Conference on Designing Interactive Systems (DIS 2014).
(Full paper oral presentation)
✤ Tsai, W.-C., Hsiao, J.C.-Y., Lee, H.-C., Huang, C.-H., Hu, J.-C., Liang, R.-H., and Hsu, J.Y. (2013). Designing
a reminiscence aid in personal soundscape. In proceedings of the 5th International Congress of International
Association of Societies of Design Research (IASDR 2013), 5150–5160.
(Full paper oral presentation)
✤ Tsai, W.-C., Lee, H.-C., Hsiao, J.C.-Y., Liang, R.-H., and Hsu, J.Y. (2013). Framing design of reminiscence
aids with transactive memory theory. In extended abstracts of the 2013 ACM Conference on Human Factors in
Computing Systems (CHI 2013), 331–336.
(Work-in-Progress extended abstract and poster presentation)
✤ Tsai, W.-C., Hsiao, C.-Y., Lee, H.-C., Huang, C.-H., and Hsu, J.Y. (2012). In search of lost sounds: designing
a reminiscence aid in everyday soundscape. In poster abstracts of the 9th ACM Conference on Designing
Interactive Systems (DIS 2012).
(Poster abstract and poster presentation)

81. Q & A
78