O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Create Engaging Healthcare Experiences with Augmented Reality

136 visualizações

Publicada em

Would you like to try hands-on how to create your first Augmented Reality app? In this session, you will see how easy it is to use 3D technologies and automated speech processing (through Amazon Sumerian) to explain healthcare-related topics. The session is defined for health professionals with little or no prior experience in app design, who would love to share their knowledge with new didactic methods. Additionally, we will provide a behind-the-scenes look on the use case “Enlightening Patients with Augmented Reality” within the funded research project “Immersive Media Lab”.

Presented at the build.well.being conference on June 14th, 2019:
https://buildwellbeing.fhstp.ac.at/

More information about the contents:
https://www.andreasjakl.com/

Publicada em: Tecnologia
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Seja a primeira pessoa a gostar disto

Create Engaging Healthcare Experiences with Augmented Reality

  1. 1. St. Pölten University of Applied SciencesSt. Pölten University of Applied Sciences Platzhalter für möglichen Bildeinsatz Create engaging Healthcare Experiences with Augmented Reality Andreas Jakl – www.andreasjakl.com build.well.being 2019 Digital Healthcare FH St. Pölten Platzhalter für möglichen Bildeinsatz
  2. 2. Andreas Jakl ▪ Focus areas ▪ AR / VR, mobile apps, sensors, interaction technology, software architecture, open source developer (NFC, Bluetooth Beacons) ▪ Microsoft MVP (Most Valuable Professional) & Amazon AWS Cloud Ambassador ▪ mobility.builders community: Mobile Developer After-Work Events ▪ Previous Experience ▪ Tieto, Start-up (Mopius), Nokia (Finland), Siemens Mobile (Munich), FH Hagenberg (Mobile Computing) Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten https://www.andreasjakl.com/ andreas.jakl@fhstp.ac.at @andijakl
  3. 3. WHAT IS AR / VR / MR? Overview & Intro Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  4. 4. Virtual Reality? ▪ Dive into another reality ▪ Mainly visual + acoustic stimulation ▪ Immersion & presence ▪ Feeling to be at another place Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Screenshot: Ready Player One, 2018
  5. 5. Augmented und Mixed Reality ▪ Virtual Reality ▪ Only virtual, completely hides reality ▪ Augmented Reality ▪ Visible world gets digital overlays ▪ Mixed Reality ▪ Virtual content seamlessly blends into the real world Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Wired.com
  6. 6. Microsoft HoloLens 2 Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten https://www.hololens.com/
  7. 7. Apple ARKit Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten https://developer.apple.com/augmented-reality/
  8. 8. Google DayDream / ARCore Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Google DayDream https://vr.google.com/daydream/ Google ARCore https://developers.google.com/ar/
  9. 9. Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Source: CCS Insight VR and AR forecast (November 2018) https://www.ccsinsight.com/press/company-news/3726-virtual-reality-device-market-declines-in-2018-but-outlook-remains-positive Shipments of VR and AR devices, worldwide, 2015-2022
  10. 10. AR / VR Estimates: 2025 Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Source: Goldman Sachs Global Investment Research January 13, 2016
  11. 11. Hype & Reality Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Source: Gartner https://www.gartner.com/smarterwithgartner/5-trends-emerge-in-gartner-hype-cycle-for-emerging-technologies-2018/
  12. 12. Digital Healthcare Explained Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten App created by Digital Healthcare @ FH St. Pölten wins global Amazon AR/VR challenge in the category: Best AR App! https://devpost.com/software/digital-healthcare-explained-czflve https://www.fhstp.ac.at/en/newsroom/news/best-augmented-reality-app-in-amazon-hackathon
  13. 13. Immersive Media Lab ▪ Storytelling in AR / VR ▪ Use case: “Enlightening Patients with Augmented Reality” ▪ Explain corrective eye surgery ▪ Pre-operation, during operation, afterwards ▪ Study acceptance of different AR interaction methods Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  14. 14. AMAZON SUMERIAN Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  15. 15. Amazon Sumerian Overview ▪ Interactive 3D scenes ▪ Browser-based 3D editor + viewer (WebGL / WebXR) ▪ Supports VR & AR ▪ Special feature: ▪ Animated avatars (“hosts”) ▪ Speech + gestures ▪ Supports conversations ▪ https://aws.amazon.com/sumerian/ Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  16. 16. AWS Account ▪ AWS = Amazon Web Services ▪ Sumerian ▪ Free tier for 12 months https://aws.amazon.com/free/ ▪ 5 GB scene traffic free ▪ Credit card required for sign-up, but not charged! ▪ Pricing after 1 year / exceeding free tier limits ▪ 3D object storage, scene traffic, speech services Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  17. 17. Scene Setup ▪ Log in to Amazon Sumerian Console ▪ https://console.aws.amazon.com/sumerian/home 1. Choose world-wide region for your service 2. Create “Default Lighting” scene Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  18. 18. 3D Editor Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Canvas / 3D view Click: select 3D object Shift + left mouse: move view Right mouse: rotate view Entities: everything in your scene Assets: currently available content Inspector: properties of selected part Top menu: new 3D models, publish scene
  19. 19. First 3D Object Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten 1) Import Assets 2) Select Table Cafe 3) Add
  20. 20. Add to Scene ▪ Drag & Drop Model from Assets to Entities ▪ Then wait a few seconds… ▪ Ensure object is visible. Some tips: ▪ Zoom in with mouse wheel ▪ Select object and press “f”-key to center Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  21. 21. Publish! ▪ Link to your scene ▪ Runs in every Chrome + Firefox! Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  22. 22. Custom 3D Models ▪ Free sources ▪ Remix 3D (Microsoft): https://www.remix3d.com ▪ Poly (Google): https://poly.google.com/ ▪ Sketchfab: https://sketchfab.com/features/download ▪ Own 3D Models ▪ Paint 3D & 3D Builder (Windows 10) ▪ Blender ▪ Autodesk Maya / 3DS Max Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  23. 23. Import to Sumerian ▪ Human Heart ▪ From Remix3D (Microsoft)* ▪ Download adapted version: https://www.andreasjakl.com/ human-heart/ ▪ Import to Sumerian ▪ Drag from “Assets” to “Entities” ▪ Adapt scale & rotation ▪ Scale: 0.1, Rotate X: -90 Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten * https://www.remix3d.com/details/059f2766c027458787256ebb47a4094e Microsoft, Creative Commons CC-BY-SA
  24. 24. Adding Speech ▪ Conversational interfaces ▪ Amazon Lex: natural language understanding ▪ Amazon Polly: text-to-speech ▪ Different Amazon Services! ▪ Secure communication? ▪ Amazon Cognito credentials ▪ One-click setup: Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Amazon Alexa Built on these two services, adds “skills” concept, skill store, etc. https://docs.sumerian.amazonaws.com/tutorials/create/beginner/aws-setup/
  25. 25. Stack → CognitoIdentyPoolID Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten 3 Copy CognitoIdentyPoolID
  26. 26. Connect with Sumerian Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten Paste
  27. 27. Create Host Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  28. 28. Position Host ▪ Drag host from Assets to Entities ▪ Then wait a few seconds! ▪ Re-position host ▪ Press Play ▪ Host already moving naturally Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  29. 29. Let the Host Explain the Heart ▪ Edit the Speech settings in the inspector of your host ▪ (1) Choose voice: “Salli” is US English ▪ (2) Click on “+” in “Speech Files” to create a new file ▪ (3) Enter text ▪ (4) Save (Default name: “Speech”) Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten 1 2 3 4
  30. 30. Create Logic: When to Speak? ▪ (1) Choose “+ Add Component” ▪ “State Machine” ▪ (2) Add a new behavior (“+”) ▪ (3) Opens editor in bottom part Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten 1 2 3
  31. 31. Add Actions ▪ Actions define what to do in a situation (“state”) or what to wait for ▪ “Add Action” to default “State 1” ▪ “Click/Tap on entity” Hint: use search in top right corner Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  32. 32. Second State: Start Speaking! ▪ Add a second state ▪ What should happen after tapping the host? ▪ (1) + (2) “Add State” ▪ (3) Add action to “State 2”: “Start Speech” ▪ (4) Choose your “Speech” from drop-down Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten 1 2 3 4
  33. 33. Connect Everything ▪ Logic ▪ Clicking should trigger speech ▪ Speech end should go back to initial state ▪ Drag & drop from action box to new state ▪ Test! Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  34. 34. More Life-Like Host: Gestures ▪ Hand-Movements of the host while speaking ▪ Select host in “Entities” ▪ (1) Inspector: “Speech” → “Gesture Map” → “+” ▪ Triggers gestures based on words in spoken text ▪ (2) Auto-assign gestures to “Speech” file Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten 1 2
  35. 35. Publish & Test! Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  36. 36. Augmented Reality & More! ▪ Extended Tutorial ▪ Animation & Timelines ▪ Communicating within the scene ▪ Publish to Android & show as Augmented Reality Scene ▪ https://www.andreasjakl.com/amazon-sumerian- augmented-reality/ Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten
  37. 37. THANK YOU! Andreas Jakl Lecturer / Digital Healthcare @ FHSTP https://www.andreasjakl.com/ Create engaging Healthcare Experiences | 2019 | Andreas Jakl | FH St. Pölten

×