2. A B O U T M E
Sanjit Singh
Student, Hacker, Sub - Team lead @ NASA SUITS, Space Coach @ AFWERX,
Research Assistant @ Rutgers University & Mixed Reality Developer @ MPLEX
Co-Founder & Former VP, XBoost
Building the future – VR/AR, Robotics, Brain computer Interfaces,
Virtual Worlds, AI, Humanoids & more cutting-edge technologies.
Connect @ https://www.linkedin.com/in/sanjit-singh-66374b6a/
Follow @ https://twitter.com/sanjitpsingh
3. What is Virtual &
Augmented Reality
Virtual Reality: 3D generated virtual
environment in which users can interact with
as if they were in that simulation to gain
immersive experiences of there
surroundings.
Augmented Reality: 3D Computer generated
environment projected from the virtual
world onto the real world as holograms.
4. Early days Mixed Reality
Dr. Ivan Sutherland Morton Heilig
Dr. Tom Furness Dr. Grigore (Greg) Burdea Dr. Mark Billinghurst
5. Types of VR/AR Headsets
HoloLens
Magic Leap
Nreal
Project North Star
Oculus Rift &
Oculus Quest
Google Cardboard VR
HTC VIVE & Windows Mixed Reality Headset
12. Where The Future is Headed
• Metaverse – virtual world with artificial life (cyberspace) with users can interact with virtual agents
in a simulation
• Sci-fi fantasy into reality
13. Mixed Reality Toolkit (MRTK)
• MRTK – A library designed for XR developers use into there projects and provides set of
components for interaction, UI, rapid prototyping and more.
• Very cross platform
• Supports: Android, IOS, HoloLens, Oculus, Magic Leap, Nreal & more
• Game engines that use MRTK: Unity3d & Unreal
• Our focus will be Unity3D - No Unity or C# experiences needed
14. Unity
Installation
guide
• Unity Hub: https://unity3d.com/get-unity/download
• Create Unity Account
• Unity Version: 2021.1.24f1
• Note: iPhone users need to have MacBook to build on
IOS platforms along with XCode installed
• Android Users also have android studio installed with
API Level minimum set to 24 and API level highest to
latest version
21. MRTK Popup
• If all goes well with
importing, then MRTK popup
from image to left is what
should be see
• Click Unity OpenXR plugin
22.
23. Unity is Restarting
- Once restart completes, you will get another MRTK popup
- Click Show XR Plugin Management Settings
24. XR Plugin Management Settings
• Click on Android logo to the right (for android users)
• Check ARCore
25. Now you can click Skip this step
On the right side of image, there is apply default settings, Click Apply
Unity will restart again
26. Popup configuration shows up again. Click next
From Right side you will get importing TMP Essentials options
Click Import TMP Essentials
27. MRTK Import All Done
- After importing TMP Essentials,
you will get MRTK Setup Completed
message shown on the left
- Click Done
28. Building on Platform
• As mentioned, we will be building on Mobile Platform (Android)
• In Unity go to File-Build Settings
• In order to build apps, unity needs to know which scenes to build
• Click Add Open Scenes
• Then Click on Android under Platforms
• Click Switch Platform
• Note after platform switches to android incase you get Popups again from MRTK
configuration, just repeat it
29. Player Settings
• Player Settings-Android icon-Other Settings
• Right Side of player settings we see many options
• Make sure minimum API level is set to 24
• Under Graphic APIs click on Vulkan and underneath press
the minus button (that will delete Vulkan)
30. Setting up MRTK Camera
- On the left you have the original Unity Scene with the built-in main camera
- Right click on Main Camera under Sample Scene and click on Delete
- Go the very top left and look for Mixed Reality
- Click on that and then go to toolkit-Add to Scene and Configure
- You will get a new camera that comes with MRTK along with other gameobjects we will use to configure
31. MRTK Build
Configuration
• Click on MixedReality Toolkit
Gameobject under Hierarchy
• Go to the right and you will see
Experimental Settings
• On the top click on Clone
• You will have clone profile popup
• Under that go to profile name
• We will rename it
MobileMixedRealityToolkitConfiguratio
nProfile
• After doing that, click clone
32. • Click on Camera
• Do the same thing again
• Clone and then rename profile to
MobileMixedRealityCameraProfile
• Click Clone
33. MRTK Camera Settings
- Look for Camera Setting Providers to the
right
- You will the current camera providers set by
default
- To the right click the minus button for all 3
to delete them
34. - We will create new camera setting for mobile AR for MRTK
- Click Add Camera Settings Provider
- You will have a new data provider
- Click Type-Microsoft.MixedReality.Toolkit.Experimental.UnityAR-UnityARCameraSettings
36. Simple Build
• Go to Mixed Reality-Toolkit-Utilities-UnityAR-
Update Scripting Defines
• Connect your Phone to Computer
• Go to File-Build Settings-Build And Run
• If all goes well and it builds on phone it should
look like the picture from the right side
• Now we can start adding some interactable
features
38. • Under Sample Scene Delete MixedRealitySceneContent
• Left Click MixedRealitySceneContent Under HandInteractionExample
• Drag it on top of HandInteractionExample
• Right Click on HandInteractionExample under Hierarchy and click Remove Scene
• You will get prompted an option to save. Click Don’t Save as we do not want to override the scene
Adding Interactable
Objects
