Mobile Graphics (part1)

www.crs4.it/vic/
Visual Computing Group
Mobile Graphics
• Marco Agus
• Marcos Balsa
June 2015
1

Marco Agus & Marcos Balsa
Outline
• Part 1: Evolution of mobile graphics
• Part 2: Graphics development for mobile systems
• Part 3: Mobile graphics trends and real time visualization
of massive models on mobile systems
2

Part 1
Evolution of mobile graphics
3

Mobile evolution…. in movies
• Motorola DynaTac
• Nickname “brick phone”
• Weight: over 2 pounds
• Cost: thousands of dollars
• Battery life: around 35
minutes.
Money never
sleeps…..This is your
wake-up call, pal…
GO TO WORK
Wall Street, 1987
Michael Douglas in
Gordon Gekko
11

Mobile evolution… in movies
Hello Neo…
Do you know who
this is?
The Matrix, 1999
Laurence Fishburne in
Morpheus
• Nokia 8110
• Nickname “banana phone”
• 145g, display
monochrome, Smart SMS
• It costed 1000 eur
12

Skyfall, 2012
Daniel Craig in James
Bond
• Sony Xperia T
• Smartphone Android
• Display 4.6” 1280x720
• It costed 600 eur
• 13 Mpixel camera +
position sensors
16

Iron Man 3, 2013
Robert Downey Jr in
Tony Stark
• Future devices?
• Transparent and foldable
high resolution screens
• Gesture interfaces
• Wearable / integrated to
body
17

Mobile evolution... in games
• Nokia Snakes
– From 1997 an estimated
350 Mdevices, making it
one of the most widely
distributed games ever
created.
– Installed on Nokia devices
until 2007
19

Mobile evolution… in games
• Angry Birds (Rovio)
– first released for Apple's
iOS in December 2009
– 2 billion downloads across
all platforms
– widespread diffusion end
popolarity
– Adventure parks (Finland
and Malaysia)
20

Mobile evolution… in games
• Unreal Engine (GDC &
Google I/O 2014)
– running on an Nvidia Tegra K1
processor
– will support Google Tango and
Samsung Gear VR
– easy porting of games
– sophisticated 3d effects
21

Mobile connectivity evolution
• Bandwidth is doubling
every 18 months
• Mobile internet users
overcame desktop
internet users
• 2017 smartphone traffic
expected at 2.7 GB per
person per month
22

Displays and User Interface
• Before 2007 – old days
– PDA  Palm OS/ Windows Pocket / Windows CE
– Stylus interaction (touch screens at early stages)
• Touch era
– 2007 – iOS /iPhone
– 2008 – Android / HTC Dream or G1
– Touch-enabled devices (no stylus required)
• Nowadays
– Wearables  <2”
– Smartphones  3-6”
– Tablets  >7-10”
– DLP projectors integrated
23

Chip evolution (1/2)
25

Chip evolution (2/2)
26

• Modern smartphones (tablets) are
compact visual computing
powerhouses
• DIFFUSION: more than 4.6 billion
mobile phone subscriptions
– [Ellison 2010]
• NETWORKING: High speed internet
connection (typical 1GB/month plan)
– 3G - < 0.6-3Mbps ~ 100KB/s - 400KB/s
(latency ~ 100-125ms)
– 4G – < 3-10Mbps ~ 400KB/s - 1MB/s
(latency ~ 60-70ms)
– 5G - 1Gbps (from 2016?)
• MEMORY: Increasing RAM and
storage space
– RAM 1-3GB
– Storage 8-64GB
• COMPUTING: Increasing processing
power
– CPU 4-8 core @ 2.5Ghz
– GPU 72-192 cores (~ALUs)
Scenario
27

Where are we going?
• Powerful devices for acquiring, processing and
visualizing information
• Accessibility of information (anybody, any time,
anywhere)
• Immense potential (integration of acquisition,
processing, visualization, cloud computing, and
collaborative tasks)
29

www.crs4.it/vic/
Visual Computing Group
Mobile Graphics
• Marco Agus
• Marcos Balsa
June 2015
Development
30

Mobile Graphics
OS
architecture
programming
languages
3D APIs
IDEs
Heterogeneity
31

Mobile Graphics
OS
architecture
programming
languages
3D APIs
IDEs
Heterogeneity
32

Mobile Graphics
• OS
• Programming Languages
• Architectures
• 3D APIs
• Cross-development
– X86 (x86_64): Intel / AMD
– ARM (32/64bit): ARM + (Qualcomm, Samsung, Apple,
NVIDIA,…)
– MIPS (32/64 bit): Ingenics, Imagination.
– Android
– iOS
– Windows Phone
– Firefox OS, Ubuntu Phone, Tizen…
– C++
– Obj-C / Swift
– Java
– C# / Silverlight
– Html5/JS/CSS
– OpenGL / GL ES
– D3D / ANGLE
– Metal / Mantle / Vulkan (GL Next)
– Qt
– Marmalade / Xamarin /
– Muio
– Monogame / Shiva3D / Unity / UDK4 / Cocos2d-x
33

Operating Systems
34

Operating Systems
• Linux based (Qt…)
– Ubuntu, Tizen, BBOS…
• Web based (HTML5)
– ChromeOS, FirefoxOS, WebOS (deceased?)…
• Windows Phone
• iOS (~unix + COCOA)
• Android (JAVA VM)
http://www.theregister.co.uk/2014/08/04/android_beats_ios_for_first_time/
2014
35

Operating Systems
• Brief comparison (focus is research here  )
– Windows phone 8
+ Best IDE – Visual Studio (2013)
- Windows development (porting Linux code?)
- Market is quite restrictive (100$ /year. – 5 free uploads/year)
- Certification + review
- OpenGL support ? Through ANGLE (over D3D)
– iOS
+ Best devices ? (homogeneity, at least)
+ Good IDE – Xcode + clang
- Market is rather restrictive (100$/year)
- Review
+ OpenGL support – PowerVR GPUs only
– Android
+ Best platform ? (more open, more devices, more flexible ?)
- Many IDEs – integration is not great (~tricky)
+ Visual Studio / Eclipse / QtCreator
+ With GCC / clang compiler
+ Market is very accessible (25$)
+ OpenGL support / OpenCL support
Monetization?
Research?
Monetization??
42

Programming Languages
http://www.tops-int.com/blog/which-programming-languages-are-used-for-web-desktop-and-mobile-apps/
43

• C/C++
– Classic, performance, codebase, control
• Objective C
– Bit different style (message based), well-documented API for iOS, mainly
COCOA/iOS
• Java
– Android is VM/JIT based, ~portability (API), well-known, extended,
codebase
• C#
– VM based, ~Java evolution, MONO (Win, Android, iOS)
• Swift
– Apple new language, simplicity, performance, easy, LLVM-based compilers
• HTML5/JS
– Web technologies, extended, compatibility
• Perl, Python, Ruby, D, GO (Google), Hack (facebook), …
– More options, not so popular ?
44

• C/C++
– Classic, performance, codebase, control
• Objective C
– Bit different style (message based), well-documented API for iOS, mainly
COCOA/iOS
• Java
– Android is VM/JIT based, ~portability (API), well-known, extended,
codebase
• C#
– VM based, ~Java evolution, MONO (Win, Android, iOS)
• Swift
– Apple new language, simplicity, performance, easy, LLVM-based compilers
• HTML5/JS
– Web technologies, extended, compatibility
• Perl, Python, Ruby, D, GO (Google), Hack (facebook), …
– More options, not so popular ?
45

Architectures
ARM
47

Architectures
CPU architectures
X86 – ARM – MIPS
48

Architectures
• x86 (CISC 32/64bit)
– Intel Atom Z3740/Z3770
• Bay Trail (2W)
– AMD Mullins (not yet in the market)
• 4.5W
-Power consumption
+Performance PartOf(desktop class GPU!)
+compatibility with old SW ?
49

Architectures
• x86 (CISC 32/64bit)
– Intel Atom Z3740/Z3770
• Bay Trail (2W)
– AMD Mullins (not yet in the market)
• 4.5W
• ARM
– RISC 32/64bit
• MIPS
– RISC 32/64bit
– Acquired by Imagination, Inc. @2014
-Power consumption
+Performance PartOf(desktop class GPU)
+compatibility with old SW ?
+Power efficiency
+Performance/watt
+Smaller area (RISC)  lower cost
+demonstrated its capacities on consoles
(PS/PS2/PSP/N64/Wii…), also on SGI 
50

Architectures – RISC vs. CISC but…
• CISC (Complex Instruction Set Computer)
– Fast program execution (optimized complex paths)
– Complex instructions (i.e. memory-to-memory instructions)
• RISC (Reduced Instruction Set Computer)
– Fast instructions (fixed cycles per instruction)
– Simple instructions (fixed/reduced cost per instruction)
• FISC (Fast Instruction Set Computer)
– Current RISC processors integrate many improvements from
CISC: superscalar, branch prediction, SIMD, out-of-order
– Philosophy  fixed/reduced cycle count/instr. (SIMD?)
– Discussion (Post-RISC):
• http://archive.arstechnica.com/cpu/4q99/risc-cisc/rvc-5.html
51

Architectures
RISC Integrate complex instructions ARM / MIPS
CISC Reduce instruction complexity Intel Atom
MMX/SSE/Out-of-Order
52

Architectures – X86
• Intel (32/64 bit)
– Competitive with Bay Trail Atom Z3470 ~4W
– Pursuing low power consumption instead of performance
– GPU: Intel HD graphics for Bay Trail ~ GF 8600M GT | GF210
– Present in many tablets (i.e. Surface) with Windows Phone/Android
– Present in a few smartphones
• AMD
– Not yet competitive in low power > 4W 
– Good GPU performance (GCN 192 core)
– No known smartphone/tablet shipped
• Supported on
– Android, Windows Phone, Tizen, Firefox OS, Ubuntu Touch,…
57

Architectures – ARM
• ARM Ltd.
– RISC processor (32/64 bit) – getting to 64bit ~ 2014/15
– IP (intellectual property) – Instruction Set / ref. implementation
– CPU / GPU (Mali)
• Licenses (instruction set OR ref. design)
– Instruction Set license -> custom made design (SnapDragon,
Hummingbird in iPhone 4 & Galaxy S)
• Optimizations (particular paths, improved core freq. control,…)
– Reference design (Cortex A9, Cortex A15, Cortex A53/A57…)
• Licensees (instruction set OR ref. design)
– Apple, Qualcomm, Samsung, Nvidia, MediaTek, AMD @<2014…
– Few IS licenses, mostly adopting reference design
• Manufacturers
– Contracted by Licensees
• GlobalFoundries, United Microelectronics, TSM, and Intel (@2013) 
59

Architectures - MIPS
• MIPS
– RISC processor (32/64 bit)
– IP (intellectual property) – licensing
– Recently acquired by Imagination, Inc.
– Can provide full solution (SystemOnChip, SoC): wireless/cpu/gpu
• Performance/watt should be comparable to that of ARM
• GPU from Imagination have demonstrated its value
– iDevices have always included its PowerVR SGX / Rogue cores
– Good integration with CPU and other components on SoC could
provide a very competitive solution (i.e. Qualcomm)
• Supported on
– Android, Mer (fork from MeeGo)
• Knowledge from previous HW (PSP, PS, PS2, WII…)
– Pretty much the same with ARM HW
61

Architectures
GPU architectures
64

Tessellation & Geom. Proc.
Architectures - GPU
Primitives
Vertex
Processing
Primitive
Assembly
Rasterization
Pixel
Processing
Framebuffer
Operations
Vertex
Shader
Geometry
Shader
Fragment
Shader
Tessellation
Eval./Control
Shader
Image courtesy of: http://rnd.azoft.com/fluid-dynamics-simulation-on-ios/
Simplified OpenGL 3D pipeline (ES 2.0  3.0)
Fixed hardware:
Vertex + Pixel Shaders
GPU
Unified Shaders:
Vertex/Pixel -- Compute
Desktop
65

Architectures - GPU
Device CPU GPU Gflops Gflops FP32 Cores/ALUs
Samsung Glxy Nexus (2011)
TI OMAP 4460
2-core A9
PowerVR SGX 540 4,8 0,01 8 cores
Adv. chinese dev (2014)
MediaTek MT6589T
4-core A7
PoverVR SGX 544MP2 9,6 0,03 16 cores
Apple iPhone 5 (2012)
Apple A6
2-core (armv7)
PowerVR SGX 543 MP3 28,8 0,08 48 cores
2013 Android flagship (HTC One, Galaxy
S4)
Snapdragon 600
4-core Krait 300
Adreno 320 51,2 0,14 64 cores
Avg Tegra 4 device (2014) (4+1)-core A15 GeForce ULP 96,8 0,20 24VS + 48PS= 72 cores
Apple iPad 4 (2012)
Apple A6x
2-core (armv7)
PowerVR SGX554 MP4 76,8 0,21 128 cores
Apple iPhone 5s (2014)
Apple A7
2-core (armv8)
PowerVR G6430 115,2 0,21 128 cores
2014 Android flagship (Galaxy S5)
Samsung Exynos 5
4-core A7 + 4 core A15
ARM Mali T628 MP6 102,4 0,28 96 cores
2014 Android flagship (Galaxy S5)
SnapDragon 801
4-core Krait 400
Adreno 330 166,5 0,46 128 cores
Sony PS 3 (2006) PowerPC 1 core + 7 SPE Nvidia G70 228,8 0,63 8VS + 24PS=136 cores
XBOX 360 (2005) 3-core x86_64 ATI R500 Xenos 240,0 0,66 192 cores
MiPad, Project Tango, HTC Volantis (2014) 4+1 cores A15 Nvidia Tegra K1 326 1,00 192 cores
One Plus 2
SnapDragon 810
64bit 4xA53 + 4xA57
Adreno 430 324-388 1,0 256 core
Core ~ #ALU/MADDs
68

Architectures - GPU
Desktop ~ 2880 cores (GTX780i) ~5000 Gflops
Vs
Mobile ~ 256 cores (Tegra X1) ~512 Gflops @ FP32
PS4 ~ 1840 cores
XBOX ONE ~ 1240 cores
69

Architectures – GPU
• Immediate Mode Rendering (IMR)
• Tile Based Rendering (TBR)
• Tile Based Deferred Rendering (TBDR)
70

• Inmediate Mode Rendering (IMR)
– Geometry is processed in submission order
• High overdraw (shaded pixels can be overwritten)
– Buffers are kept in System Memory
• High bandwidth / power / latency
– Early-Z helps depending on geometry sorting
• Depth buffer value closer than fragment  discard
http://blog.imgtec.com/powervr/understanding-powervr-series5xt-powervr-tbdr-and-architecture-efficiency-part-4
VS FS
71

• Tile Based Rendering (TBR)
– Rasterizing per-tile (triangles in bins per tile) 16x16, 32x32
• Buffers are kept on-chip memory (GPU) – fast!  geometry limit?
– Triangles processed in submission order (TB-IMR)
• Overdraw (front-to-back -> early z cull)
– Early-Z helps depending on geometry sorting
72

• Tile Based Deferred Rendering (TBDR)
– Fragment processing (tex + shade) ~waits for Hidden Surface Removal
• Micro Depth Buffer – depth test before fragment submission
– whole tile  1 frag/pixel 
• iPAD 2X slower than Desktop GeForce at HSR
(FastMobileShaders_siggraph2011)
– Possible to prefetch textures before shading/texturing
– Hard to profile!!! ~~~Timing?
Limit: ~100Ktri + complex shader
73

Architectures – Power consumption
• Reduce working set  Tiling
• Optimize bandwidth  Deferring
• Minimize area/circuitry  RISC?
Power consumption by memory access
Courtesy of: Shebanow – HPG 2013 keynote
Shared memory  Fight for the bus!
BUT
Less CPU  GPU copies! (expctd.)
75

• General issues
– Shared memory – no memory copy between CPU – GPU !!! 
• ~70% of memory available for app (GPU reserved memory + OS)
– Precision is relevant
• Halving precision ~doubles operations/second (1 FP128 = 8 FP16)
• vertex shader (medp/highp) | fragment shader (~lowp for color)
– Overdraw depending on the renderer  front-to-back for IMR/TBR
• Depth only pass can work on IMR/TBR depending on geometry count
– Texture compression!!!  bandwidth, power, performance
• Take ~5Gb/s as typical bandwidth on embedded devices (vs. 100Gb/s on desktop)
– Texture mipmapping / compression reduces bandwidth
– glReadPixels(), glCopyTexImage(), glTexSubImage() on FBO…  Block! Sync!
– glDiscardFramebufferEXT()  indicate render attachment is done with
76

Texture Compression
GL ES Format(bpp) Devices Proprietary Notes
PVRTC >=1Ext RGB (2,4),
RGBA(2,4)
PowerVR Imagination Good quality, not extended
S3TC(DXT1/3 & 5) >=1/2Ext RGB(4),
RGBA(4,8)
Tegra
Intel HD
S3 D3D
ATC >=1Ext Adreno AMD Maps to DXT with minor
conversion
ETC1 Core in 1
Ext in 2
RGB(4) All GLES 1
devices
Free Most extended, only RGB
ETC2/EAC Ext in 2
Core in 3
R(4), RG(8),
RGB(4,8),
RGBA(8)
GLES3 devices
Most GLES2
devices
Free Most extended, good
compression (ETC2),
compat. ETC1
ASTC >=2Ext Many(0.89 to
8bpp)
Mali /
GLES3.1?
Free Includes 3D, various
formats and texture types.
Not spread
77

Architectures -- GPU
• ASTC (Adaptive Scalable Texture Compression)
– ARM general solution for texture compression (open /
complex HW)
– 2D / 3D formats (normal, LDR/HDR, luminance, alpha, …)
– 128bits per block map 4x4..12x12 pixels & 3^3…6^3
– 0.59 bpp on 3D textures with 6^3 pixels per block 
– ARM Mali GPU T6xx support & next generation GPUs ?
– Quality & compression ratio! – free
– Wait till GLES3 is expanded --
• ETC2
– Core in GLES3 and GL4.3  RGB + RGBA compressed
formats (~ S3TC)
78

• Profiling tools
– ARM SDK (ARM) – Windows/Linux
• DS-5 Streamline – ($$) Sw and GPU profiling and debugging
– PowerVR SDK (Imagination) – Linux/Windows/OSX
• PVRUniSCo shader analyzer  #cycles
– Tegra SDK (NVIDIA) – Linux/Windows/OSX
• Tegra System Profiler
• NVIDIA PerfHUD ES
– Adreno SDK (Qualcomm) –
• Adreno Profiler – Windows only 
79

3D APIs
80

3D APIs
Mantle Direct3D Metal
OpenGL Next
5.0 ?
81

• Direct 3D
– 3D API from M$ for Win OS (XBOX)
– ANGLE library provides GL support on top of D3D
• Mantle
– AMD 3D API with Low-level access  D3D12 | GL_NG
• Metal
– Apple 3D API with low-level access
• OpenGL Desktop/ES/WebGL
– GL for embedded systems, now in version 3.0
• GLES3.1 ~ GL4.4 (GL_NG/Vulkan is coming…)
3D APIs
82

3D APIs
• Direct 3D
– Games on Windows (mostly) / XBOX
– Define 3D functionality state-of-the-art
• OpenGL typically following
• 3D graphic cards highly collaborative
• Multithread programming
– Proprietary – closed source – M$
– Tested & stable – good support + tools
• Metal
– Apple 3D API with low-level access
– Much in the way of Mantle?
• buffer & image, command buffers, sync…
– Lean & mean  simple + ~flexible
Win &
Game research
Mac/iOS future ?
83

3D APIs
• Mantle
– AMD effort – low level – direct access – 3D API
– Direct control of memory (CPU/GPU) – multithreading done well
• User-required synchronization
– API calls per frame <3k  100K
– Resources: buffer & image 
– Simplified driver  maintenance (vendors)
• High level API/Framework/Engines will be developed 
– Pipeline state
• shaders + targets (depth/color…) + resources + geometry
– Command queues + synchronization
• Compute / Draw / DMA(mem. Copy)
– Bindless – shaders can refer to state resources
– OpenGL NEXT seems to move into ‘Mantle direction’
– Direct 3D 12 already pursuing low-level access
84

3D APIs
Pre-compiled pipeline: shaders + resources  execute
http://www.slideshare.net/DevCentralAMD/mantle-introducing-a-new-api-for-graphics-amd-at-gdc14
85

3D APIs Command queues generated for each processing unit:
graphics/compute/memory access
86

3D APIs
The pipeline defines the association of variables to resource descriptors
87

3D APIs
• OpenGL (Desktop/ES/WebGL)
– Open / research / cross-platform
– Lagging in front of D3D  Legacy support 
• No more FIXED PIPELINE (1992)!! -- scientific visualization…
– GLSL (2003)…GL 3.1(2009)  deprecation/no fixed pipeline
• Compatibility profile  legacy again…(till GL 4)
• Core profile
– GLSL  shader required
– VAO
» group of VBO
» we need a base VAO for using VBO!
– Simplifying  VBO + GLSL only!
90

3D APIs
– OpenGL ES 1.1
• Fixed pipeline – no glBegin/End – no GL_POLYGON -- VBO
– OpenGL ES 2 (OpenGL 1.5 + GLSL) ~ GL4.1
• No fixed pipeline (shaders mandatory), ETC1 texture compress..
– OpenGL ES 3 ~ GL4.3
• Occlusion queries + geometry instancing
• 32bit integer/float in GLSL
• Core 3D textures, depth textures, ETC2/EAC, many formats…
• Uniform Buffer Objects (packed shader parameters)
– OpenGL ES 3.1 ~ GL4.4
• Compute shaders (atomics, load/store)
• Separate shader objects (reuse)
• Indirect draw (shader culling…)
• NO geometry/tessellation
91

3D APIs
• GPGPU
– OpenCL
• On Android it is not much loved
– Use GPU vendor SDK provided libs 
• On iOS is only accepted for system apps
– Use old-school GPGPU (fragment shader -> FrameBuffer)
– RenderScript
• Google solution for processing using GPU…
• Too niche! ~ Android
– Compute shaders
• GLES 3.1!!! General solution!!
– DirectCompute on D3D
93

Cross-development
http://www.appian.com/blog/enterprise-mobility-2/are-mobile-platform-choices-limiting-enterprise-process-innovation
94

Cross-development
• C++
– QtCreator – Qt 5.3 – free non-commercial / GPL
– Marmalade – free license
– Cocos-2dx -- free
• C#
– Xamarin – free basic license
– Mono for Android, Monotouch for iOS
• HTML5/JS
– Appcelerator / Titanium – free dev. license
– PhoneGap / Cordova ~ browser view  -- free
• COCOA/Objective-C
– Marmalade – Juice -- free license
• Ruby
– RhoMobile -- free license
95

Cross-development
• C++
– QtCreator – Qt 5.3
– Marmalade
– Cocos-2dx
• C#
– Xamarin
– Mono for Android, Monotouch for iOS
• HTML5/JS
– Appcelerator
– PhoneGap / Cordova ~ browser view 
– Intel XDK (on top of Cordova)
– Titanium
• COCOA/Objective-C
– Marmalade – Juice
• Ruby
– Rho
Many options – no size fits all
Platform API access
through framework
96

Cross-development
• C++ use case: QtCreator
– Qt (~supports android, iOS, windows phone, linux, windows, mac)
– Provides API abstraction for UI, in-app purchases, ~touch input
– HOWTO (i.e. android):
• Android SDK
• Android NDK (native C++ support, toolchain, libraries, GL, CL…)
• Point environment variables ANDROID_SDK, ANDROID_NDK to folders
• Qt 5.4 (+QtCreator 3.3) – community edition
• Create new android project
• Play!
– Notes:
• Go for Qt 5.4 (touch events were tricky in previous versions)
• Use QOpenGLWidget instead of QGLWidget
• Enable touch events on each widget:
– QWidget::setAttribute(Qt::WA_AcceptTouchEvents);
97

Cross-development
• Codebase
Codebase
(C/C++, …)
meta-project
Cmake
Qmake
Autoconf
Makefile
…
ARMv7
Library
ARM64
Library
MIPS
Library
x86
Library
iOS
Android
~Manually
modified scripts
Internet  toolchain (clang, gcc…) iOS, Android
CMAKE  OpenCV
QMAKE  Qt 5.4
Autoconf  manually modify 
99
Libraries:
cURL -- http
Xml -- doc
DevIL – image
Assimp – 3d loading

Cross-development
• Codebase Internet  toolchain (clang, gcc…) iOS, Android
CMAKE  OpenCV
QMAKE  Qt 5.4
Autoconf  manually modify 
100
*Setup envionment
CC= clang –arch armv7 –sysroot $SYSROOT …
CPP=clang++ …
LD=ld …
AR=ar …
*pointing to NDK_DIR/toolchain/$ARCH/bin/ where $ARCH={armv7, x86,…}
search for gcc/g++/clang inside NDK directories
*once setup the environment ~[”typically”] most tools work (DEFINES,
architecture types, platform supported functions, …)
Needed for!

Cross-development
• 3D framework / engine
– HW/platform abstraction  abstraction^2
• L1) Use 3D API  portability issues! ( HW , OS  )
– Try using GL ES 2/3/3.1 < GL 2.1/3.3/4.4 (Desktop GLES!!)
• L2) Abstract 3D API (HW , OS  )
– minimum common function set {D3D, GLES, Metal, Vulkan…}
– (~) buffer, image, shader, pipeline (config pkg)
Metal
GL ES
GL desktop
Vulkan
D3D
wrapper
Shader program
Pipeline
Buffer
Image buffer
Take a look at Metal!
WinPhone
Android
iOS
Unix/Linux
Windows
MacOS
101
Application

Cross-development
• WebGL
– Based on OpenGL ES 2 (WebGL 2  ES 3)
• Exceptions WGL2(from ES3): MappedBuffers, drawRangeElements,
ProgramBinaries
• Performance JS 
– TypedArrays [Khronos13]
• http://www.khronos.org/registry/typedarray/specs/latest/
– asm.js (Mozilla)
• JS used in optimized way (i.e. var v1= v2 |0, ensuring type is int)
• TypedArray large arrays  memory allocation (pre-reserved  )
• Porting C++ code
– Emscripten  C++  LLVM  JS (TypedArrays + asm.js)
102

Mobile Graphics – Development
• Conclusions
– 1) Native + platform UI …
• C++ [any language]  LLVM compiler  target platform
• Platform Framework front-end  1 for each platform
• Performance + flexibility
• Call native code from platform code (JNI, Object C, …)
– 2) Native through framework …
• Qt | Marmalade …
• C++ code uses framework API
– Framework API abstracts platform API [N platforms]
– BUT less flexible integration ?
– 3) Go web  HTML5/JS …
• Rewrite or Use Emscripten  JS code + WebGL
• ~Free portability (chrome / firefox / IE … ?)
• BUT performance is 0.5X at most with asm.js
103

Mobile Graphics (part1)

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (20)

Semelhante a Mobile Graphics (part1)

Semelhante a Mobile Graphics (part1) (20)

Mais de CRS4 Research Center in Sardinia

Mais de CRS4 Research Center in Sardinia (20)

Último

Último (20)

Mobile Graphics (part1)

Notas do Editor