When I learn more about Android's graphics system, and do more work about how to use CPU/GPU in more parallelized way to improve the graphics performance in Android, I start to think that there are actually some big design mistakes in Android graphics system, especially the rendering architecture in the client side.Some mistakes have been solved after 3.x, especially above 4.1, but others can never be solved due to the compatible reason. As developers, we need to know how the Android graphics system work, how to utilize the new features Android 3.x and 4.x provided, and how to do the optimization and overcome the shortage of Android.

1. Why your Android Apps Suck
易旭昕 (Roger)
roger2yi@gmail.com
en.gravatar.com/roger2yi

2. 6/20/2013 Roger, UC
About Me
http://weibo.com/roger2yi

3. 6/20/2013 Roger, UC
Why I wrote this article?
– When I learn more about Android's graphics
system, and do more work about how to use
CPU/GPU in more parallelized way to
improve the graphics performance in Android
– I start to think that there are actually some big
design mistakes in Android graphics system,
especially the rendering architecture in the
client side

4. 6/20/2013 Roger, UC
• Some mistakes have been solved after 3.x,
especially above 4.1, but others can never be
solved due to the compatible reason
• As developers, we need to know how the
Android graphics system work, how to utilize the
new features Android 3.x and 4.x provided, and
how to do the optimization and overcome the
shortage of Android

5. 6/20/2013 Roger, UC
Direct Rendering
• In the very beginning, Android's client/server
architecture of its graphics system choose the
direct rendering mode
• Android team develop a lightweight Window
Compositor - SurfaceFlinger by themself, it
response to create native windows, allocate
graphics buffers and composite the window
surface to display (via framebuffer)

6. 6/20/2013 Roger, UC
• In client side, apps (through Android runtime)
can use CPU to access the graphics buffer of
native window directly (through Skia), or use
GPU to manipulate it indirectly (through GLES)
• This direct rendeing mode is more appropriate
than the traditional X in desktop Linux

7. 6/20/2013 Roger, UC

8. 6/20/2013 Roger, UC
Window Composite in Server
Side
• SurfaceFlinger use GLES1 to do the window
composition (via texture operation) in the very
beginning, this cause two issues
– Chip may have more efficient way (Display
Controller) to do the window composition than
use GLES
– When the client side also use GLES to render
the window surface, it may competes with the
server about the GPU resources
• Just mention, when GPU do not support GLES1, Android has a built-in
SW version GLES1 stack, and it can use copybit HAL module provided
by chip vendor to accelerated the texture operation

9. 6/20/2013 Roger, UC
iOS do not need
Window Composite!!!
• It only has one visible window at one time, so
the app can render to framebuffer directly
• Advantages
– Save memory usage
– Reduce GPU usage
– Save memory bandwidth
• Disadvantages
– Only one visible app at screen
– Do not support 3rd SIP or lanucher for
security

10. 6/20/2013 Roger, UC
• So, after 3.0, Android introduce hwcomposer
HAL module to solve these issues, also abandon
the old copybit module
• Chip vendor can through the implementation of
hwcomposer module to declare they can do all
of or part of windows' composition by
themselves, usually with dedicated hardware
and always more efficient than use GLES

11. 6/20/2013 Roger, UC
• When all windows composited by hwcomposer,
and keep SurfaceFlinger idle
– Reduce GPU usage
– Save memory bandwidth
– Improve performance and save energy
• In theory, Android will has same performance
with iOS when it has only one visible window

12. 6/20/2013 Roger, UC
• Also, after 4.1, hwcomposer can provide the
vsync signal of the display, so that Android can
sync three things together
– the windows composition in server side
– the rendering of window surface in client side
– the refresh of display

13. 6/20/2013 Roger, UC

14. 6/20/2013 Roger, UC
Rendering Architecture in Client
Side
• The most mistake Android team make is the
rendering architecture of its GUI framework
– It do not has a layer rendering architecture (or
called scene graph in some GUI fw)
– It do not has a dedicated rendering thread to
render the window surface
– It's rendering only use CPU until 3.0
• The first one is partially support after 3.0, the
third is support after 3.0, but the second problem
can never be solved...

15. 6/20/2013 Roger, UC
Compare to iOS
• Every View has a Layer as its backing store, app
can create more Layers for better performance
• View's content will be drew into its Layer, as
long as the content do not changed, the View do
not need to draw itself again
• iOS do a lot of things to avoid change the
content of a View, many many properties of a
View can be changed without affect to its
content, such as background, border, opacity,
position, transformation, even the geometry!!!

16. 6/20/2013 Roger, UC
• The composition of Layers done by another
dedicated rendering thread, it always use GPU
to draw each Layer to the window surface
• The main thread only reponse to handle touch
event, relayout the Views, draw View's content
into its Layer when necessary, etc...
• So the main thread only use CPU and the
rendering thread use GPU mostly, and I think
there will be just a few synchronization between
these two threads, and they can run concurrently
in most of time

17. 6/20/2013 Roger, UC
• But in Android, the main thread need to do
everything, such as handle touch events,
relayout views, dequeue/enqueue graphics
buffer, draw views' content to window surface,
etc... And it only use CPU before 3.0!!!
• Even the position of a View just change one
pixel, Android need to redraw its content along
with the contents of other Views overlapped, and
the content drawing is very expensive for CPU!!!

18. 6/20/2013 Roger, UC
The Improvements
• A lot improvements have been made after 3.0 to
overcome the shortage of previous version
• Android 3.0 introduce a new hwui module, and it
can use GPU to accelerated the drawing of
View's content, it create a hw accelerated
Canvas to replace the old sw Canvas, the new
Canvas use OpenGL ES as its backend instead
of use SkCanvas from Skia

19. 6/20/2013 Roger, UC
• Android 3.0 also introduce the DisplayList
mechanism, DisplayList can be considered as a
2D drawing commands buffer, every View has
its own DisplayList , and when its onDraw
method called, all drawing commands issue
through the input Canvas actually store into its
own DisplayList

20. 6/20/2013 Roger, UC
• When every DisplayList are ready, Android will
draw all the DisplayLists, it actually turn the 2D
drawing commands into GLES commands to
use GPU to render the window surface
• So the rendering of View Hierarchy actually
separated into two steps, generate View's
DisplayList, and then draw the DisplayLists, and
the second one use GPU mostly

21. 6/20/2013 Roger, UC
• When app invalidate a View, Android need to
regenerate its DisplayList, but the overlapped
View's DisplayList can keep untouched
• Also, Android 4.1 introduce DisplayList
properties, DisplayList now can has many
properties such as opacity, transformation, etc...,
and the changed of some properties of View will
just cause changed of corresponding properties
of its DisplayList and need not to regenerate it
• These two improvements can save some times
by avoid regenerate DisplayLists unnecessary

22. 6/20/2013 Roger, UC
• Although Android can never has a layer
rendering architecture, it actually introduce some
Layer support after 3.0, a View can has a Layer
as its backing store
• The so called HW Layer actually back by a FBO,
if the content of View is too complicated and
unlikely to change in the future, use Layer may
help
• Also, when a View is animating (but content do
not change), cache itself and its parent with
Layers may also help

23. 6/20/2013 Roger, UC
• But use Layer with caution, because it increase
the memory usage, if your want to use Layers
during animation, your may need to release
them when the animation is finish, Android 4.2
provide new animation API to help you about
that
• Also, because Android use GLES to draw the
content of View, so most Views' drawing will be
fast enough, and the use of Layer may be
unnecessary

24. 6/20/2013 Roger, UC
• Android 4.0 introduce a new type of native window -
SurfaceTextureClient (back by a SurfaceTexture) and its
Java wrapper TextureView, app can create and own this
kind of native window and response to its composition
• If the content of View is too complicated and continue to
change, TextureView will be very helpful, app can use
another thread to generate the content of TextureView
and notify the main thread to update, and main thread
can use the TextureView as a normal texture and draw it
directly on the window of current Activity

25. 6/20/2013 Roger, UC
• Android 4.1 make the touch event handling and
ui drawing sync with the vsync signal of display,
it also use triple buffers to avoid block the main
thread too often because it need to wait the
SurfaceFlinger to release the buffer, and
SurfaceFlinger will always sync with vsync signal
of display
• The OpenGL Renderer for hw accelerated
Canvas is continue be improved and become
faster, especially for complicated shape drawing

26. 6/20/2013 Roger, UC
But...
• But Android can never has a dedicated
rendering thread...
• Although the drawing is much faster than before,
and keep the changed of everything as little as
possible during animating, it still need to share
the 16ms interval with other jobs in main thread
to achieve 60 fps

27. 6/20/2013 Roger, UC
So...
• So, as developer, we need to utilize the
improvements in higher version Android as
much as possible
– Turn on the GPU acceleration switch above
Android 3.0
– Use the new Animation Framework for your
animation
– Use Layer and TextureView when necessary
– etc...

28. 6/20/2013 Roger, UC
• And avoid to block the main thread as much as possible, that means
– If your handle the touch events too long, do it in another thread
– If your need to load a file from sdcard or read data from
database, do it in another thread
– If your need to decode a bitmap, do it in another thread
– If your View's content is too complicated, use Layer, if it continue
to change, use TextureView and render it in another thread
– Even your can use another standalone window (such as
SurfaceView) as a overlay and render it in another thread
– etc...

29. 6/20/2013 Roger, UC
Golden Rules for Butter
Graphics
• Whatever you can do in another thread, then do
it in another thread
• Whatever you must do in main thread, then do it
fast
• Always profiling, it is your most dear friend

30. 6/20/2013 Roger, UC
All you need to do is keep the loop
of main thread under 16ms interval,
and every frame will be perfect!

31. 6/20/2013 Roger, UC
The Last Word
• When I finish this article, what make me think
most is, when you make a big design mistake in
the very beginning, and you can not change it
due to some reasons, no matter how hard you
try to patch it in future, it will never get perfect
again
• Android team make huge effects to provide
features like Strict Mode, AsyncTask,
Concurrent & Background GC, hw accelerated
Canvas, hw Layer, TextureView, vsync, triple
buffers, etc...

32. 6/20/2013 Roger, UC
• All they do just about two things, do everything
you can in another thread, and make the must
thing in main thread faster, and these actually
help a lot
• But no matter how hard you try to use these
features to improve your app, it is nearly
impossible to get every frame perfect, because it
is so hard to forecast everything the main thread
will do in every loop, it may suddenly jump into
something totally uncontrollable by you app and
make you break the 16ms curse

33. 6/20/2013 Roger, UC
• And the worse is, if you has a good design, you
can continue improve it (like utilize more
advance hardware), but the developer need not
to know anything about this and do not need to
change even one line of their code, if the app
run on higher version OS with faster device, it
just got the performance boost

34. 6/20/2013 Roger, UC
• If you has a bad design, although you do a lot of
things to patch it, but the developer need to
know why, they need to rewrite their code to use
these new features, they need to know how to
avoid the trap, they need to know how to make
compatible with older version system, and many
many other things...
• This is too hard for a developer just want to build
a useful and beauty app...

35. The End
Thank you for your
listening
Yours Sincerely, Roger