Android beyond the smartphone

2. About Chris Simmonds • Consultant and trainer • Author of Mastering Embedded Linux Programming • Working with embedded Linux since 1999 • Android since 2009 • Speaker at many conferences and workshops "Looking after the Inner Penguin" blog at http://2net.co.uk/ https://uk.linkedin.com/in/chrisdsimmonds/ https://google.com/+chrissimmonds Android beyond the smartphone 2 Copyright © 2011-2016, 2net Ltd

3. Agenda 1. Embedded Android: 30 minutes Q & A 2. Android BSP: 30 minutes Q & A 3. Interfacing with the real world: 30 minutes Q & A Wrap-up and conclusion Android beyond the smartphone 3 Copyright © 2011-2016, 2net Ltd

5. Mainstream Android • Android from Google comes in several ﬂavours: • Android, for smartphones and tablets (approx. 1.5 billion of them) • Android Wear, for smart watches • Android TV • Android Auto, for in-vehicle "infotainment" • Plus Brillo, a cut-down Android (no display) for IoT Android beyond the smartphone 5 Copyright © 2011-2016, 2net Ltd

6. Non-Google Android • Android is an open source operating system • Open source means anyone can join in • Some well-known examples: • Amazon Kindle Fire • BlackBerry Priv • Cyanogen Mod and many other "modder ROMs" Android beyond the smartphone 6 Copyright © 2011-2016, 2net Ltd

7. Embedded Android • You can create your own custom Android for your own needs • Capitalize on familiar UI • Well-known programming framework • Good development tools Android beyond the smartphone 7 Copyright © 2011-2016, 2net Ltd

8. Why use Android? • User interface • Attractive • Familiar navigation gestures • Programming framework • Everyone knows Java • Developer tools • Android Studio • Many tools for debugging and proﬁling • Secure, as a result of SELinux amongst other things • Robust: each app runs in its own sandbox • Updatable: built-in OTA updater Android beyond the smartphone 8 Copyright © 2011-2016, 2net Ltd

11. How open is Android? • Need to distinguish between the open part: • AOSP • And the proprietary part: • Google Mobile Services • GMS consists of extension libraries that implement the Google APIs, such as maps, in-app payment, Gmail and so on • Let’s take a closer look at the architecture ... Android beyond the smartphone 11 Copyright © 2011-2016, 2net Ltd

12. Architecture of AOSP Android Java libraries (android.*) Android Java API Applications System services "Glue" libraries Native servers HAL libraries Other libraries Linux Binder JNI Android framework Native layer Kernel App layer Android beyond the smartphone 12 Copyright © 2011-2016, 2net Ltd

13. The four layers • Applications: stock AOSP apps and 3rd party apps • Framework, consisting of • The android.* libraries, which dispatch messages via binder to ... • ... System services, which use JNI to call down to ... • ... "Glue" libraries written in C++ • Native • The init program, various native servers (daemons) and libraries • Kernel • A Linux kernel customised for the platform Android beyond the smartphone 13 Copyright © 2011-2016, 2net Ltd

15. Google Mobile Services • Plain Android apps use only the APIs published within the AOSP • AOSP aps include Launcher2, Browser, Email, Gallery, Keyboard • GMS is a set of services that provide Google proprietary APIs • Most user-facing features require GMS, including popular Google apps such as Play Store, Chrome browser, Google Now, etc. • Requires a license from Google • (You can grab it unofﬁcially (and illegally) via the gapps zip ﬁle - google for it) Android beyond the smartphone 15 Copyright © 2011-2016, 2net Ltd

16. What do you need to run Android? • Hardware from one of the supported architectures • ARM, x86 or MIPS, in 32 or 64 bit varieties • Capable of running Linux • At least 512 MiB RAM • At least 512 MiB ﬂash storage • Touchscreen or external display (optional, but desirable) • GPU with OpenGL ES libraries (optional, but desirable) Android beyond the smartphone 16 Copyright © 2011-2016, 2net Ltd

17. Dev boards 1/3 Qualcomm DragonBoard 401c • Quad-core ARM Cortex A53 • 1 GiB RAM • Wi-Fi 802.11 b/g/n • Adreno 306 GPU • Expansion headers: 1 x UART, 2 x i2c, 12 x GPIO https://developer.qualcomm.com/hardware/dragonboard-410c Android beyond the smartphone 17 Copyright © 2011-2016, 2net Ltd

18. Dev boards 2/3 Wandboard • i.MX6 SoC: single, dual or quad core ARM Cortex A-9 • 512 MiB, 1 GiB or 2 GiB RAM • Ethernet and WiFi • Vivante 880 GPU • Expansion headers: 2 x UART, 3 x i2c, 8 x GPIO http://www.wandboard.org Android beyond the smartphone 18 Copyright © 2011-2016, 2net Ltd

19. Dev boards 3/3 BeagleBone Black • TI AM3359 SoC: single core ARM Cortex A-8 • 512 MiB RAM • Ethernet • Expansion headers: 4 x UART, 2 x i2c, 65 x GPIO https://github.com/csimmonds/android4beagle Android beyond the smartphone 19 Copyright © 2011-2016, 2net Ltd

22. DIY Android • So, you want to port Android to your platform? Here are the steps: • Make sure it boots Linux (no Android) • Create a simple minimal Android device deﬁnition • Boot Android to command shell (no display) • Get ADB USB gadget driver to work • Build Android UI, without GPU support, and get display working • Implement vendor GPU drivers and OpenGL ES libraries • Implement networking, audio, camera, etc as required Android beyond the smartphone 22 Copyright © 2011-2016, 2net Ltd

23. The Linux kernel • The AOSP repositories do not include a kernel • It is up to you to provide one • Android needs some special kernel features, including: • Android-speciﬁc device drivers: binder, logger, anonymous shared memory, etc • Aggressive power management: auto sleep and wake locks Android beyond the smartphone 23 Copyright © 2011-2016, 2net Ltd

24. Androidisms 1/2 • In mainline Linux since 3.3 • Binder: transaction-based inter-process communication • Anonymous shared memory: allows processes to share memory segments • Low memory killer: kills background applications when memory is low • In mainline Linux since 3.5 • Autosleep/wakelock: suspends CPU unless kernel or app holds a lock Android beyond the smartphone 24 Copyright © 2011-2016, 2net Ltd

25. Androidisms 2/2 • Still out of tree • Paranoid network security: restricts access to network sockets to apps with android.permission.INTERNET • The recommended kernel conﬁguration options are documented at source.android.com/devices/tech/kernel.html Android beyond the smartphone 25 Copyright © 2011-2016, 2net Ltd

26. Google reference kernels • Google provide reference implementations for Nexus devices and the emulator at https://android.googlesource.com • There are details on how to build them at http://source.android.com/source/ building-kernels.html • For example, to get the kernel for the emulator ("goldﬁsh"): $ git clone https://android.googlesource.com/kernel/goldfish $ cd goldfish $ git checkout origin/android-3.18 Android beyond the smartphone 26 Copyright © 2011-2016, 2net Ltd

27. Google kernels: some stats Kernel Latest version Patches(*) Products common 4.1 516 generic exynos 3.4.67 - Nexus 10 goldﬁsh 3.18.0 713 Emulator msm 3.10.73 32968 Nexus 4, 5, 6, 5X, 6P omap 3.10.0 1218 Minnow (Moto 360) samsung 3.0.8 - Nexus S tegra 3.10.40 - Xoom, Nexus 7, 9 x86 3.10.20 - Nexus Player (*) Number of patches from mainline, which you can ﬁnd like this: $ git remote add stable git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git $ git fetch stable $ git log v<kernel-version>..HEAD $ git format-patch v<kernel-version>..HEAD Android beyond the smartphone 27 Copyright © 2011-2016, 2net Ltd

28. Vendor kernels • Most vendors base their kernels on the Google reference • ... and then add more patches of their own • Kernel code is licensed under GPLv2, so source is (should be) generally available • Conclusions: • The kernel in most shipping products is quite old • About 500 patches account for Androidisms not in mainline • The remainder are speciﬁc to a particular SoC/board/product Android beyond the smartphone 28 Copyright © 2011-2016, 2net Ltd

29. Getting AOSP • First, you need to get repo: $ curl http://commondatastorage.googleapis.com/git-repo- downloads/repo > ∼/bin/repo $ chmod a+x ∼/bin/repo • Then use it to clone AOSP: $ mkdir ∼/myandroid $ cd myandroid $ repo init -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r55 • The -b option selects the branch • See http://source.android.com/source/ build-numbers.html for comprehensive list Android beyond the smartphone 29 Copyright © 2011-2016, 2net Ltd

30. Building Android • The Android build system uses GNU make • Controlled by a single top level Makefile plus many fragments: • AndroidProducts.mk: defines products, in sub tree device/ • BoardConfig.mk: defines boards, in sub tree device/ • Android.mk: defines Android packages, throughout AOSP Android beyond the smartphone 30 Copyright © 2011-2016, 2net Ltd

31. Creating a new device • Device configurations are in the device/ directory • There is a two-level hierarchy • organization/product • The examples that follow use sirius and marvin 1 • Configuration is defined in AndroidProducts.mk, BoardConfig.mk and supporting files • The next few slides describe these essential files... 1In the The Hitchhikers Guide to the Galaxy, Marvin was a paranoid android manufactured by the Sirius Cybernetics Corp Android beyond the smartphone 31 Copyright © 2011-2016, 2net Ltd

32. AndroidProducts.mk • This is the top-level makeﬁle for the product • Simply references a makeﬁle for each variant of the product • Often there is only one • For example: PRODUCT_MAKEFILES := $(LOCAL_DIR)/aosp_marvin.mk Android beyond the smartphone 32 Copyright © 2011-2016, 2net Ltd

33. Product makeﬁle example aosp_marvin.mk $(call inherit-product, $(SRC_TARGET_DIR)/product/aosp_base.mk) $(call inherit-product, device/sirius/marvin/device.mk) PRODUCT_NAME := aosp_marvin PRODUCT_DEVICE := marvin PRODUCT_MODEL := Paranoid Android Android beyond the smartphone 33 Copyright © 2011-2016, 2net Ltd

34. Pre-deﬁned products • In build/target/product: • embedded.mk: minimal build of native layer plus SurfaceFlinger but no Java • aosp_base.mk: embedded.mk, plus Dalvik, Android framework, apps, etc • aosp_base_telephony.mk: aosp_base.mk plus phone dialler and ril • sdk.mk: the SDK • In device/google/atv/products: • atv_base.mk: Android TV Android beyond the smartphone 34 Copyright © 2011-2016, 2net Ltd

35. Android packages • Android packages (also called Android modules) are units of code that can be compiled separately • Each package is deﬁned by an Android.mk • Name of package given by • LOCAL_PACKAGE_NAME := for Android applications • or • LOCAL_MODULE := for everything else • Package names must be unique: build will fail if not • In Marshmallow there are > 2800 Android.mk ﬁles Android beyond the smartphone 35 Copyright © 2011-2016, 2net Ltd

36. device.mk • device.mk contains additions and overrides to standard conﬁguration • For example, to add packages: PRODUCT_PACKAGES += librs_jni com.android.future.usb.accessory androidvncserver Android beyond the smartphone 36 Copyright © 2011-2016, 2net Ltd

37. BoardConfig.mk • Defines characteristics of the board • CPU architecture and type • Sizes of file system partitions • Too many settings to document here: look at other BoardConfig.mk files for ideas Android beyond the smartphone 37 Copyright © 2011-2016, 2net Ltd

38. vendorsetup.sh • lunch searches for vendorsetup.sh in • vendor/*/vendorsetup.sh • vendor/*/*/vendorsetup.sh • device/*/*/vendorsetup.sh • Used to add a product to the menu • Format is add_lunch_combo PRODUCT_NAME-[user | userdebug | eng] • Example add_lunch_combo aosp_marvin-eng Android beyond the smartphone 38 Copyright © 2011-2016, 2net Ltd

39. Running the build • Select the device $ lunch Lunch menu... pick a combo: 1. aosp_arm-eng [...] 26. aosp_marvin-eng • Start the build $ make -j8 ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=6.0.1 TARGET_PRODUCT=aosp_marvin [...] • And wait several hours Android beyond the smartphone 39 Copyright © 2011-2016, 2net Ltd

40. The problem with graphics • Android requires OpenGL ES v1.1 to boot • Many applications require OpenGL ES v2 or later • Android provides a fall-back implementation for OpenG LES v1 • A software rendering engine called PixelFlinger • OpenGL ES v2 and later requires hardware, together with kernel drivers and libraries • The problem: drivers and libraries are not open source Android beyond the smartphone 40 Copyright © 2011-2016, 2net Ltd

41. Living without GPU support • Enable PixelFlinger by adding to kernel command line qemu=1 qemu.gles=0 • Great for board bring-up and early stages of porting • Could be used in a production device with simple UI needs • But, many things need OpenGL ES 2 (e.g. Chrome browser) Android beyond the smartphone 41 Copyright © 2011-2016, 2net Ltd

44. Hardware interfacing • Android has support for the hardware you would expect in a smartphone • UI: touchscreen, HDMI monitor, keyboard, mouse, gamepad • Network: mobile data, WiFi, Ethernet • Sensors: accelerometer, magnetometer, ambient light, etc. • But, embedded applications have more diverse needs ... • USB, serial, i2c, digital and analog I/O Android beyond the smartphone 44 Copyright © 2011-2016, 2net Ltd

45. I/O, securely • In Linux, all I/O is done via ﬁle descriptors ("everything is a ﬁle") • For security and stability, we want to restrict access to I/O • Android has a complex security model that enforces this Android beyond the smartphone 45 Copyright © 2011-2016, 2net Ltd

46. The principle of least privilege • Each component (app, system service, native daemon) is only allowed to access resources necessary to do the job • Enforced by ﬁle modes and SELinux policy • Package Manager assignes a unique User ID (UID) to each app when it is installed • Apps cannot access any I/O ﬁle nodes Android beyond the smartphone 46 Copyright © 2011-2016, 2net Ltd

47. System services • System resources (display, audio, etc.) are proxied through Android system services running with greater privilege • System services fail the request if app doesn’t have the permission Client app Server process /dev/binder Android beyond the smartphone 47 Copyright © 2011-2016, 2net Ltd

48. Android permissions • Apps request permissions in AndroidManifset.xml • For example, to access the serial port: <uses-permission android:name="android.permission.SERIAL_PORT"/> • Permission checked by the service • In this case, Serial Manager: SerialService.java enforceCallingOrSelfPermission(android.Manifest.permission.SERIAL_PORT, null); // Throws SecurityException if the client app does not have permission Android beyond the smartphone 48 Copyright © 2011-2016, 2net Ltd

49. Accessing hardware in Android • There are services for • USB host • USB accessory (described later) • Serial • Other types of interface are not provided Android beyond the smartphone 49 Copyright © 2011-2016, 2net Ltd

50. USB host mode • Android supports USB connections where Android is the USB host • Applies to most USB devices • Class android.hardware.usb Android beyond the smartphone 50 Copyright © 2011-2016, 2net Ltd

51. USB accessory mode • Android also has USB "accessory mode" • Android acts as a USB device, the other end acts as the host and provides power • Android Open Accessory (AOA) protocol • AOAv1 from Honeycomb v3.1 (API level 12) • AOAv2 from Jelly Bean v4.1 (API level 16): adds audio from Android to accessessory and HID • Example use: link Android devices to car stereos Android beyond the smartphone 51 Copyright © 2011-2016, 2net Ltd

52. Serial • Android has (minimal) support for serial ports • Class android.hardware.SerialPort • Requires platform support • Enable serial ports in framework • Permit access to device nodes in SELinux Android beyond the smartphone 52 Copyright © 2011-2016, 2net Ltd

53. Serial import android.hardware.SerialManager; import android.hardware.SerialPort; [...] private SerialManager mSerialManager; private SerialPort mSerialPort; mSerialManager = (SerialManager)getSystemService(Context.SERIAL_SERVICE); mSerialPort = mSerialManager.openSerialPort("ttyS1", 115200); ret = mSerialPort.read(mInputBuffer); mSerialPort.write(mOutputBuffer, bytes.length); Android beyond the smartphone 53 Copyright © 2011-2016, 2net Ltd

54. i2c • There are no Android APIs for i2c • However, as a work round: • Modify boot scripts to set mode of /dev/i2c* to 0666 (*) • Either run SELinux in permissive mode or write SELinux policy to allow apps to read/write those nodes • Write a native library that reads/writes the registers on your i2c device (*) There are better ways, but this works. Ideally you would implement an i2c system service and use Android permissions to control access Android beyond the smartphone 54 Copyright © 2011-2016, 2net Ltd

55. Non-Android code • Use case: porting a middleware layer • Problems • Android C library (Bionic) lacks features -> code won’t compile • Android build system can’t run external Makefiles -> need to re-write as Android.mk files • Android file system layout is not standard -> need to change file paths Android beyond the smartphone 55 Copyright © 2011-2016, 2net Ltd

56. Separately-compiled binaries • Use an external GNU toolchain, and either: • static link - no library dependencies, but large executables • chroot - create your own root for your program (see later) • Copy the compiled executables and other ﬁles into the Android image from device.mk using PRODUCT_COPY_FILES Android beyond the smartphone 56 Copyright © 2011-2016, 2net Ltd

57. Communication with Android apps • Use standard IPC mechanisms that are supported by Android: • Local (UNIX) Socket (preferred) • Named pipe (also known as FIFO) • A regular ﬁle • Can use any libraries, including those not provided by Android • Avoid license restrictions so you can use GPL code • Application and your imported native code run in separate processes, so more robust Android beyond the smartphone 57 Copyright © 2011-2016, 2net Ltd

58. chroot • Standard utility that runs a command with different root directory • chroot <new root> <command> • The command (and child processes) can only access ﬁles "inside" the new root • Often called a chroot jail • See man chroot(8) for details • Android does not include chroot! • Typically, you would use a statically-compiled Busybox which has its own chroot, along with other tools. Android beyond the smartphone 58 Copyright © 2011-2016, 2net Ltd

60. What next? • Get a dev board • Start to play • Learn how to build Android from scratch • Begin tinkering with hardware • Keep on learning Android beyond the smartphone 60 Copyright © 2011-2016, 2net Ltd

61. Classses in Android and Linux • KComputing, in conjunction with 2net, offer a number of 4-day instructor-led classes, including • Android Porting • Android internals • Embedded Linux • System programming for embedded Linux • Open access or on-site • Details at http://2net.co.uk/training.html/ Android beyond the smartphone 61 Copyright © 2011-2016, 2net Ltd