The presentation for how to do image prcessing applications on android devices.

1. OpenCV
Image
Processing
on
Android
Devices
Wilson Wingston Sharon
wingston.sharon@gmail.com

2. Le me
• Day job python developer.
• Hobby technologies
o Machine Learning
o Hardware Hacking
o Image processing
o Graphics & Video design

3. Introduction
• Open Computer Vision (OpenCV) developed &
donated to FOSS by Intel.
• Open Source Image Processing library in C/C++
• Optimized for real time image / video processing
applications
• OS/hardware independent with ports to Java,
Python with wrappers to numerous other languages
and platforms.

4. Why OpenCV?
• No need to reinvent the wheel to deal with image
processing solutions.
• Has both high level and low level components.
• Lets you treat images as a 2D Matrix.
• Image processing comes down to simple mathematical
operations on this Matrix.
• Also provides a framework for machine learning, 3D
scene information, depth profile, object recognition,
motion tracking etc..

5. Suiting up
• Sun JDK 6
o Very important you DO NOT use Sun JDK 7. unless you want to deal with intermittent
weird run time issues that make no sense. In that case go right ahead.
• Android SDK + NDK
o Keep it updated once in a while.
o Remember:You need “ia32-lib” if you’re running a 64 bit ubuntu/debian
o NDK is required only if native code is erquired to be compiled.
• Android SDK tools rev14+
o fun fact: using older versions can cause heartburn.
• SDK Platform API 8 is minimum required
o But usually a device running 11 would have the processing power you need to play with
image processing application without lag.
• Eclipse with ADT + CDT
o Duh.

6. OpenCV4Android SDK
• Obtain CV-2.4.2-android-sdk.zip from sourceforge.
• SDK
o Contains the openCV libraries and Java classes for Android.
o /java, /native and /etc
• APK
o Opencv manager apk that must be installed to make the openCV libaries on the
android device available.
o Android market installation can also be done.
• Samples
o Contains a decent amount of openCV samples for android.
o These samples are written as documentation.
o See the openCV wiki for setting up Eclipse with OpenCV.

7. Overview of some
openCV classes
in /sdk/java

8. org.opencv.android
• Contains the Async loader that you’ll need
to connect your application to the openCV
libraries installed on your device.
• Utility function to convert between
openCV data types and android ones.
• Some other loader classes for your
viewing pleasure.

9. org.opencv.core
•Contains the core building blocks of openCV
•Matrix, point, rect and Bitmap data types
defined here.
org.opencv.highgui
•Allows you to draw some GUI components
on the screen.
•Also has the openCV videcapture class as an
alternative to the android camera interface.

10. org.opencv.imgproc
• The main image processing functions exist here,
like blurring edge detection, colour conversion.
• This file also contains documentation and usage
of each function.
org.openCV.ml
• Contains classes to assist in developing almost
any machine learning algorithm in images.
• Trees, statistical models, Bayes, Markov models, K
nearest all provided here with in-code
commented documentation

11. Programming
methodology

12. The application structure
• Best way (as suggested in samples) is as follows.
• MainActivity
o Extends the android activity class & is the Launcher Activity.
o Initializes the openCV libraries.
o Handles the android activity lifecycle functions.
• opencvView
o Extends opencvViewBase
o Running openCV code goes here.
• opencvViewBase
o Extends surfaceView as a holder for the video stream
o Abstract class to simplify usage of OpenCV
o Sets up and handles camera / preview & drawing

13. MainActivity.java
• onCreate()
o This is called when the activity is first created.
o OpenCVLoader.initAsync(…) attempts to start the openCV library and on
completion calls mOpenCVCallBack.
• mOpenCVCallBack()
o Inside there is a function called onManagerConnected() which checks to
see if no errors were there
o If openCV loaded sucessfully, then an instance of Sample1View can be
created called say mView
• Also this class has the onPause(), onResume()
functions which start / stop camera streaming as
per the android activity lifecycle.

14. opencvView.java
• This class contains code relevant for your application, this
directly extends the absract opencvViewBase class.
• processFrame(byte[] data)
o Input comes in as byte[] data as raw camera feed
o Using the mat.put() this picture can be stored in a matrix
o Any openCV functions from imgproc etc.. can be performed on
this mat object
o Display must be in bitmap format, so the mat object is converted
to bmp
o This bmp object is returned to run() which displays it on the screen.

15. • Camera mCamera;
o Camera object to open/close/ retrieve video information
• SurfaceHolder mHolder;
o Holder to the surfaceview to enable a live streaming of video
• Mat mRgba;
o Matrix Object that openCV uses for image manipulation
• Bitmap mBitmap;
o Bitmap object that can be displayed on the surface view
• mRgba.put(0, 0, data);
o Returns the raw data feed in an openCV matrix format.
• Imgproc.cvtColor(mGraySubmat, mRgba,
Imgproc.COLOR_GRAY2RGBA, 4);
o How to convert an RGB matrix to a greyscale
• Utils.matToBitmap(mRgba, bmp);
o Converting a matrix to a bitmap

16. Real Time Applications

17. Real time Face detection
• fdActivity example is an openCV sample that shows how to
implement face detection algorithm on a real time video
stream.
• It is also an example on how to efficiently implement an
algorithm in C and call it via the JNI interface.
• Also implements an FPS meter class to judge the usability of
the algorithm used.
• A Haar Classifier from the ml library with the openCV trained
xml file is used on the image stream to return if the image
contains a face or not.
• It is the standard openCV way of face detetcion.

18. Possible Applications on
android devices
• Obtain features from any image and send it over
network for matching against an online database
for object recognition.
• Use the front camera to monitor head movement
and provide a hands free interface for some
games.
• Auto-tagging of friends in photos taken and
associating them with facebook/android contacts.

19. Thank You..
Any Questions?
wingston.sharon@gmail.com