1. Computer Vision
OpenCV
Luigi De Russis
Politecnico di Torino
Dipartimento di Automatica e Informatica (DAUIN)
Torino - Italy
luigi.derussis@polito.it
This work is licensed under the Creative Commons (CC BY-NC-SA)
License. To view a copy of this license, visit
http://creativecommons.org/licenses/by-nc-sa/4.0/
2. What is OpenCV?
Computer Vision library
Open Source (BSD License)
http://opencv.org
Originally developed by Intel
With more than 2500 optimized algorithms
Supports a lot of different languages
C, C++, Python, and Java
but is written natively in C++
Cross platform
also available for Android and iOS
10/18/2016 Introduction to OpenCV 2
3. What is OpenCV?
10/18/2016 Introduction to OpenCV 3
SSE, NEON, IPP, OpenCL, CUDA, etc.
core, imgproc, objdetect, etc.
face, text, etc.
Bindings: Python,
Java, …
Sample, tutorials
OpenCV
OpenCV
contrib
OpenCV
HAL
4. What is used for?
Human-Computer Interaction (HCI)
Object Identification
Object Recognition
Face Recognition
Gesture Recognition
Motion Tracking
Image Processing
Mobile Robotics
… and so on
10/18/2016 Introduction to OpenCV 4
5. Why OpenCV (for this course)?
We looked for a software/library:
that covers most of Computer Vision techniques
widely used or de-facto standard
free/open, possibly
in a language known by most students
in which a GUI can be created
able to handle real-time image processing
10/18/2016 Introduction to OpenCV 5
7. OpenCV: pros
Specificity
OpenCV was made for computer vision/image
processing
Matlab is quite generic
Speed
30+ frames processed per seconds in real time image
processing with OpenCV
around 4-5 frames processed per seconds in real time
image processing with Matlab
Efficiency
Matlab needs more system resources than OpenCV
10/18/2016 Introduction to OpenCV 7
8. OpenCV: cons
Easy of use
Matlab won hands down!
Integrated Development Environment (IDE)
you can use Eclipse, Netbeans, Visual Studio, Qt,
XCode, … even a simple text editor for OpenCV
Matlab has is own IDE
Memory management
10/18/2016 Introduction to OpenCV 8
9. OpenCV: pros (final)
Price (!)
OpenCV Wrappers
Emgu CV, ruby-opencv…
More similar to industry-level frameworks
10/18/2016 Introduction to OpenCV 9
10. OpenCV in C++: why not?
The OpenCV APIs have few capabilities for
user interfaces
i.e., you can open a window with an image inside.
Do you want a button? You cannot have it!
to have a “decent” user interface you need to
install “something else” (e.g., Qt for C++)…
… and then compile OpenCV with Qt support
C++ learning curve is quite… steep
source: experiences from teaching this course in
the first years
10/18/2016 Introduction to OpenCV 10
11. OpenCV in C++: why not?
OpenCV C++ APIs exist since OpenCV 1.0
they dramatically changed starting from
OpenCV 2.0
it is possible, but not recommended, to “mix”
different OpenCV version
typically, it is a good way to get into trouble!
10/18/2016 Introduction to OpenCV 11
12. OpenCV in C++: why not?
It is possible to use OpenCV C APIs together
with C++ APIs
typically it is a good way to get into trouble!
OpenCV 3.x is mostly compatible with
OpenCV 2.x
but OpenCV 1.x API is deprecated and partially
removed
10/18/2016 Introduction to OpenCV 12
13. OpenCV in Java: why?
OpenCV provides also Java APIs
nothing to compile (on Windows)
"easy" installation on macOS thanks to
Homebrew
they works on Android, too
Java 7/8 has a first-grade user interface
not Swing, but JavaFX
https://docs.oracle.com/javase/8/javase-
clienttechnologies.htm
10/18/2016 Introduction to OpenCV 13
14. OpenCV in Java: why?
Almost everybody here should have some
(basic) experience with Java
Performances between the C++ APIs and the
Java APIs are comparable
10/18/2016 Introduction to OpenCV 14
15. OpenCV in Java: why?
The OpenCV Java APIs are really similar to the
C++ version
knowledge can be transferred!
examples:
to store an image in memory both use the Mat object
to write an image on disk both use the imwrite() method
Something changes a bit
examples:
CV_RGB2GRAY (C++) becomes COLOR_RGB2GRAY (Java)
Point, Point2d, Point2f (C++) becomes 3 overloaded
constructor of Point (Java)
10/18/2016 Introduction to OpenCV 15
17. OpenCV 3.x
It is modular and extendible
It includes mature and clean algorithms that
are fully supported
A separate contribution repository is for new
algorithms that people want to share
10/18/2016 Introduction to OpenCV 17
OpenCV 3.x
contributions
OpenCV
3.x
OpenCV 2.x
18. Main Modules
10/18/2016 Introduction to OpenCV 18
core
basic structures and algorithms
imgproc
image processing algorithms (such as image filtering,
geometrical image transformations, histograms, etc.)
videoio
media I/O
OpenCV has a modular structure:
each package includes several shared or static libraries
New
19. Main Modules
10/18/2016 Introduction to OpenCV 19
video
video analysis (such as motion estimation and object
tracking)
imgcodecs
image file reading and writing
calib3d
camera calibration and 3D reconstruction
features2d
2D features framework (feature detectors, descriptors, and
descriptor matchers)
objdetect
detection of objects and other items (e.g., faces, eyes,
mugs, people, …)
New
20. Main Modules
10/18/2016 Introduction to OpenCV 20
highgui
provides basic UI capabilities (in C, C++, and Python)
ml
machine learning classes used for statistical
classification, regression and clustering of data
photo
computational photography
cuda*
various modules with CUDA-optimized algorithms
New
Not available in Java
21. Data Structures
10/18/2016 Introduction to OpenCV 21
We speak about Java APIs
All the OpenCV classes and methods are placed
into the org.opencv.* packages
Mat
the primary image structure in OpenCV 2.x/3.x
overcomes the “old” IplImage/CvMat problems
(OpenCV 1.x/C API)
automatic memory management (more or less in C++)
two data parts:
matrix header (contains information about the matrix)
a pointer to the matrix containing the pixel values
22. Duplicate Mat objects…
Mat first = Mat.eye(3, 3, CvType.CV_8U1C);
10/18/2016 Introduction to OpenCV 22
1 0 0
0 1 0
0 0 1
Header
first
23. Duplicate Mat objects…
Mat first = Mat.eye(3, 3, CvType.CV_8U1C);
Mat second = first;
10/18/2016 Introduction to OpenCV 23
1 0 0
0 1 0
0 0 1
Header
first
Header
second
24. Duplicate Mat objects…
Mat first = Mat.eye(3, 3, CvType.CV_8U1C);
Mat second = first;
first.put(0, 0, 2, 2, 2);
10/18/2016 Introduction to OpenCV 24
Header
first
Header
second
2 2 2
0 1 0
0 0 1
25. Duplicate Mat objects…
Mat first = Mat.eye(3, 3, CvType.CV_8U1C);
Mat second = new Mat();
second = first.clone(); // first.copyTo(second);
first.put(0, 0, 2, 2, 2);
10/18/2016 Introduction to OpenCV 25
2 2 2
0 1 0
0 0 1
Header
first
Header
second
1 0 0
0 1 0
0 0 1
26. Data Structures
10/18/2016 Introduction to OpenCV 26
Point
2D point
defined by x, y coordinates
Point first = new Point(2, 3);
Size
2D size structure
specify the size (width and height) of an image
or rectangle
Rect
2D rectangle object
27. Basic Drawing Operations
10/18/2016 Introduction to OpenCV 27
Imgproc.circle()
draws a simple or filled circle with a given center and radius
on a given image
Imgproc.line()
draws a line between two point in the given image
Imgproc.ellipse()
draws an ellipse outline, a filled ellipse, an elliptic arc, a filled
ellipse sector, …
Imgproc.rectangle()
draws a rectangle outline or a filled rectangle
note that negative thickness will fill the rectangle
28. Basic Image I/O
10/18/2016 Introduction to OpenCV 28
Imgcodecs.imread()
loads an image from file and return the corresponding
Mat object
Mat Imgcodecs.imread(String filename,
int flags)
Imgcodecs.imwrite()
save an image on disk
bool Imgcodecs.imwrite(String filename,
Mat img, MatOfInt params)
29. Color Spaces
10/18/2016 Introduction to OpenCV 29
Imgproc.cvtColor
converts an input image from one color space to another
examples:
cvtColor(src, dest, Imgproc.COLOR_RGB2GRAY);
cvtColor(src, dest, Imgproc.COLOR_HSV2BGR);
cvtColor(src, dest, Imgproc.COLOR_RGB2BGR);
Important! Images in OpenCV uses BGR instead of
RGB
31. How can we use OpenCV?
10/18/2016 Introduction to OpenCV 31
LABINF:
already installed under Windows
Java 8 and OpenCV version 3.1
Eclipse (Neon) is the IDE
At home:
we recommend Eclipse Neon, Java 8 and OpenCV
3.1
feel free to use any Java IDE you like
Installation:
http://opencv-java-
tutorials.readthedocs.io/en/latest/01-installing-opencv-
for-java.html
32. What if I got problems?
10/18/2016 Introduction to OpenCV 32
Small problems
drop me a line
luigi.derussis@polito.it
Normal problems
come to office hours
please send an e-mail beforehand
Enormous problems
pray?
no, seriously, we can schedule an extra “lesson”
Awesome student to me
Hi,
[…] I’m using “cvtColor(image, gray, COLOR_BGR2GRAY);” but it give this exception: […]
Can you help me?
Regards,
…
Problems with JavaFX and a gray scale image
33. What if I got problems?
10/18/2016 Introduction to OpenCV 33
Small problems
drop me a line
luigi.derussis@polito.it
Normal problems
come to office hours
please send an e-mail beforehand
Enormous problems
pray?
no, seriously, we can schedule an extra “lesson”
Not-So-Awesome student to me
Hi,
[…] I followed the guide for installing OpenCV on my Mac but I have an error after step 3. Can
we meet on next Wednesday to solve the problem?
Thanks!
Regards,
…
OpenCV installation
34. What if I got problems?
10/18/2016 Introduction to OpenCV 34
Small problems
drop me a line
luigi.derussis@polito.it
Normal problems
come to office hours
please send an e-mail beforehand
Enormous problems
pray?
no, seriously, we can schedule an extra “lesson”
Good student to me
Hi,
[…] I see the solution of Exercise 2.1 but I don’t understand the following expressions:
- main();
- System.out.println();
- @Override.
Can you explain to me what they are?
Regards,
…
Help with OpenCV
35. An e-mail not to be sent!
10/18/2016 Introduction to OpenCV 35
36. Useful Resources…
10/18/2016 Introduction to OpenCV 36
OpenCV Wiki
https://github.com/Itseez/opencv/wiki
OpenCV Official Documentation
http://docs.opencv.org/3.1.0/
User Q&A forum
http://answers.opencv.org/questions/
OpenCV Javadocs
http://docs.opencv.org/java/3.1.0/
JavaFX Documentation
https://docs.oracle.com/javase/8/javase-
clienttechnologies.htm
37. Useful Resources…
10/18/2016 Introduction to OpenCV 37
Daniel Lélis Baggio, OpenCV 3.0 Computer
Vision with Java, PACKT Publishing, 2015,
ISBN 978-1-78328-397-2
Gary Bradski, Adrian Kaehler, Learning
OpenCV 3 - Computer Vision in C++ with the
OpenCV Library, O'Reilly Media, TBP, ISBN
978-1-4919-3799-0
Other books:
http://opencv.org/books.html
39. GUI? No, thanks
Write a simple OpenCV program to:
1. create and print a 3x3 identity matrix
2. read an image from disk
3. convert the image in grayscale
4. save the grayscale image back on disk
10/18/2016 Introduction to OpenCV 39
40. OpenCV meets JavaFX (I)
Write a graphical application with OpenCV to:
1. read an image from disk
2. convert it in grayscale
3. show the result on screen
10/18/2016 Introduction to OpenCV 40
41. OpenCV meets JavaFX (II)
Write a graphical application with OpenCV to:
1. acquire the video stream from the webcam
2. convert it in grayscale
3. show the result of the acquisition on screen
10/18/2016 Introduction to OpenCV 41
42. License
18/10/2016 Computer Vision - Introduction 1 42
This work is licensed under the Creative Commons “Attribution-
NonCommercial-ShareAlike International (CC BY-NC-SA 4.0)”
License.
You are free to:
Share - copy and redistribute the material in any medium or format
Adapt - remix, transform, and build upon the material
for any purpose, even commercially.
Under the following terms:
Attribution - You must give appropriate credit, provide a link to the
license, and indicate if changes were made. You may do so in any
reasonable manner, but not in any way that suggests the licensor
endorses you or your use.
Noncommercial - You may not use the material for commercial
purposes.
Share Alike - If you remix, transform, or build upon the material, you
must distribute your contributions under the same license as the
original.