This document discusses the Generic Image Library (GIL), which is part of the Boost C++ libraries. GIL abstracts image representations and algorithms, allowing code to work with any image format. It introduces key GIL concepts like image views, iterators, and locators that provide uniform access to image pixels. Examples are given of using GIL to write algorithms that compute the horizontal and vertical gradients of a grayscale image in a format-independent way.
Cvpr2010 open source vision software, intro and training part iv generic image library - bourdev, henning - agenda - 2010
1. Generic Image
Library
Lubomir Bourdev
Adobe Systems and U.C. Berkeley
Hailin Jin
Adobe Systems.
Christian Henning
Independent consultant
1
2010 Adobe Systems Incorporated. All Rights Reserved.
2. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL extensions
Is GIL for you?
2
2010 Adobe Systems Incorporated. All Rights Reserved.
3. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL extensions
Is GIL for you?
3
2010 Adobe Systems Incorporated. All Rights Reserved.
4. Image Represenations
4x3 image in which the second pixel is hilighted
color space (RGB, CMYK…)
In interleaved (chunky) form:
channel order (RGB vs. BGR)
row padding policy
channel depth 8-bit, 16-bit…
planar vs. interleaved
In planar form:
It is challenging to write code that is reusable and efficient
4
2010 Adobe Systems Incorporated. All Rights Reserved.
5. Generic Image Library (GIL)
GIL abstracts image representations and algorithms
Allows you to write an algorithm once and have it work on any image format
GIL is one of the boost C++ libraries (see www.boost.org)
GIL is all header files – requires no linking
GIL is open-source and free, including for commercial use
5
2010 Adobe Systems Incorporated. All Rights Reserved.
6. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL extensions
Is GIL for you?
6
2010 Adobe Systems Incorporated. All Rights Reserved.
7. Example – Computing the Gradient Image
Simplifying assumptions
- Only horizontal gradient
- Only 8-bit grayscale image
- Central difference:
( - )/2 →
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
7
2010 Adobe Systems Incorporated. All Rights Reserved.
8. Example – Computing the Gradient Image
Simplifying assumptions
- Only horizontal gradient
- Only 8-bit grayscale image
- Central difference
- Ignore boundary cases
( - )/2 →
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
8
2010 Adobe Systems Incorporated. All Rights Reserved.
9. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
}
9
2010 Adobe Systems Incorporated. All Rights Reserved.
10. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
GIL image view
} - Provides uniform access to the pixels of an image
- Lightweight (16 bytes: pointer + dimensions + rowbytes)
10
2010 Adobe Systems Incorporated. All Rights Reserved.
11. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
gray8s_view_t dst = interleaved_view(width, height,
(gray8s_pixel_t*)dst_pix, dst_rowbytes);
x_gradient(src,dst);
}
11
2010 Adobe Systems Incorporated. All Rights Reserved.
12. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
grayscale
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
gray8s_view_t dst = interleaved_view(width, height,
(gray8s_pixel_t*)dst_pix, dst_rowbytes);
x_gradient(src,dst);
}
12
2010 Adobe Systems Incorporated. All Rights Reserved.
13. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
8-bit (unsigned char)
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
gray8s_view_t dst = interleaved_view(width, height,
(gray8s_pixel_t*)dst_pix, dst_rowbytes);
x_gradient(src,dst);
}
13
2010 Adobe Systems Incorporated. All Rights Reserved.
14. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
8-bit (signed char)
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
gray8s_view_t dst = interleaved_view(width, height,
(gray8s_pixel_t*)dst_pix, dst_rowbytes);
x_gradient(src,dst);
}
14
2010 Adobe Systems Incorporated. All Rights Reserved.
15. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
const (read-only)
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
gray8s_view_t dst = interleaved_view(width, height,
(gray8s_pixel_t*)dst_pix, dst_rowbytes);
x_gradient(src,dst);
}
15
2010 Adobe Systems Incorporated. All Rights Reserved.
16. Interfacing with code outside GIL
void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
image view
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);
gray8s_view_t dst = interleaved_view(width, height,
(gray8s_pixel_t*)dst_pix, dst_rowbytes);
x_gradient(src,dst);
}
16
2010 Adobe Systems Incorporated. All Rights Reserved.
17. GIL Concrete Types
rgb32fc_planar_step_view_t
rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …
17
2010 Adobe Systems Incorporated. All Rights Reserved.
18. GIL Concrete Types
rgb32fc_planar_step_view_t
rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …
18
2010 Adobe Systems Incorporated. All Rights Reserved.
19. GIL Concrete Types
rgb32fc_planar_step_view_t
rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …
19
2010 Adobe Systems Incorporated. All Rights Reserved.
20. GIL Concrete Types
rgb32fc_planar_step_view_t
rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …
20
2010 Adobe Systems Incorporated. All Rights Reserved.
21. GIL Concrete Types
rgb32fc_planar_step_view_t
rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …
21
2010 Adobe Systems Incorporated. All Rights Reserved.
22. GIL Concrete Types
rgb32fc_planar_step_view_t
rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …
22
2010 Adobe Systems Incorporated. All Rights Reserved.
23. GIL Concrete Types
rgb32fc_planar_step_view_t
rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …
23
2010 Adobe Systems Incorporated. All Rights Reserved.
24. Simplest Implementation
#include <boost/gil/gil_all.hpp>
using namespace boost::gil;
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
for (int y=0; y<src.height(); ++y)
for (int x=1; x<src.width()-1; ++x)
dst(x,y) = (src(x+1,y) - src(x-1,y)) / 2;
}
24
2010 Adobe Systems Incorporated. All Rights Reserved.
25. The simplest version is too slow
Better to remember the beginning of each row and offset from it
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
25
2010 Adobe Systems Incorporated. All Rights Reserved.
26. Faster Version
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
for (int y=0; y<src.height(); ++y)
{
gray8c_view_t::x_iterator src_it = src.row_begin(y);
gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<src.width()-1; ++x)
dst_it[x] = (src_it[x+1] - src_it[x-1]) / 2;
}
}
26
2010 Adobe Systems Incorporated. All Rights Reserved.
27. Faster Version
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
pixel iterator
for (int y=0; y<src.height(); ++y)
{
gray8c_view_t::x_iterator src_it = src.row_begin(y);
gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<src.width()-1; ++x)
dst_it[x] = (src_it[x+1] - src_it[x-1]) / 2;
}
}
27
2010 Adobe Systems Incorporated. All Rights Reserved.
28. GIL constructs are very lightweight
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
a raw C pointer
for (int y=0; y<src.height(); ++y)
{
gray8c_view_t::x_iterator src_it = src.row_begin(y);
gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<src.width()-1; ++x)
dst_it[x] = (src_it[x+1] - src_it[x-1]) / 2;
}
} raw C pointer indexing operator
28
2010 Adobe Systems Incorporated. All Rights Reserved.
29. Compute gradient in the vertical dimension
I I I I I
I I I I I D D D D D
I I I I I D D D D D
I I I I I D D D D D
I I I I I
29
2010 Adobe Systems Incorporated. All Rights Reserved.
30. Computing the X-Gradient
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
for (int y=0; y<src.height(); ++y)
{
gray8c_view_t::x_iterator src_it = src.row_begin(y);
gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<src.width()-1; ++x)
dst_it[x] = (src_it[x+1] - src_it[x-1]) / 2;
}
}
30
2010 Adobe Systems Incorporated. All Rights Reserved.
31. Computing the Y-Gradient
void y_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
for (int x=0; x<src.width(); ++x)
{
gray8c_view_t::y_iterator src_it = src.col_begin(x);
gray8s_view_t::y_iterator dst_it = dst.col_begin(x);
for (int y=1; y<src.height()-1; ++y)
dst_it[y] = (src_it[y+1] - src_it[y-1]) / 2;
}
}
31
2010 Adobe Systems Incorporated. All Rights Reserved.
32. Computing the Y-Gradient
void y_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
pixel step iterator (8 bytes)
for (int x=0; x<src.width(); ++x)
{
gray8c_view_t::y_iterator src_it = src.col_begin(x);
gray8s_view_t::y_iterator dst_it = dst.col_begin(x);
for (int y=1; y<src.height()-1; ++y)
dst_it[y] = (src_it[y+1] - src_it[y-1]) / 2;
}
} multiplies index by the step
32
2010 Adobe Systems Incorporated. All Rights Reserved.
33. Using Locators
Pixel locators are the 2D equivalents of pixel iterators
Lightweight and fast:
I I I I I
I I I I I D D D D D
I I I I I D D D D D
I I I I I D D D D D
I I I I I
gray8c_view_t::xy_locator loc = src.xy_at(0,1);
loc.x()++;
(“below” – “above”) / 2
loc.y()+=5;
(*dst) = (loc(0,1) - loc(0,-1)) / 2;
33
2010 Adobe Systems Incorporated. All Rights Reserved.
34. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL extensions
Is GIL for you?
34
2010 Adobe Systems Incorporated. All Rights Reserved.
35. Making the Code Generic
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst) {
GIL constructs and algorithms operate on Concepts
You can use your own images, image views, locators, iterators, pixels
channels, color spaces, etc.
35
2006 Adobe Systems Incorporated. All Rights Reserved.
36. Making the Code Generic
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst) {
gil_function_requires<ImageViewConcept<SrcView> >();
Using boost::concept_check to enforce concept syntactic requirements
Zero run-time overhead (gets compiled out)
Sometimes significant compile-time overhead
36
2006 Adobe Systems Incorporated. All Rights Reserved.
37. Making the Code Generic
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst) {
gil_function_requires<ImageViewConcept<SrcView> >();
SrcView must be satisfy all requirements of an image view
37
2006 Adobe Systems Incorporated. All Rights Reserved.
38. Making the Code Generic
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst) {
gil_function_requires<ImageViewConcept<SrcView> >();
gil_function_requires<MutableImageViewConcept<DstView> >();
DstView must be an image view that is mutable (i.e. writable)
38
2006 Adobe Systems Incorporated. All Rights Reserved.
39. Making the Code Generic
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst) {
gil_function_requires<ImageViewConcept<SrcView> >();
gil_function_requires<MutableImageViewConcept<DstView> >();
gil_function_requires<ColorSpacesCompatibleConcept<
typename SrcView::color_space_t,
typename DstView::color_space_t> >();
Both views must have the same number and interpretation of channels.
39
2006 Adobe Systems Incorporated. All Rights Reserved.
40. The Full Version
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst)
{
int max_x = src.width()-1;
for (int y=0; y<src.height(); ++y)
{
typename SrcView::x_iterator src_it = src.row_begin(y);
typename DstView::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<max_x; ++x)
for (int c=0; c<src.num_channels(); ++c)
dst_it[x][c] = (src_it[x+1][c]-src_it[x-1][c])/2;
}
} explicit loop over
the channels
40
2010 Adobe Systems Incorporated. All Rights Reserved.
41. And if you are a generic programming enthusiast…
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst)
{
int max_x = src.width()-1;
for (int y=0; y<src.height(); ++y)
{
typename SrcView::x_iterator src_it = src.row_begin(y);
typename DstView::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1], dst_it[x],
(_1 - _2)/constant(2));
}
}
41
2010 Adobe Systems Incorporated. All Rights Reserved.
42. And if you are a generic programming enthusiast…
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst)
{
int max_x = src.width()-1;
for (int y=0; y<src.height(); ++y)
{
typename SrcView::x_iterator src_it = src.row_begin(y);
typename DstView::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1], dst_it[x],
(_1 - _2)/constant(2));
} GIL channel-level
algorithm
}
42
2010 Adobe Systems Incorporated. All Rights Reserved.
43. And if you are a generic programming enthusiast…
template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst)
{
int max_x = src.width()-1;
for (int y=0; y<src.height(); ++y)
{
typename SrcView::x_iterator src_it = src.row_begin(y);
typename DstView::x_iterator dst_it = dst.row_begin(y);
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1], dst_it[x],
(_1 - _2)/constant(2));
}
} boost::lambda
expression
43
2010 Adobe Systems Incorporated. All Rights Reserved.
44. GIL Channel Level Algorithms
STL GIL
std::for_each gil::static_for_each
std::transform gil::static_transform
std::fill gil::static_fill
std::max_element gil::static_max
std::min_element gil::static_min
Channel algorithms are compile-time recursive (no explicit loop)
Channels are paired semantically (consider RGB and BGR)
44
2006 Adobe Systems Incorporated. All Rights Reserved.
45. GIL Type Metafunctions
Generators
image_type<Channel, Layout, IsPlanar>::type
view_type<Channel, Layout, IsPlanar, …>::type
derived_image_type<Img, Channel, Layout, …>::type
derived_view_type<View, Channel, Layout, …>::type
45
2010 Adobe Systems Incorporated. All Rights Reserved.
46. GIL Type Metafunctions
Generators
image_type<Channel, Layout, IsPlanar>::type
view_type<Channel, Layout, IsPlanar, …>::type
derived_image_type<Img, Channel, Layout, …>::type
derived_view_type<View, use_default, …>::type
Type analysis metafunctions:
[image/view/locator/iterator] + _is_ + [basic/mutable/planar/step]
if (view_is_planar<View>::value) {
…
}
46
2010 Adobe Systems Incorporated. All Rights Reserved.
47. But how generic is it?
47
2010 Adobe Systems Incorporated. All Rights Reserved.
48. Interfacing with Your Code
void XGradientRGB8_BGR16(int w, int h,
const unsigned char* src_pixels, int src_rowbytes,
signed short* dst_pixels, int dst_rowbytes)
{
rgb8c_view_t src = interleaved_view(w,h,(const rgb8_pixel_t*)src_pix,src_rowbytes);
rgb16s_view_t dst = interleaved_view(w,h,( rgb16s_pixel_t*)dst_pix,dst_rowbytes);
x_gradient(src,dst);
}
void XGradientPlanarLAB8_LAB32(int w, int h,
const uint16* src_l, const uint16* src_a, const uint16* src_b, int src_rowbytes,
signed int* dst_pixels, int dst_rowbytes)
{
lab16c_planar_view_t src=planar_lab_view (w,h, src_l,src_a,src_b, src_rowbytes);
lab32s_view_t dst=interleaved_view(w,h,(lab32s_pixel_t*)dst_pixels,dst_rowbytes);
x_gradient(src,dst);
}
48
2010 Adobe Systems Incorporated. All Rights Reserved.
49. Interfacing with Your Code
void XGradientRGB8_BGR16(int w, int h,
const unsigned char* src_pixels, int src_rowbytes,
signed short* dst_pixels, int dst_rowbytes)
{
rgb8c_view_t src = interleaved_view(w,h,(const rgb8_pixel_t*)src_pix,src_rowbytes);
rgb16s_view_t dst = interleaved_view(w,h,( rgb16s_pixel_t*)dst_pix,dst_rowbytes);
x_gradient(src,dst);
any color space
}
void XGradientPlanarLAB8_LAB32(int w, int h,
const uint16* src_l, const uint16* src_a, const uint16* src_b, int src_rowbytes,
signed int* dst_pixels, int dst_rowbytes)
{
lab16c_planar_view_t src=planar_lab_view (w,h, src_l,src_a,src_b, src_rowbytes);
lab32s_view_t dst=interleaved_view(w,h,(lab32s_pixel_t*)dst_pixels,dst_rowbytes);
x_gradient(src,dst);
}
49
2010 Adobe Systems Incorporated. All Rights Reserved.
50. Interfacing with Your Code
void XGradientRGB8_BGR16(int w, int h,
const unsigned char* src_pixels, int src_rowbytes,
signed short* dst_pixels, int dst_rowbytes)
{
rgb8c_view_t src = interleaved_view(w,h,(const rgb8_pixel_t*)src_pix,src_rowbytes);
rgb16s_view_t dst = interleaved_view(w,h,( rgb16s_pixel_t*)dst_pix,dst_rowbytes);
x_gradient(src,dst);
} any channel depth
void XGradientPlanarLAB8_LAB32(int w, int h,
const uint16* src_l, const uint16* src_a, const uint16* src_b, int src_rowbytes,
signed int* dst_pixels, int dst_rowbytes)
{
lab16c_planar_view_t src=planar_lab_view (w,h, src_l,src_a,src_b, src_rowbytes);
lab32s_view_t dst=interleaved_view(w,h,(lab32s_pixel_t*)dst_pixels,dst_rowbytes);
x_gradient(src,dst);
}
50
2010 Adobe Systems Incorporated. All Rights Reserved.
51. Interfacing with Your Code
void XGradientRGB8_BGR16(int w, int h,
const unsigned char* src_pixels, int src_rowbytes,
signed short* dst_pixels, int dst_rowbytes)
any channel ordering
{
rgb8c_view_t src = interleaved_view(w,h,(const rgb8_pixel_t*)src_pix,src_rowbytes);
rgb16s_view_t dst = interleaved_view(w,h,( rgb16s_pixel_t*)dst_pix,dst_rowbytes);
x_gradient(src,dst);
}
void XGradientPlanarLAB8_LAB32(int w, int h,
const uint16* src_l, const uint16* src_a, const uint16* src_b, int src_rowbytes,
signed int* dst_pixels, int dst_rowbytes)
{
lab16c_planar_view_t src=planar_lab_view (w,h, src_l,src_a,src_b, src_rowbytes);
lab32s_view_t dst=interleaved_view(w,h,(lab32s_pixel_t*)dst_pixels,dst_rowbytes);
x_gradient(src,dst);
}
51
2010 Adobe Systems Incorporated. All Rights Reserved.
52. Interfacing with Your Code
void XGradientRGB8_BGR16(int w, int h,
const unsigned char* src_pixels, int src_rowbytes,
signed short* dst_pixels, int dst_rowbytes)
{
rgb8c_view_t src = interleaved_view(w,h,(const rgb8_pixel_t*)src_pix,src_rowbytes);
rgb16s_view_t dst = interleaved_view(w,h,( rgb16s_pixel_t*)dst_pix,dst_rowbytes);
x_gradient(src,dst);
}
void XGradientPlanarLAB8_LAB32(int w, int h,
const uint16* src_l, const uint16* src_a, const uint16* src_b, int src_rowbytes,
signed int* dst_pixels, int dst_rowbytes)
{
lab16c_planar_view_t src=planar_lab_view (w,h, src_l,src_a,src_b, src_rowbytes);
lab32s_view_t dst=interleaved_view(w,h,(lab32s_pixel_t*)dst_pixels,dst_rowbytes);
x_gradient(src,dst);
}
any pixel organization
52
2010 Adobe Systems Incorporated. All Rights Reserved.
53. Reading between the bytes
Channels may not be byte aligned
R R G G G B B B R R G G G B B B R R G G G B B B
Byte 1 Byte 2 Byte 3
typedef packed_image3_type<uint8_t,2,3,3,rgb_layout_t>::type img_t;
Pixels may not be byte aligned:
Pixel 1 Pixel 2 Pixel 3
R R R G G G B B B R R R G G G B B B R R R G G G
Byte 1 Byte 2 Byte 3
typedef bit_aligned_image3_type<3,3,3,rgb_layout_t>::type img_t;
You may need to define operator- and operator/ for new channels
53
2010 Adobe Systems Incorporated. All Rights Reserved.
54. Synthetic Image Views
Define a view of an arbitrary function
Apply an arbitrary view transformation on it
Run an arbitrary algorithm on it
Details in the GIL Tutorial
54
2010 Adobe Systems Incorporated. All Rights Reserved.
55. Extending GIL
Every component of GIL can be replaced
Rachel can make a channel with a range -1..5
Monica can construct his own XYZab color space
Chandler can create a Pixel iterator that has variable step
Matt can make view of an image that is inside a jpeg file
… and you can use your x_gradient on Matt’s view with
Monica’s color space and Rachel’s channel
… and no developers need to coordinate with each other
55
2010 Adobe Systems Incorporated. All Rights Reserved.
56. GIL is generic.
But how fast is it?
Test a complex case (both src and dst are planar, say 8-bit rgb)
Use a very generic version (with function objects and boost::lambda)
56
2010 Adobe Systems Incorporated. All Rights Reserved.
57. Contestant 1: The “generic enthusiasts” version
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1],
dst_it[x], (_1 - _2)/constant(2));
57
2010 Adobe Systems Incorporated. All Rights Reserved.
58. Contestant 1: The “generic enthusiasts” version
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1],
dst_it[x], (_1 - _2)/constant(2));
Planar pointer:
struct {
char *r, *g, *b;
};
58
2010 Adobe Systems Incorporated. All Rights Reserved.
59. Contestant 1: The “generic enthusiasts” version
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1],
dst_it[x], (_1 - _2)/constant(2));
operator[] returns by
value a reference proxy:
struct {
char &r, &g, &b;
};
59
2010 Adobe Systems Incorporated. All Rights Reserved.
60. Contestant 1: The “generic enthusiasts” version
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1],
dst_it[x], (_1 - _2)/constant(2));
- Uses compile-time recursion
- Uses specialization to pair
channels semantically
60
2010 Adobe Systems Incorporated. All Rights Reserved.
61. Contestant 1: The “generic enthusiasts” version
for (int x=1; x<max_x; ++x)
static_transform(src_it[x+1], src_it[x-1],
dst_it[x], (_1 - _2)/constant(2));
boost::lambda
anonymous function object
61
2010 Adobe Systems Incorporated. All Rights Reserved.
62. Contestant 2: Good Ol’ C version
char *sr, *sg, *sb;
unsigned char *dr, *dg, *db;
int max_x;
for (int x=1; x<max_x; ++x) {
dr[x] = (sr[x+1] - sr[x-1]) / 2;
dg[x] = (sg[x+1] - sg[x-1]) / 2;
db[x] = (sb[x+1] - sb[x-1]) / 2;
}
62
2010 Adobe Systems Incorporated. All Rights Reserved.
63. Assembly code of the inner loop (MSVC 8)
Good Ol’ C
00410C12 movzx edx,byte ptr [ecx]
00410C15 movzx eax,byte ptr [ecx-2]
00410C19 sub eax,edx
00410C1B cdq
00410C1C sub eax,edx
00410C1E mov edx,dword ptr [esp+14h]
00410C22 sar eax,1
00410C24 mov byte ptr [edx+esi],al
00410C27 movzx edx,byte ptr [ebx+ecx]
00410C2B mov eax,dword ptr [esp+18h]
00410C2F movzx eax,byte ptr [eax+edi]
00410C33 sub eax,edx
00410C35 cdq
00410C36 sub eax,edx
00410C38 sar eax,1
00410C3A mov byte ptr [esi],al
00410C3C movzx edx,byte ptr [ecx+ebp]
00410C40 movzx eax,byte ptr [edi]
00410C43 sub eax,edx
00410C45 cdq
00410C46 sub eax,edx
00410C48 mov edx,dword ptr [esp+1Ch]
00410C4C sar eax,1
00410C4E mov byte ptr [edx+esi],al
00410C51 add ecx,1
00410C54 add esi,1
00410C57 add edi,1
00410C5A sub dword ptr [esp+38h],1
00410C5F jne gil::x_gradient_no_gil+72h (410C12h)
63
2006 Adobe Systems Incorporated. All Rights Reserved.
64. Assembly code of the inner loop (MSVC 8)
Good Ol’ C Generic Enthusiasts
00414FD0 mov edx,dword ptr [esp+14h]
00410C12 movzx edx,byte ptr [ecx] 00414FD4 movzx edx,byte ptr [edx+ecx]
00410C15 movzx eax,byte ptr [ecx-2] 00414FD8 mov eax,dword ptr [esp+18h]
00410C19 sub eax,edx 00414FDC movzx eax,byte ptr [eax+edi]
00410C1B cdq 00414FE0 sub eax,edx
00410C1C sub eax,edx 00414FE2 cdq
00410C1E mov edx,dword ptr [esp+14h] 00414FE3 sub eax,edx
00410C22 sar eax,1 00414FE5 mov edx,dword ptr [esp+1Ch]
00410C24 mov byte ptr [edx+esi],al 00414FE9 sar eax,1
00410C27 movzx edx,byte ptr [ebx+ecx] 00414FEB mov byte ptr [edx+esi],al
00410C2B mov eax,dword ptr [esp+18h] 00414FEE movzx edx,byte ptr [ecx]
00410C2F movzx eax,byte ptr [eax+edi] 00414FF1 movzx eax,byte ptr [ecx-2]
00410C33 sub eax,edx 00414FF5 sub eax,edx
00410C35 cdq 00414FF7 cdq
00410C36 sub eax,edx 00414FF8 sub eax,edx
00410C38 sar eax,1 00414FFA sar eax,1
00410C3A mov byte ptr [esi],al 00414FFC mov byte ptr [esi],al
00410C3C movzx edx,byte ptr [ecx+ebp] 00414FFE movzx edx,byte ptr [ecx+ebp]
00410C40 movzx eax,byte ptr [edi] 00415002 movzx eax,byte ptr [edi]
00410C43 sub eax,edx 00415005 sub eax,edx
00410C45 cdq 00415007 cdq
00410C46 sub eax,edx 00415008 sub eax,edx
00410C48 mov edx,dword ptr [esp+1Ch] 0041500A mov edx,dword ptr [esp+20h]
00410C4C sar eax,1 0041500E sar eax,1
00410C4E mov byte ptr [edx+esi],al 00415010 mov byte ptr [edx+esi],al
00410C51 add ecx,1 00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1
00410C5A sub dword ptr [esp+38h],1 0041501C sub ebx,1
00410C5F jne gil::x_gradient_no_gil+72h (410C12h) 0041501F jne gil::x_gradient1...+0B0h (414FD0h)
64
2006 Adobe Systems Incorporated. All Rights Reserved.
65. Assembly code of the inner loop (MSVC 8)
Good Ol’ C Generic Enthusiasts
00414FD0 mov edx,dword ptr [esp+14h]
00410C12 movzx edx,byte ptr [ecx] 00414FD4 movzx edx,byte ptr [edx+ecx]
00410C15 movzx eax,byte ptr [ecx-2] 00414FD8 mov eax,dword ptr [esp+18h]
00410C19 sub eax,edx 00414FDC movzx eax,byte ptr [eax+edi]
00410C1B cdq 00414FE0 sub eax,edx
00410C1C sub eax,edx 00414FE2 cdq
00410C1E mov edx,dword ptr [esp+14h] 00414FE3 sub eax,edx
00410C22 sar eax,1 00414FE5 mov edx,dword ptr [esp+1Ch]
00410C24 mov byte ptr [edx+esi],al 00414FE9 sar eax,1
00410C27 movzx edx,byte ptr [ebx+ecx] 00414FEB mov byte ptr [edx+esi],al
00410C2B mov eax,dword ptr [esp+18h] 00414FEE movzx edx,byte ptr [ecx]
00410C2F movzx eax,byte ptr [eax+edi] 00414FF1 movzx eax,byte ptr [ecx-2]
00410C33 sub eax,edx 00414FF5 sub eax,edx
00410C35 cdq 00414FF7 cdq
00410C36 sub eax,edx 00414FF8 sub eax,edx
00410C38 sar eax,1 00414FFA sar eax,1
00410C3A mov byte ptr [esi],al 00414FFC mov byte ptr [esi],al
00410C3C movzx edx,byte ptr [ecx+ebp] 00414FFE movzx edx,byte ptr [ecx+ebp]
00410C40 movzx eax,byte ptr [edi] 00415002 movzx eax,byte ptr [edi]
00410C43 sub eax,edx 00415005 sub eax,edx
00410C45 cdq 00415007 cdq
00410C46 sub eax,edx 00415008 sub eax,edx
00410C48 mov edx,dword ptr [esp+1Ch] 0041500A mov edx,dword ptr [esp+20h]
00410C4C sar eax,1 0041500E sar eax,1
00410C4E mov byte ptr [edx+esi],al 00415010 mov byte ptr [edx+esi],al
00410C51 add ecx,1 00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1
00410C5A sub dword ptr [esp+38h],1 0041501C sub ebx,1
00410C5F jne gil::x_gradient_no_gil+72h (410C12h) 0041501F jne gil::x_gradient1...+0B0h (414FD0h)
Some code shifted around
65
2006 Adobe Systems Incorporated. All Rights Reserved.
66. Assembly code of the inner loop (MSVC 8)
Good Ol’ C Generic Enthusiasts
00414FD0 mov edx,dword ptr [esp+14h]
00410C12 movzx edx,byte ptr [ecx] 00414FD4 movzx edx,byte ptr [edx+ecx]
00410C15 movzx eax,byte ptr [ecx-2] 00414FD8 mov eax,dword ptr [esp+18h]
00410C19 sub eax,edx 00414FDC movzx eax,byte ptr [eax+edi]
00410C1B cdq 00414FE0 sub eax,edx
00410C1C sub eax,edx 00414FE2 cdq
00410C1E mov edx,dword ptr [esp+14h] 00414FE3 sub eax,edx
00410C22 sar eax,1 00414FE5 mov edx,dword ptr [esp+1Ch]
00410C24 mov byte ptr [edx+esi],al 00414FE9 sar eax,1
00410C27 movzx edx,byte ptr [ebx+ecx] 00414FEB mov byte ptr [edx+esi],al
00410C2B mov eax,dword ptr [esp+18h] 00414FEE movzx edx,byte ptr [ecx]
00410C2F movzx eax,byte ptr [eax+edi] 00414FF1 movzx eax,byte ptr [ecx-2]
00410C33 sub eax,edx 00414FF5 sub eax,edx
00410C35 cdq 00414FF7 cdq
00410C36 sub eax,edx 00414FF8 sub eax,edx
one extra
00410C38 sar eax,1 00414FFA sar eax,1
00410C3A mov byte ptr [esi],al 00414FFC mov byte ptr [esi],al
mov
00410C3C movzx edx,byte ptr [ecx+ebp] 00414FFE movzx edx,byte ptr [ecx+ebp]
00410C40 movzx eax,byte ptr [edi] 00415002 movzx eax,byte ptr [edi]
00410C43 sub eax,edx 00415005 sub eax,edx
00410C45 cdq 00415007 cdq
00410C46 sub eax,edx 00415008 sub eax,edx
00410C48 mov edx,dword ptr [esp+1Ch] 0041500A mov edx,dword ptr [esp+20h]
00410C4C sar eax,1 0041500E sar eax,1
00410C4E mov byte ptr [edx+esi],al 00415010 mov byte ptr [edx+esi],al
00410C51 add ecx,1 00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1
00410C5A sub dword ptr [esp+38h],1 0041501C sub ebx,1
00410C5F jne gil::x_gradient_no_gil+72h (410C12h) 0041501F jne gil::x_gradient1...+0B0h (414FD0h)
Some very minor changes
66
2006 Adobe Systems Incorporated. All Rights Reserved.
67. Assembly code of the inner loop (MSVC 8)
Good Ol’ C Generic Enthusiasts
00414FD0 mov edx,dword ptr [esp+14h]
00410C12 movzx edx,byte ptr [ecx] 00414FD4 movzx edx,byte ptr [edx+ecx]
00410C15 movzx eax,byte ptr [ecx-2] 00414FD8 mov eax,dword ptr [esp+18h]
00410C19 sub eax,edx 00414FDC movzx eax,byte ptr [eax+edi]
00410C1B cdq 00414FE0 sub eax,edx
00410C1C sub eax,edx 00414FE2 cdq
00410C1E mov edx,dword ptr [esp+14h] 00414FE3 sub eax,edx
00410C22 sar eax,1 00414FE5 mov edx,dword ptr [esp+1Ch]
00410C24 mov byte ptr [edx+esi],al 00414FE9 sar eax,1
00410C27 movzx edx,byte ptr [ebx+ecx] 00414FEB mov byte ptr [edx+esi],al
00410C2B mov eax,dword ptr [esp+18h] 00414FEE movzx edx,byte ptr [ecx]
00410C2F movzx eax,byte ptr [eax+edi] 00414FF1 movzx eax,byte ptr [ecx-2]
00410C33 sub eax,edx 00414FF5 sub eax,edx
00410C35 cdq 00414FF7 cdq
00410C36 sub eax,edx 00414FF8 sub eax,edx
one extra
00410C38 sar eax,1 00414FFA sar eax,1
00410C3A mov byte ptr [esi],al 00414FFC mov byte ptr [esi],al
mov
00410C3C movzx edx,byte ptr [ecx+ebp] 00414FFE movzx edx,byte ptr [ecx+ebp]
00410C40 movzx eax,byte ptr [edi] 00415002 movzx eax,byte ptr [edi]
00410C43 sub eax,edx 00415005 sub eax,edx
00410C45 cdq 00415007 cdq
00410C46 sub eax,edx
indexed
00415008 sub eax,edx
00410C48 mov edx,dword ptr [esp+1Ch] 0041500A mov edx,dword ptr [esp+20h]
00410C4C sar eax,1
mem.
0041500E sar eax,1
00410C4E mov byte ptr [edx+esi],al 00415010 mov byte ptr [edx+esi],al
00410C51 add ecx,1
addressing
00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1
00410C5A sub dword ptr [esp+38h],1 0041501C sub ebx,1
00410C5F jne gil::x_gradient_no_gil+72h (410C12h) 0041501F jne gil::x_gradient1...+0B0h (414FD0h)
Some very minor changes
No measurable performance difference
67
2006 Adobe Systems Incorporated. All Rights Reserved.
68. GIL is generic
GIL is fast
But how readable is it?
68
2010 Adobe Systems Incorporated. All Rights Reserved.
69. But how readable is it?
Generic:
for (int x=1; x<max_x; ++x)
static_transform(src_it[x-1], src_it[x+1],
dst_it[x], (_1 - _2)/constant(2));
Non-generic:
for (int x=1; x<max_x; ++x) {
dr[x] = (sr[x+1] - sr[x-1]) / 2;
dg[x] = (sg[x+1] - sg[x-1]) / 2;
db[x] = (sb[x+1] - sb[x-1]) / 2;
}
69
2010 Adobe Systems Incorporated. All Rights Reserved.
71. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL extensions
Is GIL for you?
71
2010 Adobe Systems Incorporated. All Rights Reserved.
72. Using Image View Transformations
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
template <typename SrcView, typename DstView>
void y_gradient(const SrcView& src, const DstView& dst)
{
x_gradient(rotated90cw_view(src),
rotated90cw_view(dst));
}
72
2010 Adobe Systems Incorporated. All Rights Reserved.
73. Using Image View Transformations
I
I
I
I
I
D D D
D D D
D D D
D D D
D D D
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
template <typename SrcView, typename DstView> I
void y_gradient(const SrcView& src, const DstView& dst)
{
x_gradient(rotated90cw_view(src),
rotated90cw_view(dst));
} returns a new view
Fast: no copying of data!
73
2010 Adobe Systems Incorporated. All Rights Reserved.
74. Compute gradient just over the N-th channel
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
x_gradient(nth_channel_view(src, n), dst);
returns a step view
74
2010 Adobe Systems Incorporated. All Rights Reserved.
75. Compute gradient just over the N-th channel
R R R R R G G G G G B B B B B
R R R R R G G G G G B B B B B
R R R R R G G G G G B B B B B
R R R R R G G G G G B B B B B
R R R R R G G G G G B B B B B
x_gradient(nth_channel_view(src, n), dst);
returns a simple, non-
step view
75
2010 Adobe Systems Incorporated. All Rights Reserved.
76. Compute gradient just over the even pixels
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
R G B R G B R G B R G B R G B
x_gradient(subsampled_view(src,2,2), dst);
76
2010 Adobe Systems Incorporated. All Rights Reserved.
77. Compute luminosity gradient of the image
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
x_gradient(color_converted_view<gray8_pixel_t>(src),dst);
77
2010 Adobe Systems Incorporated. All Rights Reserved.
79. Sample Code – Using Image View Transformations
jpeg_read_image(in_dir + "monkey.jpg“, img);
step1 = subimage_view(view(img), 200,300,150,150);
step2 = color_converted_view<gray8_pixel_t>(step1);
step3 = rotated180_view(step2);
step4 = subsampled_view(step3, 2,1);
x_gradient(step4, grad_view);
No extra memory used
No copying of pixels (working on original data)
All the work is done inside x_gradient
79
2010 Adobe Systems Incorporated. All Rights Reserved.
80. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL extensions
Is GIL for you?
80
2010 Adobe Systems Incorporated. All Rights Reserved.
81. Subimage View
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
81
2010 Adobe Systems Incorporated. All Rights Reserved.
82. Subimage View
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
82
2010 Adobe Systems Incorporated. All Rights Reserved.
83. Subimage View
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst) {
if (src.width()<=2) return;
x_gradient_unsafe(
subimage_view(src,1,0,src.width()-2,src.height()),
subimage_view(dst,1,0,src.width()-2,src.height())
);
}
83
2010 Adobe Systems Incorporated. All Rights Reserved.
84. Subimage View
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst) {
if (src.width()<=2) return;
x_gradient_unsafe(
subimage_view(src,1,0,src.width()-2,src.height()),
subimage_view(dst,1,0,src.width()-2,src.height())
);
}
84
2010 Adobe Systems Incorporated. All Rights Reserved.
85. Subimage View
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst) {
if (src.width()<=2) return;
x_gradient_unsafe(
subimage_view(src,1,0,src.width()-2,src.height()),
subimage_view(dst,1,0,src.width()-2,src.height())
);
}
85
2010 Adobe Systems Incorporated. All Rights Reserved.
86. Subimage View
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst) {
if (src.width()<=2) return; canonical algorithm
x_gradient_unsafe(
subimage_view(src,1,0,src.width()-2,src.height()),
subimage_view(dst,1,0,src.width()-2,src.height())
);
}
86
2010 Adobe Systems Incorporated. All Rights Reserved.
88. 1-D pixel iterators
void x_gradient_unsafe(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
gray8c_view_t::iterator src_it = src.begin();
gray8s_view_t::iterator dst_it = dst.begin();
while (dst_it!=dst.end()) {
*dst_it = (src_it.x()[1] - src_it.x()[-1]) / 2;
++src_it;
++dst_it;
} P P P P
} 1-D iterator over all pixels:
P P P P
P P P P
88
2010 Adobe Systems Incorporated. All Rights Reserved.
90. Image View as 1-D array
void x_gradient_unsafe(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
for (size_t i=0; i<src.size(); ++i)
dst[i] = (src.at(i).x()[1] - src.at(i).x()[-1])/2;
}
90
2010 Adobe Systems Incorporated. All Rights Reserved.
91. Image View as 1-D array
void x_gradient_unsafe(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
for (size_t i=0; i<src.size(); ++i)
dst[i] = (src.at(i).x()[1] - src.at(i).x()[-1])/2;
}
Can use view like
std::vector
91
2010 Adobe Systems Incorporated. All Rights Reserved.
92. STL-like pixel algorithms
struct half_x_difference {
int operator()(const gray8c_loc_t& src_loc) const {
return (src_loc.x()[1] - src_loc.x()[-1]) / 2;
}
};
void x_gradient_unsafe(const gray8c_view_t& src,
const gray8s_view_t& dst) {
transform_pixel_positions(src, dst,
half_x_difference());
}
GIL pixel-level algorithm
92
2010 Adobe Systems Incorporated. All Rights Reserved.
93. GIL Pixel-Level Algorithms
gil::fill_pixels std::fill
gil::for_each_pixel std::for_each
gil::transform_pixels std::transform
gil::equal_pixels std::equal
gil::copy_pixels std::copy
gil::for_each_pixel_position adobe::for_each_position
gil::transform_pixel_positions
gil::copy_and_convert_pixels
93
2006 Adobe Systems Incorporated. All Rights Reserved.
94. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL extensions
Is GIL for you?
94
2010 Adobe Systems Incorporated. All Rights Reserved.
95. GIL Extensions
I/O (Christian Henning) Support for reading/writing common
read/write jpeg/tiff/png/bmp image formats
Dynamic Image (Lubomir Bourdev, Hailin Jin)
Support for images with runtime instantiated types
Support for images whose type
is specified at run time
Numeric (Hailin Jin, Lubomir Bourdev)
bilinear resampling, 1D convolution (resize, blur, etc)
Basic image processing
FreetypeGIL (Tom Brinkman)
Freetype wrapper to GIL
GIL2.Bindings (Hirotaka Niitsuma)
GIL wrappers for popular
image libraries
Interface to VIGRA, OpenCV, uBLAS, TEO
OpenCV (Christian Henning)
Improved Debugging
Support for OpenCV algorithms
SDL (Christian Henning)
Improves debugging – displays current image in a window Multithreading
GIL-Threaded (Victor Bogado)
multithreading and multi-core support for GIL algorithms
Rendering with antialiasing
WulineGIL (Tom Brinkman)
Draw antialiased lines using the Wu algorithm
Toolbox (Christian Henning)
Additional color spaces
Support for HSL and HSV color spaces and some new RGB algorithms
95
2010 Adobe Systems Incorporated. All Rights Reserved.
96. GIL Extensions Wanted
Comprehensive image processing
FFT, even 2D convolution, high-quality efficient resampling
Computer vision algorithms
Canny edge detection, SIFT, GB, segmentation, HOG…
Support for video and multi-dimensional data
Support for Intel’s IPP
Can you help?
96
2010 Adobe Systems Incorporated. All Rights Reserved.
97. Agenda
What is GIL?
Basic concepts and navigation
Generic code
Image view transformations
1D iterators; pixel algorithms
GIL Extensions
Is GIL for you?
97
2010 Adobe Systems Incorporated. All Rights Reserved.
98. “I would use GIL but…”
1. “… I can’t hand-tune and SSE-optimize generic code”
2. “… I am planning to adopt / have adopted another imaging library”
3. “… I already have imaging code. I can’t afford to throw it away”
4. “… I don’t want to have any extra library dependencies”
5. “… who is going to support me?”
6. “… I am concerned about portability. How portable is GIL?”
7. “… compile errors in generic code are a nightmare”
8. “… GIL has a steep learning curve”
9. “… generic code can result in code bloat”
10. “… generic code is opaque. I can’t easily tell if the generated code is
efficient”
98
2010 Adobe Systems Incorporated. All Rights Reserved.
99. Can generic code be as efficient as hand-written code?
GENERIC copy_pixels while (first!=last)
*dst++ = *first++;
copy copy copy copy copy
chunky rgb chunky rgb planar rgb planar rgb planar cmyk
CONCRETE chunky rgb planar rgb chunky rgb planar rgb planar cmyk
99
2010 Adobe Systems Incorporated. All Rights Reserved.
100. Concrete versions can be made much faster
GENERIC copy_pixels while (first!=last)
*dst++ = *first++;
an order of magnitude faster
than the generic version
memmove(…)
memmove(…)
memmove(…)
copy copy copy copy copy
chunky rgb chunky rgb planar rgb planar rgb planar cmyk
CONCRETE chunky rgb planar rgb chunky rgb planar rgb planar cmyk
100
2010 Adobe Systems Incorporated. All Rights Reserved.
101. GIL provides overloads at appropriate level of genericity
GENERIC copy_pixels while (first!=last)
*dst++ = *first++;
memmove each
single color plane
memmove
SEMI- copy copy
GENERIC chunky to planar to
chunky planar
copy copy copy copy copy
chunky rgb chunky rgb planar rgb planar rgb planar cmyk
CONCRETE chunky rgb planar rgb chunky rgb planar rgb planar cmyk
101
2010 Adobe Systems Incorporated. All Rights Reserved.
102. Advantage: New image types can be handled efficiently
GENERIC copy_pixels while (first!=last)
*dst++ = *first++;
memmove each
single color plane
memmove
SEMI- copy copy
GENERIC chunky to planar to
chunky planar
copy copy copy copy copy copy
chunky rgb chunky rgb planar rgb planar xylab planar rgb planar cmyk
CONCRETE chunky rgb planar rgb chunky rgb planar xylab planar rgb planar cmyk
102
2010 Adobe Systems Incorporated. All Rights Reserved.
103. “I would use GIL but…”
1. “… I can’t hand-tune and SSE-optimize generic code”
2. “… I am planning to adopt / have adopted another imaging library”
3. “… I already have imaging code. I can’t afford to throw it away”
4. “… I don’t want to have any extra library dependencies”
5. “… who is going to support me?”
6. “… I am concerned about portability. How portable is GIL?”
7. “… compile errors in generic code are a nightmare”
8. “… GIL has a steep learning curve”
9. “… generic code can result in code bloat”
10. “… generic code is opaque. I can’t easily tell if the generated code is
efficient”
103
2010 Adobe Systems Incorporated. All Rights Reserved.
104. “I would use GIL but…”
1. “… I can’t hand-tune and SSE-optimize generic code”
2. “… I am planning to adopt / have adopted another imaging library”
3. “… I already have imaging code. I can’t afford to throw it away”
4. “… I don’t want to have any extra library dependencies”
5. “… who is going to support me?”
6. “… I am concerned about portability. How portable is GIL?”
7. “… compile errors in generic code are a nightmare”
8. “… GIL has a steep learning curve”
9. “… generic code can result in code bloat”
10. “… generic code is opaque. I can’t easily tell if the generated code is
efficient”
104
2010 Adobe Systems Incorporated. All Rights Reserved.
105. “I would use GIL but…”
1. “… I can’t hand-tune and SSE-optimize generic code”
2. “… I am planning to adopt / have adopted another imaging library”
3. “… I already have imaging code. I can’t afford to throw it away”
4. “… I don’t want to have any extra library dependencies”
5. “… who is going to support me?”
6. “… I am concerned about portability. How portable is GIL?”
7. “… compile errors in generic code are a nightmare”
8. “… GIL has a steep learning curve”
9. “… generic code can result in code bloat”
10. “… generic code is opaque. I can’t easily tell if the generated code is
efficient”
105
2010 Adobe Systems Incorporated. All Rights Reserved.
106. “I would use GIL but…”
1. “… I can’t hand-tune and SSE-optimize generic code”
2. “… I am planning to adopt / have adopted another imaging library”
3. “… I already have imaging code. I can’t afford to throw it away”
4. “… I don’t want to have any extra library dependencies”
5. “… who is going to support me?”
6. “… I am concerned about portability. How portable is GIL?”
7. “… compile errors in generic code are a nightmare”
8. “… GIL has a steep learning curve”
9. “… generic code can result in code bloat”
10. “… generic code is opaque. I can’t easily tell if the generated code is
efficient”
106
2010 Adobe Systems Incorporated. All Rights Reserved.
107. “I would use GIL but…”
1. “… I can’t hand-tune and SSE-optimize generic code”
2. “… I am planning to adopt / have adopted another imaging library”
3. “… I already have imaging code. I can’t afford to throw it away”
4. “… I don’t want to have any extra library dependencies”
5. “… who is going to support me?”
6. “… I am concerned about portability. How portable is GIL?”
7. “… compile errors in generic code are a nightmare”
8. “… GIL has a steep learning curve”
9. “… generic code can result in code bloat”
10. “… generic code is opaque. I can’t easily tell if the generated code is
efficient”
107
2010 Adobe Systems Incorporated. All Rights Reserved.