Color Harmonization

Universidade Federal do Rio de Janeiro - UFRJ - Campus Cidade Universitária - Rio de Janeiro - Ilha do Fundão, CEP: 21941-972 - COPPE/PESC/LCG
Color Harmonization :: Método de Recolorização Baseado em Agrupamentos Harmônicos :: Seminários de Computação Gráﬁca II :: 15 de Agosto de 2013
Color Harmonization
Método de Recolorização Baseado em Agrupamentos Harmônicos
Michel Alves dos Santos
Pós-Graduação em Engenharia de Sistemas e Computação
Universidade Federal do Rio de Janeiro - UFRJ - COPPE
Cidade Universitária - Rio de Janeiro - CEP: 21941-972
Docentes Responsáveis: Prof. Dsc. Ricardo Marroquim & Prof. PhD. Cláudio Esperança
{michel.mas, michel.santos.al}@gmail.com
15 de Agosto de 2013
Michel Alves dos Santos: Laboratório de Computação Gráﬁca - LCG Pós-Graduação em Engenharia de Sistemas e Computação - PESC

Introdução - Qual é a importância das cores?
A Importância das Cores
Figure: O Nascimento de Vênus (Sandro Botticelli) e Composition VII (Wassily Kandinsky). Obras
que encantam pela maestria da combinação de tons, estética apurada e inigualável senso harmônico.
Destacam informação e atraem atenção;
Aumentam a capacidade cognitiva;
Associam síntaxe à semântica.

Perguntas Cotidianas
Perguntas Geradas Pela Importância das Cores

Propriedade em Comum
Propriedade Desejada em Comum
E todas as perguntas feitas anteriormente giram em torno de uma
propriedade intensamente desejada...
A Harmonia

A Apresentação
O que podemos esperar desta apresentação?
Nesta apresentação iremos discutir sobre o conceito de harmonia de cores,
as primeiras intenções de categorização do tema de maneira cientíﬁca e
apresentar uma técnica para busca e aplicação de esquemas harmônicos.

Objetivos
Principais Objetivos da Apresentação
Color Harmonization
Daniel Cohen-Or Olga Sorkine Ran Gal Tommer Leyvand Ying-Qing Xu
Tel Aviv University∗ Microsoft Research Asia†
original image harmonized image
Figure 1: Harmonization in action. Our algorithm changes the colors of the background image to harmonize them with the foreground.
Abstract
Harmonic colors are sets of colors that are aesthetically pleasing
in terms of human visual perception. In this paper, we present a
method that enhances the harmony among the colors of a given
photograph or of a general image, while remaining faithful, as much
as possible, to the original colors. Given a color image, our method
finds the best harmonic scheme for the image colors. It then allows
a graceful shifting of hue values so as to fit the harmonic scheme
while considering spatial coherence among colors of neighboring
pixels using an optimization technique. The results demonstrate
that our method is capable of automatically enhancing the color
“look-and-feel” of an ordinary image. In particular, we show the
results of harmonizing the background image to accommodate the
colors of a foreground image, or the foreground with respect to the
background, in a cut-and-paste setting. Our color harmonization
technique proves to be useful in adjusting the colors of an image
composed of several parts taken from different sources.
Keywords: image enhancement, color harmonization, cut-and-
paste, computational aesthetics
1 Introduction
Much of what we perceive and feel about an image is experienced
through colors. Although our perception of colors depends on the
context, and is culture-related, it is impossible to look at an im-
age without being affected by the harmony of its colors. Harmonic
∗{dcor|sorkine|galran|tommer}@ tau.ac.il
†yqxu@microsoft.com
colors are sets of colors that hold some special internal relation-
ship that provides a pleasant visual perception. Harmony among
colors is not determined by specific colors, but rather by their rel-
ative position in color space. Generating harmonic colors has been
an open problem among artists and scientists [Holtzschue 2002].
Munsell [1969] and Goethe [1971] have defined color harmony as
balance, in an effort to transfer the concept of color harmony from
a subjective perspective to an objective one. Although currently
there is no formulation that defines a harmonic set, there is a con-
sensus among artists that defines when a set is harmonic, and there
are some forms, schemes and relations in color space that describe
a harmony of colors [Matsuda 1995; Tokumaru et al. 2002].
Professional artists usually rely on experience and intuition to
choose their favorite harmonic colors. The artist can choose a
harmonic set from prescribed sets provided in handbooks (e.g.,
[Krause 2002]) or by using an interactive application (e.g., [Meier
1988]). Once the set is defined, the artist needs to color or recolor
his/her product with this set, a task that can be tedious when the
image is complex and contains many colors.
In this paper we introduce a novel application that provides the user
with an automatic recoloring tool, which is useful in different set-
tings. Given an arbitrary image, possibly a photograph, the colors
of the image are modified to enhance the relationship among them
and to increase their harmony. We term this color-enhancement
process color harmonization.
Our technique can deal with an arbitrarily complex image or color
palette, with a rich variety of colors. Color harmonization frees
the designer from choosing a specific harmonic set, since he/she
can use any desired color palette, and our automatic method sub-
sequently optimizes the image towards a harmonic setting while
remaining as faithful as possible to the original color palette. The
application is suitable for both professional designers and amateurs,
seeking to enhance their artistic work.
The harmonization technique can be applied in a compositing sce-
nario in which some regions of the input image remain intact. This
allows, for instance, harmonizing the colors of a background im-
age with respect to the foreground or adapting the colors of a fore-
Debater sobre o conteúdo do artigo
intitulado Color Harmonization.
Listar alguns trabalhos relacionados e
complementares ao tema (backward
and forward papers).
Discutir sobre os requerimentos
necessários a implementação do
método proposto.
Exibir o fluxograma de execução da
técnica categorizando as etapas de
processamento.
Analisar os resultados encontrados e
casos onde o método não surti o
efeito esperado.
Apresentar conclusões (sobre a técnica).
Do que se trata o artigo Color Harmonization?

O Trabalho: Color Harmonization
Busca e Aplicação de Esquemas Harmônicos em Imagens
Color Harmonization
Daniel Cohen-Or Olga Sorkine Ran Gal Tommer Leyvand Ying-Qing Xu
Tel Aviv University∗ Microsoft Research Asia†
original image harmonized image
Figure 1: Harmonization in action. Our algorithm changes the colors of the background image to harmonize them with the foreground.
Abstract
Harmonic colors are sets of colors that are aesthetically pleasing
in terms of human visual perception. In this paper, we present a
method that enhances the harmony among the colors of a given
photograph or of a general image, while remaining faithful, as much
as possible, to the original colors. Given a color image, our method
finds the best harmonic scheme for the image colors. It then allows
a graceful shifting of hue values so as to fit the harmonic scheme
while considering spatial coherence among colors of neighboring
pixels using an optimization technique. The results demonstrate
that our method is capable of automatically enhancing the color
“look-and-feel” of an ordinary image. In particular, we show the
results of harmonizing the background image to accommodate the
colors of a foreground image, or the foreground with respect to the
background, in a cut-and-paste setting. Our color harmonization
technique proves to be useful in adjusting the colors of an image
composed of several parts taken from different sources.
Keywords: image enhancement, color harmonization, cut-and-
paste, computational aesthetics
1 Introduction
Much of what we perceive and feel about an image is experienced
through colors. Although our perception of colors depends on the
context, and is culture-related, it is impossible to look at an im-
age without being affected by the harmony of its colors. Harmonic
∗{dcor|sorkine|galran|tommer}@ tau.ac.il
†yqxu@microsoft.com
colors are sets of colors that hold some special internal relation-
ship that provides a pleasant visual perception. Harmony among
colors is not determined by specific colors, but rather by their rel-
ative position in color space. Generating harmonic colors has been
an open problem among artists and scientists [Holtzschue 2002].
Munsell [1969] and Goethe [1971] have defined color harmony as
balance, in an effort to transfer the concept of color harmony from
a subjective perspective to an objective one. Although currently
there is no formulation that defines a harmonic set, there is a con-
sensus among artists that defines when a set is harmonic, and there
are some forms, schemes and relations in color space that describe
a harmony of colors [Matsuda 1995; Tokumaru et al. 2002].
Professional artists usually rely on experience and intuition to
choose their favorite harmonic colors. The artist can choose a
harmonic set from prescribed sets provided in handbooks (e.g.,
[Krause 2002]) or by using an interactive application (e.g., [Meier
1988]). Once the set is defined, the artist needs to color or recolor
his/her product with this set, a task that can be tedious when the
image is complex and contains many colors.
In this paper we introduce a novel application that provides the user
with an automatic recoloring tool, which is useful in different set-
tings. Given an arbitrary image, possibly a photograph, the colors
of the image are modified to enhance the relationship among them
and to increase their harmony. We term this color-enhancement
process color harmonization.
Our technique can deal with an arbitrarily complex image or color
palette, with a rich variety of colors. Color harmonization frees
the designer from choosing a specific harmonic set, since he/she
can use any desired color palette, and our automatic method sub-
sequently optimizes the image towards a harmonic setting while
remaining as faithful as possible to the original color palette. The
application is suitable for both professional designers and amateurs,
seeking to enhance their artistic work.
The harmonization technique can be applied in a compositing sce-
nario in which some regions of the input image remain intact. This
allows, for instance, harmonizing the colors of a background im-
age with respect to the foreground or adapting the colors of a fore-
Paper: Color Harmonization
Ano: 2006
Autores: Daniel Cohen-Or, Olga Sorkine,
Ran Gal, Tommer Leyvand & Ying-Qing
Xu
Conference: ACM SIGGRAPH 2006
www.igl.ethz.ch/projects/color-harmonization/
O artigo em questão é fortemente baseado nos trabalhos de Masataka Tokumaru (2002) e Yutaka Matsuda (1995).

Trabalhos Relacionados
Trabalhos relacionados a técnica apresentada
Primeiras descobertas no campo da teoria da harmonização:
Newton, Goethe, Young e Maxwell;
Introdução de uma representação quantitativa de harmonia:
Moon & Spencer [1944], Granville & Jacobson [1944];
Moderna Teoria das Cores: Munsell [1969], Ostwald & Birren
[1969] e Itten [1960];
Introdução de um novo círculo de cores onde a harmonia é
enfatizada pela componente de matiz: Itten [1960];
Introdução de 80 esquemas harmônicos baseados nos estudos
de Itten: Matsuda [1995];
Esquemas harmônicos de Tokumaru: Tokumaru [2002].

Genealogia e Trabalhos Complementares
Genealogia e Trabalhos Complementares ao Artigo em Questão
Quais foram os trabalhos que serviram de inspiração principal e
quais foram os trabalhos de destaque que usaram a técnica
proposta na comparação de resultados posteriormente obtidos?
Abstract - Color design is very important for a product design.
In this paper, we propose a system which aims to support such a
color design. Proposed system is composed of 5 parts, such as the
part which evaluates the harmony of colors, the color combining
part, color scheme image judging part, image word output part
and lastly image comparison part. First, the system requires the
user to input a color and his preferring image of color scheme
including his inputting color with image keyword. Next, the system
selects colors from the Munsell color database, which are in
harmony with the color inputted into the system . Then, the system
builds color schemes to combine the color inputted by the user
with the colors selected from the database by the system. Finally,
images of the color schemes are evaluated and outputted the color
combinations whose images accord with the image keyword which
the user inputs into the system. Experimental result shows that
effective judgments of color harmony and color image are executed
and we can get some good color schemes by the system.
I. INTRODUCTION
In resent years, importance of color design increases to design
products. The color design of appearance becomes remarkable
about personal computers and home-use electrification products
in addition to furniture and furnishings. We can enjoy colorful
life, but it may cause new problem to combine colors. When
we plan to purchase new object to place in our living room, we
should select it into many similar objects by considering not
only its function and price but also its color. The color of new
object needs to harmonize well with colors of other existing
objects and interior decorations, but it is very difficult for us to
choose an object considering such things. So we propose a
system to support color design. The system possesses some
advanced functions to support selecting colors of products,
which are evaluating harmony of color scheme, building color
schemes by combining with harmonized colors, color scheme
image judging, color image expression by using image keyword
and so on.
The conventional system of the Kansei information
processing about color design mainly uses the following three
procedures[1]~[5]. 1) The system investigates the relationship
between the colors and the image words by questionnaire. 2) It
quantifies their relationship by factor analysis. 3) It learns the
quantified relationship between input (colors) and output (image
Color Design Support System Considering Color Harmony
Masataka Tokumaru
Faculty of Engineering
Kansai University
toku@ipcku.kansai-u.ac.jp
Noriaki Muranaka
Kansai University
muranaka@ipcku.kansai-u.ac.jp
Shigeru Imanishi
Kansai University
imanishi@k3ki.densi.kansai-u.ac.jp
words) by neural networks[2][3]. This method is convenient
because the system does not require any rules for the relationship
between the input and the output, but it is difficult to correct
parts of the system and to introduce technical knowledge into
the system because it is difficult to grasp the internal state of
the system.
On the other hand, many researchers who study color science
proposed methods and logics to analyze color schemes and color
images. However a lot of them are not a computer system and
they require estimation and interpolation by user about the part
where investigation isn't accomplished. Then we compose a
computer system which automatically designs color scheme
whose harmony is well and whose image corresponds with
user’s preferring color image[6]~[8].
This paper is comprised of 6 chapters. We show the outline
of the system in the next chapter. The system adopts following
two methods to analyze color scheme. One of them is so called
"Matsuda's Color Coordination" which is described in the
chapter 3. Another is "Color Image Scale" which is proposed
by Shigenobu Kobayashi who is director of Nippon Color and
Design Research Institute (NCD). It is explained in the chapter
4. In the chapter 5, we show the experimental result of the
system. We mention the conclusion of our study in the chapter
6.
II. OUTLINE OF THE PROPOSED SYSTEM
Outline of the proposed system is shown in Fig.1. First, user
of the system put a color and his preferring image of color
scheme with image keyword (for example, Casual, Pretty,
Modern and so on) into the system. Next the system chooses
two colors from the color database considering color scheme
harmony and builds the color scheme database to combine them
with a color which user put into the system. Then the system
estimates image of color schemes each of which contains three
colors and compare its image with the image keyword which
the user put into the system. Finally, the user gets some
combinations of three colors which harmonize well with each
other and whose images fit user's imagination by the system.
Color scheme harmony evaluation in the system is based on
the distribution of hues and tones of three combined colors.
Types of hue distribution and tone distribution are expressed
0-7803-7280-8/02/$10.00 ©2002 IEEE
Journal of Electronic Imaging 20(2), 023001 (Apr–Jun 2011)
Color harmonization for images
Zhen Tang
Zhenjiang Miao
Yanli Wan
Zhifei Wang
Beijing Jiaotong University
Institute of Information Science
No. 3 of Shangyuan Residence
Haidian District, Beijing 100044 China
E-mail: tangzhen1005@bjtu.edu.cn
Abstract. Color harmonization is an artistic technique to adjust a
set of colors in order to enhance their visual harmony so that they
are aesthetically pleasing in terms of human visual perception. We
present a new color harmonization method that treats the harmo-
nization as a function optimization. For a given image, we derive a
cost function based on the observation that pixels in a small win-
dow that have similar unharmonic hues should be harmonized with
similar harmonic hues. By minimizing the cost function, we get a har-
monized image in which the spatial coherence is preserved. A new
matching function is proposed to select the best matching harmonic
schemes, and a new component-based preharmonization strategy is
proposed to preserve the hue distribution of the harmonized images.
Our approach overcomes several shortcomings of the existing color
harmonization methods. We test our algorithm with a variety of im-
ages to demonstrate the effectiveness of our approach. © 2011 SPIE
and IS&T. [DOI: 10.1117/1.3574097]
1 Introduction
Much of what we perceive and feel about an image is ex-
perienced through colors. Although our perception of colors
depends on the context and is culture related, it is impossible
to look at an image without being affected by the harmony
of its colors.1
Image color harmonization is an important and
useful image editing operation that is widely used in image
composition, poster and advertisement production, painting,
animation, etc. In recent years, some automatic color harmo-
nization algorithms1,2
have been proposed. However, many
problems still exist as discussed in the following section.
Automatic image color harmonization aims to find the un-
harmonic pixels and shift their colors to a harmonic position
according to a certain harmonic template. A harmonic tem-
plate is a wheel defined on the hue channel of HSV space,
and uses one or two gray sectors to point out its harmonic
positions.3,4
The challenges of automatic image harmoniza-
tion are first to find the best matching harmonic template
and then to harmonize the image in a coherent way. The best
matching template relies on a good matching function and
a good template search strategy. Image harmonization in a
coherent way means that continuous spatial regions in the in-
Paper 10164RR received Oct. 4, 2010; revised manuscript received Feb. 19,
2011; accepted for publication Mar. 7, 2011; published online Apr. 13, 2011.
1017-9909/2011/20(2)/023001/12/$25.00 C 2011 SPIE and IS&T
put image should remain continuous after harmonization, and
this requires a good spatial coherence preservation algorithm.
In this paper, we propose a new optimization-based au-
tomatic color harmonization algorithm for images. This al-
gorithm is based on the observation that pixels in a small
window that have similar unharmonic hues should be as-
signed similar harmonic hues. On the basis of this observa-
tion, we formulate a quadratic cost function and minimize it
with common optimization algorithms. It preserves the spa-
tial coherence of the harmonized image. In order to get the
best matching harmonic template for an image, we propose a
relative-distance–based matching function, which overcomes
the disadvantages of the existing absolute-distance–based
functions. We also propose a novel component-based two-
step template search algorithm that can find the best matching
preharmonic template correctly.
In summary, two contributions are presented in this paper:
1. A new relative-distance–based template matching
function and a component based two-step template
search algorithm are proposed. This can automati-
cally and correctly find the best matching harmonic
schemes for images.
2. A two-level optimization-based color harmonization
algorithm is proposed. It harmonizes the images in
a coherent way and guarantees that the disconnected
areas of the same object are harmonized with simi-
lar harmonic hues—a problem that cannot be solved
using the previous method.1
The remainder of the paper is arranged as follows.
Section 2 is a brief review of related work, and Sec. 3 de-
scribes our algorithm in detail. After showing and analyzing
some experiment results in Sec. 4, the discussion and con-
clusion are presented in Secs. 5 and 6.
2 Related Work
Although color harmony has been a historically studied prob-
lem since the early days,5–8
the definition of color harmony
is still a problem in color harmony research, especially for
those developing color-harmony models based on quantita-
tive evaluations.9
An objective definition of color harmony
was given by Judd and Wyszecki:10
“when two or more colors
seen in neighboring areas produce a pleasing effect, they are
Journal of Electronic Imaging Apr–Jun 2011/Vol. 20(2)023001-1
Downloaded From: http://electronicimaging.spiedigitallibrary.org/ on 11/14/2012 Terms of Use: http://spiedl.org/terms
Os conhecidos backward and forward papers.

Trabalhos Complementares - Moon & Spencer
Geometric Formulation of Classical Color Harmony
Informações
Ano: 1943
Autores: Parry Moon & Domina Eberle
Spencer
Journal: Journal of the Optical
Society of America
Volume 34, Number 1
O artigo em questão possui como propósito apresentar as bases cientíﬁcas para a compreensão teórica da harmonização de cores

Trabalhos Complementares - Tokumaru et al.
Abstract - Color design is very important for a product design.
In this paper, we propose a system which aims to support such a
color design. Proposed system is composed of 5 parts, such as the
part which evaluates the harmony of colors, the color combining
part, color scheme image judging part, image word output part
and lastly image comparison part. First, the system requires the
user to input a color and his preferring image of color scheme
including his inputting color with image keyword. Next, the system
selects colors from the Munsell color database, which are in
harmony with the color inputted into the system . Then, the system
builds color schemes to combine the color inputted by the user
with the colors selected from the database by the system. Finally,
images of the color schemes are evaluated and outputted the color
combinations whose images accord with the image keyword which
the user inputs into the system. Experimental result shows that
effective judgments of color harmony and color image are executed
and we can get some good color schemes by the system.
I. INTRODUCTION
In resent years, importance of color design increases to design
products. The color design of appearance becomes remarkable
about personal computers and home-use electrification products
in addition to furniture and furnishings. We can enjoy colorful
life, but it may cause new problem to combine colors. When
we plan to purchase new object to place in our living room, we
should select it into many similar objects by considering not
only its function and price but also its color. The color of new
object needs to harmonize well with colors of other existing
objects and interior decorations, but it is very difficult for us to
choose an object considering such things. So we propose a
system to support color design. The system possesses some
advanced functions to support selecting colors of products,
which are evaluating harmony of color scheme, building color
schemes by combining with harmonized colors, color scheme
image judging, color image expression by using image keyword
and so on.
The conventional system of the Kansei information
processing about color design mainly uses the following three
procedures[1]~[5]. 1) The system investigates the relationship
between the colors and the image words by questionnaire. 2) It
quantifies their relationship by factor analysis. 3) It learns the
quantified relationship between input (colors) and output (image
Masataka Tokumaru
Kansai University
toku@ipcku.kansai-u.ac.jp
Noriaki Muranaka
Kansai University
muranaka@ipcku.kansai-u.ac.jp
Shigeru Imanishi
Kansai University
imanishi@k3ki.densi.kansai-u.ac.jp
words) by neural networks[2][3]. This method is convenient
because the system does not require any rules for the relationship
between the input and the output, but it is difficult to correct
parts of the system and to introduce technical knowledge into
the system because it is difficult to grasp the internal state of
the system.
On the other hand, many researchers who study color science
proposed methods and logics to analyze color schemes and color
images. However a lot of them are not a computer system and
they require estimation and interpolation by user about the part
where investigation isn't accomplished. Then we compose a
computer system which automatically designs color scheme
whose harmony is well and whose image corresponds with
user’s preferring color image[6]~[8].
This paper is comprised of 6 chapters. We show the outline
of the system in the next chapter. The system adopts following
two methods to analyze color scheme. One of them is so called
"Matsuda's Color Coordination" which is described in the
chapter 3. Another is "Color Image Scale" which is proposed
by Shigenobu Kobayashi who is director of Nippon Color and
Design Research Institute (NCD). It is explained in the chapter
4. In the chapter 5, we show the experimental result of the
system. We mention the conclusion of our study in the chapter
6.
II. OUTLINE OF THE PROPOSED SYSTEM
Outline of the proposed system is shown in Fig.1. First, user
of the system put a color and his preferring image of color
scheme with image keyword (for example, Casual, Pretty,
Modern and so on) into the system. Next the system chooses
two colors from the color database considering color scheme
harmony and builds the color scheme database to combine them
with a color which user put into the system. Then the system
estimates image of color schemes each of which contains three
colors and compare its image with the image keyword which
the user put into the system. Finally, the user gets some
combinations of three colors which harmonize well with each
other and whose images fit user's imagination by the system.
Color scheme harmony evaluation in the system is based on
the distribution of hues and tones of three combined colors.
Types of hue distribution and tone distribution are expressed
0-7803-7280-8/02/$10.00 ©2002 IEEE
Informações
Ano: 2002
Autores: Masataka Tokumaru, Noriaki
Muranaka & Shigeru Imanishi
Conference: Proceedings of the IEEE
International Conference on Fuzzy
Systems
IEEE Press, 378-383
O artigo em questão é fortemente baseado no método ‘Matsuda’s Color Coordination’ de autoria de Yutaka Matsuda (1995).

Trabalhos Complementares - Shapira et al.
Image Appearance Exploration by Model-Based Navigation
Informações
Ano: 2009
Autores: L. Shapira, A. Shamir & D.
Cohen-Or
Conference: Proceedings of the
EUROGRAPHICS 2009
Volume 28 (2009), Number 2
O artigo em questão é baseado no método ‘Gaussian Mixture Model’ (GMM).

Trabalhos Complementares - Zhen Tang et al.
Color Harmonization for Images
Journal of Electronic Imaging 20(2), 023001 (Apr–Jun 2011)
Color harmonization for images
Zhen Tang
Zhenjiang Miao
Yanli Wan
Zhifei Wang
Beijing Jiaotong University
Institute of Information Science
No. 3 of Shangyuan Residence
Haidian District, Beijing 100044 China
E-mail: tangzhen1005@bjtu.edu.cn
Abstract. Color harmonization is an artistic technique to adjust a
set of colors in order to enhance their visual harmony so that they
are aesthetically pleasing in terms of human visual perception. We
present a new color harmonization method that treats the harmo-
nization as a function optimization. For a given image, we derive a
cost function based on the observation that pixels in a small win-
dow that have similar unharmonic hues should be harmonized with
similar harmonic hues. By minimizing the cost function, we get a har-
monized image in which the spatial coherence is preserved. A new
matching function is proposed to select the best matching harmonic
schemes, and a new component-based preharmonization strategy is
proposed to preserve the hue distribution of the harmonized images.
Our approach overcomes several shortcomings of the existing color
harmonization methods. We test our algorithm with a variety of im-
ages to demonstrate the effectiveness of our approach. © 2011 SPIE
and IS&T. [DOI: 10.1117/1.3574097]
1 Introduction
Much of what we perceive and feel about an image is ex-
perienced through colors. Although our perception of colors
depends on the context and is culture related, it is impossible
to look at an image without being affected by the harmony
of its colors.1
Image color harmonization is an important and
useful image editing operation that is widely used in image
composition, poster and advertisement production, painting,
animation, etc. In recent years, some automatic color harmo-
nization algorithms1,2
have been proposed. However, many
problems still exist as discussed in the following section.
Automatic image color harmonization aims to find the un-
harmonic pixels and shift their colors to a harmonic position
according to a certain harmonic template. A harmonic tem-
plate is a wheel defined on the hue channel of HSV space,
and uses one or two gray sectors to point out its harmonic
positions.3,4
The challenges of automatic image harmoniza-
tion are first to find the best matching harmonic template
and then to harmonize the image in a coherent way. The best
matching template relies on a good matching function and
a good template search strategy. Image harmonization in a
coherent way means that continuous spatial regions in the in-
Paper 10164RR received Oct. 4, 2010; revised manuscript received Feb. 19,
2011; accepted for publication Mar. 7, 2011; published online Apr. 13, 2011.
1017-9909/2011/20(2)/023001/12/$25.00 C 2011 SPIE and IS&T
put image should remain continuous after harmonization, and
this requires a good spatial coherence preservation algorithm.
In this paper, we propose a new optimization-based au-
tomatic color harmonization algorithm for images. This al-
gorithm is based on the observation that pixels in a small
window that have similar unharmonic hues should be as-
signed similar harmonic hues. On the basis of this observa-
tion, we formulate a quadratic cost function and minimize it
with common optimization algorithms. It preserves the spa-
tial coherence of the harmonized image. In order to get the
best matching harmonic template for an image, we propose a
relative-distance–based matching function, which overcomes
the disadvantages of the existing absolute-distance–based
functions. We also propose a novel component-based two-
step template search algorithm that can find the best matching
preharmonic template correctly.
In summary, two contributions are presented in this paper:
1. A new relative-distance–based template matching
function and a component based two-step template
search algorithm are proposed. This can automati-
cally and correctly find the best matching harmonic
schemes for images.
2. A two-level optimization-based color harmonization
algorithm is proposed. It harmonizes the images in
a coherent way and guarantees that the disconnected
areas of the same object are harmonized with simi-
lar harmonic hues—a problem that cannot be solved
using the previous method.1
The remainder of the paper is arranged as follows.
Section 2 is a brief review of related work, and Sec. 3 de-
scribes our algorithm in detail. After showing and analyzing
some experiment results in Sec. 4, the discussion and con-
clusion are presented in Secs. 5 and 6.
2 Related Work
Although color harmony has been a historically studied prob-
lem since the early days,5–8
the definition of color harmony
is still a problem in color harmony research, especially for
those developing color-harmony models based on quantita-
tive evaluations.9
An objective definition of color harmony
was given by Judd and Wyszecki:10
“when two or more colors
seen in neighboring areas produce a pleasing effect, they are
Journal of Electronic Imaging Apr–Jun 2011/Vol. 20(2)023001-1
Downloaded From: http://electronicimaging.spiedigitallibrary.org/ on 11/14/2012 Terms of Use: http://spiedl.org/terms
Informações
Ano: 2011
Autores: Zhen Tang, Zhenjiang Miao,
Yanli Wan & Zhifei Wang
Journal: Journal of Electronic Imaging
Volume 20(2), (Apr–Jun 2011)
www.eletronicimaging.spiedigitallibrary.org
O artigo em questão apresenta um novo algoritmo de harmonização baseado em otimização que preserva a coerência espacial.

Trabalhos Complementares - Zhen Tang et al.
Image Composition with Color Harmonization
Informações
Ano: 2012
Autores: Zhen Tang, Zhenjiang Miao,
Yanli Wan & Bo Li
Journal: International Journal of Pattern
Recognition and Artiﬁcial Intelligence
Vol. 26, No. 3 (2012)(23 pages)
O artigo em questão apresenta um método de otimização para lidar com a harmonização de cores na composição de imagens.

Trabalhos Complementares - Xiao-Hui Wang et al.
Aﬀective Image Colorization
Informações
Ano: 2012
Autores: Xiao-Hui Wang, Jia Jia, Han-Yu
Liao & Lian-Hong Cai
Journal: Journal of Computer Science
and Technology - Springer
Nov. 2012, Volume 27, No.6
O artigo em questão propõe um sistema completo de colorização e harmonização (de imagens intensidade) baseado em similaridades.

Conceito de Harmonização
Como podemos entender o conceito de Harmonização?
Harmonização é a propriedade que faz com que determinados conjuntos de cores
detenham uma relação interna especial que consiga prover uma percepção visual
agradável.
Cores harmônicas são prazerosas aos olhos.
Esses conjuntos envolvem o observador humano e fornecem um sentido de
ordem e equilíbrio na experiência visual.

Conceito de Harmonização
Como podemos entender o conceito de Harmonização?
Harmonização é a propriedade que faz com que determinados conjuntos de cores
detenham uma relação interna especial que consiga prover uma percepção visual
agradável.
Cores harmônicas são prazerosas aos olhos.
Esses conjuntos envolvem o observador humano e fornecem um sentido de
ordem e equilíbrio na experiência visual.
« Porém, como encontrar o conjunto de cores mais harmonioso? »

Colorização/Recolorização
Os processos de colorização e recolorização
A obtenção desses conjuntos harmônicos pode ser enfadonha devido devido ao
tedioso trabalho de colorização/recolorização.

É nesse momento que se fazem necessárias algumas abordagens mais inteligentes...

Tais como, as Técnicas de Harmonização!

Técnicas de Harmonização
Veremos a técnica de Harmonização de Cores proposta por
Cohen-Or et al.

Técnica de Harmonização - Diagrama
Diagrama simpliﬁcado da técnica proposta por Cohen-Or et al.

Técnica de Harmonização - Diagrama
Diagrama simpliﬁcado da técnica proposta por Cohen-Or et al.
O método proposto envolve:
Conversão de sistemas de cores, extração e manipulação de histogramas de
matiz (hue), rotulação de pixels e minimização de distâncias entre esquemas e
histogramas extraídos. Tal técnica possui várias aplicações práticas...

Aplicações da Técnica
Aplicações da técnica proposta por Cohen-Or et al.

Auxilia na escolha de paletas de cores que vão compor um cenário;
Auxilia na combinação de cores vindas de diferente fontes ou elementos;
Auxilia na harmonização de imagens advindas do processo de foto-composição.

Auxilia na escolha de paletas de cores que vão compor um cenário;
Auxilia na combinação de cores vindas de diferente fontes ou elementos;
Auxilia na harmonização de imagens advindas do processo de foto-composição.
A seguir, iremos conhecer um pouco mais sobre as etapas da técnica...

Espaço de Cores HSV
Conversão para o espaço de cores HSV
Dimensões perceptualmente mais signiﬁcativas;
HSV = Hue, Saturation and Value (Intensity)

Hue Histogram
Extração do histograma de matiz
Para analisar a harmonia de uma imagem de entrada X devemos analisar seu
histograma de matiz;
Cada pixel p ∈ X contribui com a saturação S(p) e a matiz H(p).

Esquemas Harmônicos
Busca do esquema harmônico mais adequado
Os gabaritos ou modelos harmônicos podem ser arbitrariamente rotacionados;
Um esquema harmônico é composto de um gabarito e uma rotação: Tm + α.
Os esquemas harmônicos são peças importantes na função de harmonização...

Função de Harmonização
Harmonia de uma determinada imagem X
A harmonia de uma imagem X em relação a um esquema (Tm, α) é dada por:
F(X, (Tm, α)) =
p∈X
H(p) − ETm(α)(p) · S(p)
Onde H e S denotam os canais de matiz e saturação da imagem X;
E ETm(α)(p) a borda do(s) setor(es) pertencente(s) ao esquema (Tm, α)

O Melhor Esquema Harmônico
Busca do esquema harmônico mais adequado
Automaticamente computamos α que minimiza F(X, (Tm, α)) para cada Tm
O Esquema mais apropriado é dado por: (Tm, α) = argminm,αF(X, (Tm, α))

O Processo de Harmonização
O processo de harmonização de uma imagem
Dada a imagem X e o esquema harmônico mais apropriado (Tm, α)
Iremos deslocar os valores de matiz para a região compreendida em (Tm, α)
Este deslocamento irá obdecer a uma dada função de comportamento...

Deslocamento de Cores
Como é realizado o deslocamento de cores?
O valor de matiz do pixel p é deslocado para o setor associado a ETm(α)(p);
A quantidade de compressão é controlada por uma função gaussiana de
decaimento (esse núcleo pode ser modiﬁcado para melhores resultados).

Superando Problemas de Segmentação
Como superar problemas de segmentação de regiões?

A maioria dos problemas de segmentação para o método proposto ocorrem
devido a medidas equidistantes de bordas dos setores de um esquema, que
fazem com que valores de matiz sejam deslocados erroneamente.

Para fazer com que a colorização/recolorização seja mais coerente, os autores,
usam uma técnica de otimização de rotulação de pixels associados: graph-cut
otimization. Para maiores informações consultar [Boykov and Jolly, 2001]

Resultados
Resultados da aplicação da técnica apresentada

Resultados
Resultados da aplicação da técnica apresentada
Escolha de cores e paletas;
Harmonização de planos de fundo;
Posters e demais peças publicitárias.

Conclusões
Conclusões a respeito dos tópicos apresentados e do
trabalho
O método proposto aperfeiçoa a harmonia de
uma dada imagem.
Opera através do deslocamento de valores de matiz
para uma determinada distribuição harmônica.
Especialmente útil para ‘cores artiﬁciais’ e
composição/combinação de imagens.

Videos Demonstrativos
Videos Demonstrativos das Técnicas Discutidas
PLAY FULLSCREEN
VIEW INSIDE
PLAY FULLSCREEN
VIEW INSIDE

Uma Último Questionamento...
Será que um dia seremos capazes de:
Captar e descrever o processo criativo e a sensibilidade de maneira
analítica através de procedimentos reproduzíveis por máquinas?

Agradecimentos
Grato Pela Atenção!
Michel Alves dos Santos - michel.mas@gmail.com

Color Harmonization

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Color Harmonization

Semelhante a Color Harmonization (9)

Mais de Michel Alves

Mais de Michel Alves (20)

Último

Último (20)

Color Harmonization