Ph.D Dissertation Defense Slides on Efficient VLSI Architectures for Image Enhancement Techniques

Introduction
Motivation & Objectives
Contributions
Conclusions & Future Scope
Eﬃcient VLSI Architectures for Image
Enhancement Techniques
Ph. D Dissertation Defense
M. C. Hanumantharaju - 1DS07MEN02
Research Scholar
Dr. M. Ravishankar
Research Advisor
Department of Information Science and Engineering
Dayananda Sagar College of Engineering, Bangalore-560078
March 7, 2014
M. C, Hanumantharaju, Ph.D Dissertation Defense on Development of VLSI Architectures for Image Enhancement Techniques: slide 1/146

Introduction
Contributions
Eﬃcient VLSI Architectures for Image
Enhancement Techniques
Ph. D Dissertation Defense
M. C. Hanumantharaju - 1DS07MEN02
Research Scholar
Dr. M. Ravishankar
Research Advisor
Department of Information Science and Engineering
Dayananda Sagar College of Engineering, Bangalore-560078
March 7, 2014

Introduction
Contributions
Outline
1 Introduction
2 Motivation & Objectives
3 Contributions
AROF Architectures and Its FPGA Implementation
Adaptive Color Image Enhancement using GMF
Gaussian Image Enhancement: Algorithm &
Architecture
4 Conclusions & Future Scope

Introduction
Contributions
Image Enhancement : An Introduction
Image processing
2-D signal processing
Improves characteristics, properties and parameters
Image enhancement
Key step in image processing
Modiﬁes the attributes of an image
Makes it appropriate for analysis, diagnosis, and display.
Some of the image enhancement applications include
Sharpening: improves car license plate number
Contrast enhancement: medical image enhancement.
Edge enhancement: enhances objects in aerial image.
The realm of image enhancement wraps up
Reconstruction & Restoration
Filtering
Segmentation
Compression & Transmission

Introduction
Contributions
Image enhancement : An Example
(a) Original Image (b) Enhanced Image

Introduction
Contributions
Challenges
Image enhancement algorithms have numerous parame-
ters to specify and that needs to be adjusted to obtain
satisfactory results.
Lack of integrated algorithms.
Presently, image enhancement research demands better
reconstruction of high quality images than possible with
available researcher methods.
Image enhancement algorithms depends on the input im-
ages instead of adapting to its local features.
Limited speed achieved in software implementation since
image enhancement algorithms consists of large array of
data.

Introduction
Contributions
Choice of Implementations
General Purpose Processors
(GPPs)
Flexible.
Technology limits the pro-
cessing speed.
Limited performance.
Instruction sets are not
suitable for fast processing
of high resolution images.
GPP instructions are se-
quential and hence system
throughput decreases.
Digital Signal Processors
(DSPs)
Improvement over GPPs.
Falls between GPPs and
ASICs.
Inadequate pipelining and
parallel processing.
Fixed architectures that
limits the performance.
Parallel operation is possi-
ble with multiple DSPs.

Introduction
Contributions
Choice of Implementation Cont.,
Application Specific ICs
(ASICs)
Fast & efficient.
Fixed circuit.
Large time to market.
High cost, except for large
volume commercial appli-
cations.
No optimization.
Field Programmable Gate
Arrays (FPGAs)
High throughput.
Dynamically reconfig-
urable.
Massive pipelining and
parallelism.
Cost effective.
Attractive choice for real-
ization of DIP algorithms.
Present Research Work uses FPGAs for
Implementation

Introduction
Contributions
Motivation
The motivation behind this work is to bring out the fea-
tures in the image that are not clearly visible owing to
different illumination conditions.
Current market demands better reconstruction of high
quality images than is possible with currently available
research outputs.
Limitations of image enhancement schemes: difficult to
tune parameters, deficit of integrated algorithms, lack of
quantitative standard, dependence on inputs instead of
adapting to local features.
Software implementation : inadequate speed.
Hardware implementation of image enhancement algo-
rithm is in great demand for applications such as medical,
forensic and surveillance etc.

Introduction
Contributions
Objectives
Development of efficient VLSI architectures for image en-
hancement algorithms.
Design & simulate the algorithm using software approach
(C or Matlab).
Test the algorithm for images having different environ-
mental conditions.
Realize the algorithm using HDL (Verilog or VHDL).
Verify both software & hardware implementation results.
Evaluate the efficiency of the algorithm using performance
metrics such as PSNR, contrast, luminance, IEF and wavelet
energy etc.
Compare proposed approach with other existing methods.

Introduction
Contributions
Hardware Design Flow

Introduction
Contributions
Gaussian Image Enhancement: Algorithm & Architecture
Adaptive Rank Order Filter (AROF)
Non-linear ﬁlter.
AROF is a powerful technique for denoising an image cor-
rupted by salt & pepper noise or impulse noise.
Impulse noise is often introduced into digital images dur-
ing image acquisition or Interference during transmission.
AROF not only adapts ﬁlter output but also window size
: iterative algorithm.
AROF window expands : All Pixels within the current
window are noisy or median itself is noisy.

Introduction
Contributions
Adaptive Rank Order Filter : Flow Chart

Introduction
Contributions
Noisy Lena (90% Noise Level) and Restored Image
Figure: First Image : Lena Image with High Noise Density (90% Salt &
Pepper Noise) Second Image : Restored Lena Image using AROF.

Introduction
Contributions
Related Work
Andreadis et al.1 proposed FPGA implementation of real-
time adaptive image impulse noise suppression. However,
the system slows down for highly corrupted images.
An efficient hardware implementation of weighted median
filter using cumulative histogram proposed by Fahmy et
al.2 reduces impulse noise satisfactorily. However, hard-
ware complexity is high for smaller window size.
1
I. Andreadis, G. Louverdis, ”Real-time Adaptive Image Impulse Noise Sup-
pression”, IEEE Tran. on Instrumentation and Measurement, Vol. 53, Issue 3,
pp. 798-806, 2004.
2
S. A Fahmy, P.Y.K. Cheung and W. Luk, ”Novel FPGA-based implementa-
tion of median and weighted median filters for image processing”, International
Conference on Field Programmable Logic and Applications, pp. 142-147, 2005.

Introduction
Contributions
Related Work Cont..,
Meena et al.3 proposed a optimized architectures for rank
Order filter. However, optimizations are done for sorting
network with out considering noise levels in an image.
Chih et al.4 proposed an efficient denoising architecture
for impulse noise removal in images. Although, decision
tree based approach used in this scheme is effective for
hardware implementation, technique may not provide sat-
isfactory results for images corrupted with high noise den-
sity.
3
S. M Meena and K. Linganagouda, ”Implementation and Analysis of Opti-
mized Architectures for Rank Order Filter”, Journal of Real Time Image Pro-
cessing, Vol. 3, Issue 1-3, pp. 33-41, 2008.
4
Chih-Yuan Lien, Chien-Chuan Huang, Pei-Yin Chen and Yi-Fan Lin in IEEE
Tran. on Computers, Vol. 62, No. 4, pp. 631-643, April 2013.

Introduction
Contributions
Advantages of Proposed Method
The proposed AROF reduces impulse noise without
degrading image information.
The reconstructed images using AROF provides better
visual quality than possible with Adaptive Median Filter
(AMF).
AROF consists of sliding window, sorting network,
median selection etc. Therefore, the proposed AROF is
best suited for FPGA implementation.
AROF has better performance than the AMF when the
noise density is moderate or high.

Introduction
Contributions
Adaptive Rank Order Filter : Illustration

Introduction
Contributions
Adaptive Rank Order Filter : Illustration Cont..,

Introduction
Contributions

Introduction
Contributions
Top Level Module of AROF

Introduction
Contributions
Detailed Architecture of AROF

Introduction
Contributions
3 × 3 Sliding Window Example

Introduction
Contributions
Detailed Architecture of 3 × 3 Sliding Window

Introduction
Contributions

Introduction
Contributions
Noise Detection Unit and Sorting Network Modules
(a) Impulse Noise Detector (b) Nine Element Sorting Module

Introduction
Contributions
Details of Nine Element Sorter

Introduction
Contributions
Architecture of Compare & Swap Module
(c) Top Architecture of Compare &
Swap
(d) Detailed Architecture of Com-
pare & Swap

Introduction
Contributions
Sorting Networks : Comparison for 9 Elements
Bubble sort : 36 comparators, 14 parallel Operations.
Bose Nelson sort : 27 comparators, 11 parallel
operations.
Hibbard sort : 27 comparators, 12 parallel operations.
Bitonic sort5 : 28 comparators, 8 parallel operations.
Batchers merge exchange sort6 : 27 comparators, 8
parallel operations.
Optimal sort : 25 comparators, 8 parallel operations
(proposed).
5
Zdenek Vasicek and Lukas Sekanina, ”Novel Hardware Implementation of
Adaptive Median Filters”, in proceedings of 11th IEEE Workshop on Design and
Diagnostics of Electronic Circuits and Systems, pp. 1-6, April, 2008.
6
Baddar, Sherenaz W. Al-Haj, and Kenneth E. Batcher, ”The AKS Sorting
Network,” Designing Sorting Networks, Springer New York, pp. 73-80, 2011.

Introduction
Contributions
Matlab Simulation Results of AMF and AROF for
”Lena” Image
Figure: First Row: Original Images with Noise Level 20%, 40%, & 60%,
Second Row: AMF Reconstructed Images, Third Row: AROF
Recontructed Images

Introduction
Contributions
Matlab Simulation Results of AMF and AROF for
”House” Image
Figure: First Row: Original Images with Noise Level 20%, 40%, & 60%,
Second Row: AMF Reconstructed Images, Third Row: AROF
Recontructed Images

Introduction
Contributions
PSNR & IEF Computation
The Peak Signal to Noise Ration (PSNR) & Image Enhancement
Factor (IEF) are computed as follows:
PSNR = 10 log10
2552
MSE
MSE = 1
M×N
M
i=1
N
j=1
ˆI(i, j) − I(i, j)
2
where E(x, y) is the enhanced gray element at position (x, y), I(x,
y) is the original gray element at position (x, y) and, p and q denote
the size of the gray image.
IEF =
M
i=1
N
j=1[n(i,j)−I(i,j)]2
M
i=1
N
j=1[f (i,j)−I(i,j)]2
where n(x, y) is the noisy image, I(x, y) is the original image and
f(x, y) is the reconstructed image.

Introduction
Contributions
Quality Assessment using PSNR (dB) and IEF for
Lena and House Image
Figure: First Row: PSNR for Lena & House Image, Second Row: IEF
for Lena & House Image

Introduction
Contributions
ModelSim Simulation Results : Validity of Input

Introduction
Contributions
ModelSim Simulation Results : AROF Pixel Output

Introduction
Contributions
Timing Diagram for Illustrating the Pipelining
Operation of AROF System

Introduction
Contributions
RTL View of the Top Module AROF System

Introduction
Contributions
Zoomed View of the Top Module
Figure: U1: Impulse Noise Detection, U2: Sliding Window, U3: Sorting
Network, U4: Median Computation, U5: Delay Unit and U6: Output
Selection

Introduction
Contributions
FPGA (XC5VLX50-1FF1153) Device Utilization

Introduction
Contributions
Timing Summary
The timing summary for the AROF system as reported by Xilinx ISE
tool is as follows:
Speed Grade : -1
Minimum period: 4.392ns (Maximum Frequency:
227.668 MHz)
Minimum input arrival time before clock: 1.154ns
Maximum output required time after clock: 4.101ns
Clock period: 4.392ns (frequency: 227.668MHz)
Total number of paths / destination ports: 50736 / 5200
Delay: 4.392ns (Levels of Logic = 5)
Total number of paths / destination ports: 8 / 8
Oﬀset: 1.154ns (Levels of Logic = 1)

Introduction
Contributions
Matlab & Hardware Results of ”Butterﬂy” Image
Figure: First Row: Original Images with Noise Levels 20%, 40%, &
60%, Second Row: Matlab Reconstructed Images, Third Row:
Hardware Recontructed Images

Introduction
Contributions
Matlab & Hardware Results of ”Traﬃc Signal”
Image
Figure: First Row: Original Images with Noise Levels 20%, 40%, &
60%, Second Row: Matlab Reconstructed Images, Third Row:
Hardware Recontructed Images

Introduction
Contributions
Quality Assessment using PSNR (dB) and IEF for
Lena and House Image
Figure: First Row: PSNR for ”Butterfly” & ”Traffic Signal” Image,
Second Row: IEF for ”Butterfly” & ”Traffic Signal” Image

Introduction
Contributions
Timing Summary
Table: Comparison of Present Implementation of AROF with
Another Implementation
Parameter Present Implementation Benkrid7
Picture Size (Pixels) 1600 × 1200 512 × 512
Frame Rate 118 25
7
K. Benkrid, D. Crookes, and A. Benkrid, ”Design and Implementation of
a Novel Algorithm for General Purpose Median Filtering on FPGA’s”, In IEEE
International Symposium on Circuits and Systems, (ISCAS2002), Vol. 4, pp.
425-428, 2002.

Introduction
Contributions
Publication Details
An Eﬃcient VLSI Architecture for Adaptive Rank Order
Filter for Image Noise Removal, in the International Con-
ference on Signal Acquisition & Processing (ICSAP2011),
Singapore, 26-28 Feb, 2011.
A Novel FPGA Implementation of Adaptive Rank Order
Filter for Image Noise Removal, In an International Jour-
nal of Computer and Electrical Engineering (IJCE), Vol.
4, No. 3, June 2012.

Introduction
Contributions
HSV color space is adapted since it separates color from
intensity.
The contrast of color image is enhanced by providing high
frequency spatial information from the saturation compo-
nent into luminance component.
Local correlation is computed for luminance enhancement
using Geometric Mean Filter (GMF).
Saturation component is enhanced by stretching its dy-
namic range.
The hue component of HSV is preserved in order to avoid
color distortion or shifting.
Sobel operator is used in order to smooth edges.

Introduction
Contributions
Related Work
Jayanta Mukherjee et al.8 proposed enhancement of color
images by scaling DCT coeﬃcients.
Advantages
Chromatic components are processed along with
luminance.
Visual quality is improved by reducing halo artifacts.
Disadvantages
This scheme violate gray world assumption.
Improves the quality of an image at the cost of
computational complexity.
8
Jayanta Mukherjee and Sanjit K. Mitra, ”Enhancement of Color Images by
Scaling the DCT Coeﬃcients,” IEEE Transactions on Image Processing, Vol. 17,
No. 10, pp. 1783-1794, 2008.

Introduction
Contributions
Related Work Cont.,
Gang Song et al.9 proposed adaptive color image
enhancement based on human visual properties in HSV
space.
Advantages
Image enhancement is based on arithmetic mean &
variance computation.
Smooths local variations in an image.
Noise is reduced to some extent.
Disadvantages
Results in blurring eﬀect.
Image details such as sharpness and edges are not
satisfactory.
9
Gang Song and Xiang-Lei Qiao, ”Adaptive Color Image Enhancement based
on Human Visual Properties,” in 3rd IEEE Conference on Industrial Electronics
and Applications (ICIEA 2008), pp. 1892-1895, 2008.

Introduction
Contributions
Related Work Cont.,
Tsai et al.10 proposed fast dynamic range compression
with local contrast preservation algorithm.
Advantages
The contrast enhancement operation in this scheme is
achieved by adaptive intensity transfer function and linear
color remapping techniques.
Processes 30 frames per second with the resolution of
640 × 480 pixels.
Disadvantages
However, there is still considerable room for optimization
of the design based on Verilog coding since the author
uses C++ for the design entry.
10
Chi-Yi Tsai, ”A Fast Dynamic Range Compression with Local Contrast
Preservation Algorithm and its Application to Real-Time Video Enhancement,”
IEEE Transactions on Multimedia, Vol. 14, Issue 4, pp. 1140-1152, 2012.

Introduction
Contributions
Related Work Cont.,
Ming et al.11 has developed high performance
architecture for enhancement of the video stream
captured under non-uniform lightning conditions.
Advantages
The RGB to HSV color space conversion adapted in this
scheme use log-domain in order to avoid complex division
process.
Processes 30 frames per second with the resolution of
640 × 480 pixels.
Disadvantages
Color space conversion and image enhancement operation
increases the latency of the system.
The HSV to RGB converter modules developed to achieve
reconstructed RGB images is not eﬃcient from
computation point of view.
11
M. Z Zhang, M. J Seow, and V. K Asari, ”A High Performance Architecture
for Color Image Enhancement using a Machine Learning Approach,” in Interna-
tional Journal of Computational Intelligence, Vol. 2, Issue 1, pp. 40-47, 2006.

Introduction
Contributions
Related Work Cont.,
Stefano Marsi et al.12 proposed illumination reﬂectance
video enhancement based on FPGA implementation. Al-
though, this scheme reduces halo artifacts, large sized ﬁl-
ter module, multiplier module and divider block exploited
in this work increases the computation complexity.
Faming et al.13 proposed a new pixel based variational
model for remote sensing multi-source image fusion using
gradient features. Visual inspection of the reconstructed
image reveals distortion in the spectral information while
merging the multi-spectral data.
12
Stefano Marsi and G. Ramponi, ”A Fexible FPGA Implementation for
Illuminance-refectance video enhancement,” Journal of Real-Time Image Pro-
cessing, pp. 1-13, 2011.
13
Guixu Zhanga Faming Fang, Fang Li and Chaomin Shen, ”A Variational
Method for Multi-source Remote-Sensing Image Fusion,” in International Journal
of Remote Sensing, Vol. 34, Issue 7, pp. 2470-2486, 2013.

Introduction
Contributions
Algorithm Steps
Step 1 : Read the color image.
Step 2 : Transform the image from RGB to HSV color
space.
Step 3 : Separate composite HSV into individual hue
(H), saturation (S), and value (V) components.
Step 4 : Enhance value and saturation components
adaptively based on geometric mean ﬁlter.
Step 5 : Preserve hue in order to avoid color distortion.
Step 6 : Combine separated components of H, S and V
into composite HSV.
Step 7 : Transform back HSV to RGB color space.
Step 8 : Display the enhanced color image.

Introduction
Contributions
Proposed Enhancement Equations
Local Geometric Mean for Value or Luminance is given by :
Vw = 1
mn [ i,j∈w V (i, j)]
1
mn
where m, n represents window
size, V is the luminance,Vw is the geometric mean for
luminance.
Local Geometric Mean for Saturation is given by :
Sw = 1
mn [ i,j∈w S(i, j)]
1
mn
where m, n represents window
size, S is the saturation,Sw is the geometric mean for
saturation.
Local Variance for Value or Luminance is given by :
σ2
v (x, y) = i,j∈w V (i, j) − Vw (x, y)
2
where x, y is the
center pixel within the window.

Introduction
Contributions
Proposed Enhancement Equations Cont.,
Local Variance for Saturation is given by :
σ2
s (x, y) = i,j∈w S(i, j) − Sw
2
Local Correlation Coeﬃcient for luminance and saturation is
given by :
ρ(x, y) = i,j∈w [V (i,j)−Vw ][S(i,j)−Sw ]√
σ2
v (x,y)σ2
s (x,y) where ρ(x, y) is an
adaptive measure for luminance enhancement.
T1(x, y) = K1 V (x, y) − Vw (x, y)
T2(x, y) = K2 S(x, y) − Sw (x, y) ρ(x, y) where K1 and K2
are constants which is assumed as 2.

Introduction
Contributions
New Luminance Enhancement with Saturation Feedback is
given by :
Venh(x, y) = V (x, y) + T1(x, y) − T2(x, y)
where Venh(x, y) is the enhanced luminance.
The saturation enhancement is given by :
Senh(x, y) = [S(x, y)]γ
where S(x, y) is the saturation component, γ is the stretch
coeﬃcient which determines the degree of saturation
enhancement, Senh(x, y) is the enhanced saturation
component.

Introduction
Contributions
The proposed method has low artifacts or noise.
Better contrast & Improved sharpness.
Our method avoids color distortion since hue is preserved.
Enhanced images are richer in color, have more clarity and
better visual eﬀects.
Provides a general framework for ”Cross-Component Un-
Sharp Masking (USM)”, which is a color enhancement
strategy that may be applied to components from any
color space.

Introduction
Contributions
Block Diagram of the Proposed AGMF
Figure: Block Diagram of the Proposed Adaptive Color Image
Enhancement Method based on Geometric Mean Filter

Introduction
Contributions
Flow Chart for RGB to HSV Color Space Conversion
Figure: Flow Chart for RGB to HSV Color Space Conversion

Introduction
Contributions
Flow Chart for HSV to RGB Color Space Conversion
Figure: Flow Chart for HSV to RGB Color Space Conversion

Introduction
Contributions
Top Architecture of RGB-HSV Color Space
Conversion
(a) Signal diagram for HSV to RGB
Color Space Converter
(b) Signal diagram for RGB to HSV
Color Space Converter

Introduction
Contributions
Architecture for RGB to HSV Color Space Converter
Figure: Detailed Architecture for RGB to HSV Color Space

Introduction
Contributions
Architecture for HSV to RGB Color Space Converter
Figure: Detailed Architecture for HSV to RGB Color Space

Introduction
Contributions
Architecture Details of AGMF
Figure: Detailed Signal Diagram of Adaptive Color Image
Enhancement based on Geometric Mean Filter

Introduction
Contributions
Architecture for Geometric Mean Filter Module

Introduction
Contributions
Architecture for Geometric Mean Filter Module
Cont.,

Introduction
Contributions
Architecture for Histogram Equalization

Introduction
Contributions
Matlab Simulation Results of ”Tractor” Image
Figure: (a) Original Image (b) Histogram Equalized Image (c) Image
Enhanced using Gang et al. Method & (d) Proposed Method

Introduction
Contributions
Matlab Simulation Results of ”Oﬃce” Image

Introduction
Contributions
Matlab Simulation Results of ”Girl” Image

Introduction
Contributions
Performance Comparison

Introduction
Contributions
ModelSim Simulation Results : Validity of RGB
Input

Introduction
Contributions
ModelSim Simulation Results : Validity of the
Enhanced Output

Introduction
Contributions
RTL Top View of AGMF

Introduction
Contributions
RTL Detailed View of RGB to HSV Color Space
Converter

Introduction
Contributions
RTL Detailed View of HSV to RGB Color Space
Converter

Introduction
Contributions
FPGA Resource Utilization for RGB to HSV
Converter

Introduction
Contributions
FPGA Resource Utilization for HSV to RGB
Converter

Introduction
Contributions
FPGA Resource Utilization for Adaptive Geometric
Mean Filter based Color Image Enhancement

Introduction
Contributions
Timing Summary for RGB to HSV Conversion
The timing summary for the RGB to HSV Color Space Converter as
reported by Xilinx ISE tool is as follows:
Speed Grade : -10
Minimum period: 29.730 ns (Maximum Frequency:
33.635 MHz)
Minimum input arrival time before clock: 3.971 ns
Maximum output required time after clock: 8.047 ns
Clock period: 29.730 ns (frequency: 33.635 MHz)
Total number of paths /destination ports:
158142043944387 / 824
Delay: 29.730 ns (Levels of Logic = 35)

Introduction
Contributions
Timing Summary for HSV to RGB Conversion
The timing summary for the HSV to RGB Color Space Converter as
reported by Xilinx ISE tool is as follows:
Speed Grade : -10
127.958 MHz)
Total number of paths /destination ports: 4867 / 727

Introduction
Contributions
Timing Summary for AGMF Design
The timing summary for the AGMF Design as reported by Xilinx ISE
tool is as follows:
Speed Grade : -10
233.323 MHz)
Total number of paths /destination ports: 187392 /
18456

Introduction
Contributions
RGB to HSV and vice versa: Software Approach
Figure: First Row : Original Image, Image in HSV Space, Restored
Image from HSV Space. Second Row :Original Image, Image in HSV
Space, Restored Image from HSV Space.

Introduction
Contributions
RGB to HSV and vice versa: Hardware Approach
Figure: First Row : Original Image, Image in HSV Space, Restored
Image from HSV Space. Second Row :Original Image, Image in HSV
Space, Restored Image from HSV Space.

Introduction
Contributions
Matlab & Hardware Results of ”Nature” Image
Figure: First Row: Original ”Nature” Image, Photoﬂair Software En-
hanced Image (PSNR:28.12 dB), Image Enhanced using Histogram Equal-
ization (PSNR:29.01 dB) Second Row: Matlab Reconstructed Image us-
ing Proposed AGMF (PSNR:30.12 dB), Verilog Reconstructed Image using
Proposed AGMF (PSNR:30.98 dB)

Introduction
Contributions
Matlab & Hardware Results of ”Big Ben” Image
Figure: First Row: Original ”Big Ben” Image, Photoﬂair Software En-
hanced Image ((PSNR:27.68 dB), Image Enhanced using Histogram Equal-
ization (PSNR: 28.3 dB) Second Row: Matlab Reconstructed Image us-
ing Proposed AGMF (PSNR:29.07 dB), Verilog Reconstructed Image using
Proposed AGMF (PSNR:29.16 dB)

Introduction
Contributions
Conclusion
Adaptive color image enhancement in HSV color space
based on Geometric Mean Filter was presented.
Luminance Enhancement is achieved using Saturation feed-
back.
Geometric mean filter offers better reconstructed image
quality compared to arithmetic mean filter.
Experimental results presented shows that the color im-
ages enhanced by the proposed algorithm are clearer,
more vivid and more brilliant.
Performance of the proposed algorithm validated by ap-
plying luminance, contrast and PSNR.
The FPGA implementation of AGMF processes images of
size 1600 × 1200 pixels at 121 frames per second.

Introduction
Contributions
Publication Details
Adaptive Color Image Enhancement based on Geomet-
ric Mean Filter, In Proceedings of International Confer-
ence on Communication, Computing and Security (ICCCS
2011) NIT, Rourkela, India, Feb 12-14, ACM, 2011.
A Novel Reconﬁgurable Architecture for Enhancing Color
Image Based on Adaptive Saturation Feedback, In the In-
ternational Conference on Advanced Information and Mo-
bile Communication (AIM 2011), pp. 162-169, Nagpur,
Maharashtra, India, April 21-22, Springer, 2011.
A Novel FPGA Implementation of Adaptive Color Image
Enhancement based on HSV Color Space, In the Inter-
national Conference on Electronics and Computer Tech-
nology (ICECT 2011), pp. 162-169, Kanyakumari, India,
08-10 April, IEEE, 2011.

Introduction
Contributions
Gaussian Image Enhancement : Introduction
Retinex is a popular image enhancement method for bridg-
ing the gap between images and the human observation
of scenes.
Retinex Algorithm was Proposed by Edwin Herbert Land
in 1986.
Proposed Gaussian based color image enhancement tech-
nique is the modiﬁed version of NASA’s Retinex Algo-
rithm
Retinex is a model of lightness and color perception of
human vision.
Retinex is an adaptive imaging algorithm.

Introduction
Contributions
Gaussian Image enhancement : An Example
Figure: Original Image and Enhanced Image using Proposed Method

Introduction
Contributions
Related Work
D. J Jobson et al.14 proposed a color image enhancement
based on multiscale retinex for bridging the gap between
color images and Human observation of scenes.
Advantages
Good dynamic range compression and color constancy.
Reconstructed images are favorable for Human visual per-
ception and improve contrast.
Disadvantages
This method fails to produce good color rendition for a
class of images that contain violations of the gray world
assumption.
Unable to remove Halo artifacts completely.
Many parameters were assumed such as alpha, beta, gain,
Gaussian scales etc.
14
D. J Jobson, Z. Rahman and G. A Woodell, ”A Multiscale Retinex for
Bridging the Gap Between Color Images and the Human Observation of Scenes”,
in IEEE Transactions on Image Processing, Vol. 6, pp. 965-976, 1997.

Introduction
Contributions
Related Work Cont.,
Digital Signal Processors (DSPs)15 has been used for the
implementation of image enhancement algorithms.
Advantages
Improved eﬃciency compared to general purpose comput-
ers.
Disadvantages
Only marginal improvement has been achieved since paral-
lelism and pipelining incorporated in the design are inade-
quate.
This scheme uses optimized DSP libraries for complex op-
erations and does not take full advantage of inherent par-
allelism of image enhancement algorithm.
The enhancement of 25-30 frames per second of large size
video frames with 1024 × 768 pixel resolution is still not
possible with DSPs.
15
D. J Jobson, G. D Hines, Z. Rahman and G. A Woodell, ”DSP Implemen-
tation of the Retinex Image Enhancement Algorithm”, In Visual Information
Processing XIII, Proc. SPIE, Vol. 5438, pp. 13-24, 2004.

Introduction
Contributions
Related Work Cont.,
The neural network based learning algorithm16 provides an
excellent solution for the color image enhancement with
color restoration.
Advantages
The hardware implementation of the algorithm parallelizes
the computation and delivers real time throughput for color
image enhancement.
Disadvantages
The window related operations such as convolution, sum-
mation and matrix dot products in an image enhancement
architecture demands an tremendous amount of hardware
resources.
16
M. Z Zhang, M. J Seow, and V. K Asari, ”A High Performance Architecture
for Color Image Enhancement using a Machine Learning Approach”, In Interna-
tional Journal of Computational Intelligence Research-Special Issue on Advances
in Neural Networks, Vol 2, Issue 1, pp. 40-47, 2006.

Introduction
Contributions
Related Work Cont.,
Hiroshi Tsutsui et al.17 proposed an FPGA implementa-
tion of adaptive real-time video image enhancement based
on variational model of the Retinex theory.
Authors claimed that the architectures developed in this
scheme are eﬃcient and can handle color picture of size
1900 × 1200 pixels at the real time video rate of 60 fps.
However, the computational cost of the algorithm de-
pends on the number of processing layers while the maxi-
mum layers and iterations used are 5 and 30, respectively.
Also, authors have not justiﬁed how high throughput has
been achieved in spite of time consuming iterations to the
tune of 30.
17
Hiroshi Tsutsui, H. Nakamura, R. Hashimoto, H. Okuhata, and T. Onoye,
”An FPGA Implementation of Real-time Retinex Video Image Enhancement”, In
IEEE World Automation Congress (WAC), pp. 1-6. 2010.

Introduction
Contributions
Related Work Cont.,
Abdullah M. Alsuwailem et al.18 proposed a new approach
for HE using FPGAs.
Although eﬃcient architectures were developed for HE,
the reconstructed images using this scheme are generally
not acceptable.
The HE process loses the details such as edges, contrast
and leads to over enhancement of noise in images.
The enhancement approach using adaptive HE scheme
also fails to produce satisfactory results since the process
enlarges the contrast of background noise while lessening
the exploitable signal.
18
Abdullah M. Alsuwailem and S. A Alshebeili, ”A New Approach for Real-
time Histogram Equalization using FPGA”, In IEEE Proceedings of International
Symposium on Intelligent Signal Processing and Communication Systems (IS-
PACS2005), pp. 397-400, 2005.

Introduction
Contributions
Proposed Gaussian Based Image Enhancement
Method
Original image (which is of poor quality and needing en-
hancement) is read in RGB color space.
The color components are separated followed by the con-
volution with 3×3 Gaussian kernel in order to smooth the
image.
Logarithmic operation is accomplished in order to com-
press the dynamic range of the image and to improve low
intensity pixel values.
Gain/Oﬀset adjustment is done in order to translate the
pixels into the display range of 0 to 255.
The No. of scales, scales, gain and oﬀset do not vary from
one image to another. This implies that the algorithm is
canonical.

Introduction
Contributions
Depending on circumstances, the proposed method
could achieve :
Sharpening : Compensates for the blurring introduced by
image formation process.
Color constancy processing : Improves consistency of
output as illumination changes.
Good dynamic range compression and color rendition
eﬀect.
Canonical constant : independent of inputs.
General enhancement algorithm for all types of pictures.
Provides satisfactory results for bi-modal pictures.

Introduction
Contributions
Flow Diagram of Proposed Enhancement Method

Introduction
Contributions
Gaussian Kernels: 3 × 3 kernel and 5 × 5 kernel
1
16
1 2 1
2 4 2
1 2 1
1
273
1 4 7 4 1
4 16 26 16 4
7 26 41 26 7
4 16 26 16 4
1 4 7 4 1
Figure: Gaussian Kernels: 3 × 3 kernel and 5 × 5 kernel

Introduction
Contributions
Proposed Enhancement Equations
The two dimension (2D) Gaussian function is deﬁned by
g(x, y) = 1
2πσ2 e− x2+y2
2σ2
The Gaussian convolution matrix is given by
G(x, y) = I(x, y) ⊗ g(x, y)
Mathematically, 2D convolution can be represented as
G(x, y) = M
i=1
N
j=1 I(i, j) × g(x − i, y − j)
The convolution operation for a mask of 5 × 5 is given by
P(x, y) =
4
i=0 Wi ×Pi
4
i=0 Wi

Introduction
Contributions
The logarithmic processing on a 2D image is carried out by using
the following Eqn. GL(x, y) = K × log2 [1 + G(x, y)]
This gain/oﬀset correction is accomplished by using Eqn. given
below:
I (x, y) = dmax
GLmax −GLmin
[GL(x, y) − GLmin]
where dmax is the maximum intensity, which is chosen as, 255 for
an image with 8-bit representation, GL(x, y) is the log-transformed
image, GLmin is the minimum value of log transformed image,
GLmax is the maximum value of log transformed image, I (x, y) is
the enhanced image and, x and y represent spatial coordinates.

Introduction
Contributions
Block Diagram & Signal Description for the Top
Module of Gaussian Based Image Enhancement
System
Block Diagram Signal Description
Signals Description
clk This is the global clock signal
reset n Active low system reset
rin [7:0] Red color component
gin [7:0] Green color component
bin [7:0] Blue color component
ro [7:0] Enhanced red color component
go [7:0] Enhanced Green color component
bo [7:0] Enhanced Blue color component
pixel valid Valid signal for enhanced RGB pixel

Introduction
Contributions
Detailed Architecture of Gaussian Based Color
Image Enhancement System

Introduction
Contributions
Top Architecture of Serpentine Memory
Signal Diagram Signal Description
Signals Description
clk Global clock signal
reset n Active low system reset
pixel in [7:0] R/G/B Input pixel
window valid Valid signal for sliding window
w11 [7:0] to w15 [7:0] First row pixel values
w21 [7:0] to w25 [7:0] Second row pixel values
w31 [7:0] to w35 [7:0] Third row pixel values
w41 [7:0] to w45 [7:0] Fourth row pixel values
w51 [7:0] to w55 [7:0] Fifth row pixel values

Introduction
Contributions
Detailed Architecture of Serpentine Memory System

Introduction
Contributions
Top Architecture of 2D Gaussian Convolution
Signal Diagram Signal Description
Signals Description
window valid Valid signal from sliding window
W11 to W15 First row pixel values
W21 to W25 Second row pixel values
W51 to W55 Fifth row pixel values
G11 to G15 First row Gaussian kernel values
G21 to G25 Second row Gaussian kernel values
G51 to G55 Fifth row Gaussian kernel values
conv out [7:0] Gaussian convolved output pixels
conv valid Valid signal

Introduction
Contributions
Detailed Architecture of 2D Gaussian Convolution
Processor for R/G/B Color Channels

Introduction
Contributions
Architectures of Adder, Multiplier and Logarithm
Adder & Multiplier Module
Logarithm Module

Introduction
Contributions
Detailed Architecture of 24-bit Unsigned Adder

Introduction
Contributions
Detailed Architecture of Pipelined Multiplier Design

Introduction
Contributions
Results and Discussions
Simulation Tool : Matlab Ver. 7.6.0.324 (R2008a).
Resolution of Test Images : 640 × 480 Pixels (VGA),
800 × 600 Pixels (SVGA), 1024 × 768 Pixels (XGA).
Test Images :
http://dragon.larc.nasa.gov/retinex/pao/news/
http://visl.technion.ac.il/1999/99-07/www/
http://ivrg.epfl.ch/index.html
Performance Metrics : Contrast Enhancement,
Luminance Enhancement, Peak Signal to Noise Ratio
(PSNR).
Histogram : Plotted to show Pixel Distribution.

Introduction
Contributions
Comparison of Matlab Reconstructed Pictures Using
Image Enhancement Algorithms: ”Trees” Image
Figure: First Row:Original Image, Histogram Equalization, NASA’s
MSRCR. Second Row:Chih et al. MSRCR, Proposed Method

Introduction
Contributions
Image Enhancement Algorithms: ”Palette” Image

Introduction
Contributions
Image Enhancement Algorithms: ”House” Image

Introduction
Contributions
ModelSim Simulation Waveforms for Inputting
Image Data

Introduction
Contributions
Waveforms of Sliding Window Module for one of the
R/G/B Color Components

Introduction
Contributions
Waveforms for Gaussian Convolution Output at
21090 ns

Introduction
Contributions
Starting of Enhanced Pixel Data

Introduction
Contributions
Waveforms for Ending of Reconstructed Pixels at
13,31,990 ns

Introduction
Contributions
Processing Time Report for the Top Design Module
of Gaussian Based Image Enhancement System
Module Clock Cycles Required to Process each pixel data
Sliding Window 1029
Gaussian convolution 23
Logarithm 1
gain/oﬀset correction 7

Introduction
Contributions
Timing Diagram for Illustrating the Pipelining
Operation of the Proposed System

Introduction
Contributions
RTL View of the ”Gaussian IE”
RTL View of the Top Module
aZoomed RTL View
a
Note: U1: Red Color,U2: Green
Color, U3: Blue Color Component Pro-
cessors

Introduction
Contributions
Zoomed View of U1 or U2 or U3 Module

Introduction
Contributions
FPGA Resource Utilization for Gaussian Based Color
Image Enhancement Design

Introduction
Contributions
Matlab and Verilog Reconstructed ”Tree”, ”House”
and ”Color Palette” Images
Figure: First Column: Original Images, Second Column: Matlab
Reconstructed Images, Third Column: Hardware Reconstructed Images

Introduction
Contributions
Matlab and Verilog Reconstructed ”Couple”, ”Dark
Road” and ”Memorial Church” Images
Figure: First Column: Original Images, Second Column: Matlab
Reconstructed Images, Third Column: Hardware Reconstructed Images

Introduction
Contributions
Performance Evaluation : Approximate Wavelet
Energy Metric

Introduction
Contributions
Performance Evaluation : Detailed Wavelet Energy
Metric

Introduction
Contributions
Conclusion
Gaussian based color image enhancement algorithm was
designed and architectures were developed.
The proposed method is eﬃcient from computation point
of view as compared to other researcher methods.
The visual quality of reconstructed pictures and image en-
hancement achieved for various test images are compared
using wavelet energy metric.
Color image enhancement system is implemented on Xil-
inx Virtex-II Pro XC2VP40-7FF1148 FPGA device and is
capable of processing high resolution videos up to 1600 ×
1200 pixels at 117 frames per second.
RTL compliant Verilog coding of our system ﬁts into a
single FPGA chip with a gate count utilization of about
321,804.

Introduction
Contributions
Publication Details
Design of Novel Algorithm and Architecture for Gaussian
based Color Image Enhancement, in International Con-
ference on Advances in Computing, Communication and
Control, Mumbai, India, 18-19 Jan, 2013.
Design and FPGA Implementation of a 2D Gaussian Sur-
round Function with Reduced On-Chip Memory Utiliza-
tion, in International Conference on Advances in Com-
puting, Communication and Informatics (ICACCI2013),
Mysore, India, 22-25 Aug, 2013.
A Novel Full-Reference Color Image Quality Assessment
Based on Energy Computation in the Wavelet Domain”,
In Journal of Intelligent Systems, Vol. 22, No. 2, pp.
155-177, May 2013.

Introduction
Contributions
Conclusions
The AROF have been used in this work to remove impulse
noise since AROF has better ﬁltering properties compared
to AMF.
The core modules of the AROF system, namely, sliding
window, impulse noise detection, sorting network, median
computation and output selection were realized using Ver-
ilog for ASIC/FPGA implementation.
A Novel algorithm for color image enhancement based
on AGMF is proposed with the architecture development
suitable for FPGA/ASIC implementations.
The Verilog codes for the functional modules of AGMF,
namely, RGB to HSV, histogram equalization, value com-
ponent enhancement, HSV to RGB converter etc. have
been developed and successfully simulated.

Introduction
Contributions
Conclusions Cont.,
Design of new algorithm and architectures for Gaussian
based color image enhancement system for real-time ap-
plications has been presented.
The Gaussian color image enhancement functional mod-
ules, namely, serpentine memory, 2D Gaussian convolu-
tion, logarithm base-2 and gain/oﬀset correction were re-
alized using Verilog conforming to RTL coding guidelines
practised in Industries.
The designs presented exploits high degrees of pipelin-
ing and parallel processing in order to achieve real time
performance.
Quality assessment of image enhancement algorithms are
based metrics: CEP, LEP, PSNR, and WE etc.

Introduction
Contributions
Future Scope
The work presented throws open a number of work that
may be undertaken by researchers in the near future.
The design of the proposed AROF is modular and ﬂexible,
and therefore, it can be upgraded to accommodate new
modules, both present and future, without appreciable
increase in hardware.
The functional modules of AROF, AGMF & Gaussian im-
age enhancement residing in FPGAs presently can be re-
placed by ASIC resulting in more compact, low power,
high speed and cost eﬀective system suitable for volume
production.
A medical image enhancement technique based on retinex
can also be designed and implemented on FPGA/ASIC by
modifying the algorithms and architectures.

Introduction
Contributions
VLSI Image Processing Groups in Foreign
Universities
Dr. Vijayan K. Asari, Old Dominion University, Norfolk, USA
http://www.ece.odu.edu/~vasari/
Dr. Ryan Kastner, University of California, Sandiego http:
//cseweb.ucsd.edu/~kastner/main
Dr. Junguk Cho, University of California, Sandiego http://
cseweb.ucsd.edu/~j10cho/index.html
Dr. Venkatesan Muthukumar, University of Nevada Las Vegas,
USA http://www.ee.unlv.edu/~venkim/index.html
Dr. Ming Z. Zhang, Old Dominion University, Norfolk, USA
http://caprolibra.com/Prfdex.html
Dr. Sudha Natarajan, NTU, Singapore http://www.ntu.edu.
sg/home/sudha/

Introduction
Contributions
VLSI Image Processing Groups in Indian Universities
Dr. Swapna Banerjee, Dept. of EE, CAD and VLSI Laboratory,
IIT Kharagpur, India.
Dr. Nitin Chandrachoodan, Dept. of EE, VLSI Laboratory, IIT
Madras, India.
Dr. S. Srinivasan, VLSI Laboratory, Dept. of EE, IIT Madras,
India.
Dr. V. Kamakoti, Reconﬁgurable and Intelligent Systems Engi-
neering Group (RISE Laboratory), Dept. of CSE, IIT Madras,
India.
Sanjay Sing, Scientist Fellow, IC Design Group, CEERI, Pilani,
India.
Dr. S. S. S. P Rao, Dept. of CSE, IIT Bombay, India.

Introduction
Contributions
VLSI Image Processing Journals
Elsevier Journal on Microprocessors and Micro-systems.
Springer Journal of VLSI Signal Processing Systems for Signal,
Image and Video Technology.
IEEE Transactions on Very Large Scale Integration (VLSI) Sys-
tems.
IEEE Transactions on Circuits and Systems for Video Technol-
ogy.
IEEE Journal on Computer Architectures for Intelligent Ma-
chines.
Journal of Circuits, Systems and Computers.

Introduction
Contributions
VLSI Image Processing Industries

Introduction
Contributions
Image Processing Books

Introduction
Contributions
VLSI Signal Processing Books

Introduction
Contributions
Refrences
Mohd Firdaus Zakaria, Haidi Ibrahim and Shahrel Azmin
Suandi, ”A Review: Image Compensation Techniques”, Pro-
ceedings of Second International Conference on Computer En-
gineering and Technology (ICCET-2010), 16-18 April, 2010.
C. Iakovidou, V. Vonikakis and I. Andreadis, ”FPGA implemen-
tation of a real-time biologically inspired image enhancement
algorithm”, Journal of Real Time Image Processing, Vol. 3, No.
4, pp. 269-287, 2008.

Introduction
Contributions
Refrences
Ming Z. Zhanga,Ming-Jung Seowa, Li Tao and Vijayan K.
Asari,”A tunable high-performance architecture for enhance-
ment of stream video captured under non-uniform lighting con-
ditions”, Journal of Micrprocessors and Microsystems, Vol. 32,
Issue 7, pp. 386-393, 2008.
Hiroshi Tsutsui, Hideyuki Nakamura, Ryoji Hashimoto, Hi-
royuki Okuhata and Takao Onoye, ”An FPGA Implementation
of Real-Time Retinex Video Image Enhancement”, Proceedings
of World Automation Congress (WAC), pp. 1-6, 19-23 Sept,
2010.

Introduction
Contributions
Refrences
D. J Jobson, Z. Rahman, and G. A Woodell, A Multiscale retinex
for bridging the gap between color images and the human ob-
servation of scenes, IEEE Transaction Image Processing, Vol. 6,
No. 7, pp. 965-976, July 1997.
Xinghao Ding, Xinxin Wang, Quan Xiao, ”Color Image En-
hancement with a Human Visual System based Adaptive Filter”,
Proceedings of International Conference on Image Analysis and
Signal Processing, April, 2010.
Hongqing Hu and Guoqiang Ni, ”The improved algorithm for
the defect of the Retinex Image Enhancement”, Proceedings
of International Conference on Anti-Counterfeiting Security and
Identiﬁcation in Communication (ASID), pp. 257-260, July,
2010.

Ph.D Dissertation Defense Slides on Efficient VLSI Architectures for Image Enhancement Techniques

Ph.D Dissertation Defense Slides on Efficient VLSI Architectures for Image Enhancement Techniques

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Ph.D Dissertation Defense Slides on Efficient VLSI Architectures for Image Enhancement Techniques

Similar to Ph.D Dissertation Defense Slides on Efficient VLSI Architectures for Image Enhancement Techniques (20)

More from BMS Institute of Technology and Management

More from BMS Institute of Technology and Management (8)

Recently uploaded

Recently uploaded (20)

Ph.D Dissertation Defense Slides on Efficient VLSI Architectures for Image Enhancement Techniques