1. Introduction to Camera
By: Sergio Aguirre <sergio.a.aguirre@gmail.com>

2. Basics

Camera: Device that records and stores
images.

Still Images:

Videos or movies:

3. Basics

But essentially, what defines a camera device?

Recording medium

Lens

Light
Lens
Light
Recording Medium

4. Basics

Now, what about a typical digital camera?

Image sensor (CCD or CMOS)

Lens

Flash

ISP: Image Signal Processor

Image and video Encoders (To save space)

Storage Media (SD Cards, USB drives, Etc.)

5. Image sensors

6. Image sensor technology

CCD: Charge Coupled
device.


Moves electrical
charges to an area
where it can be
manipulated.
CMOS: Complementary
metal-oxide
semiconductor.

Array of pixel sensors,
each one containing a
photodetector and an
active amplifier.

7. Image sensor pixel arrays

Foveon:


100% Green


100% Red
100% Blue
Bayer:

25% Red

50% Green

25% Blue

8. Image sensor formats
(Most typical)

RAW 10-bits.


10-bit digital value obtained thru an ADC (Analog to
Digital Converter) of the analog voltage coming
from each pixel's associated photodetector chargeto-voltage converter.
YUV 4:2:2 8-bits.

Represent pairs of horizontally contiguous pixels. 8bits per component (Y: Luminance, UV:
Chrominance). Order is usually YUYV, or UYVY.
Croma components (UV) are subsampled at half
the sample rate than luma (Y).

9. Image sensor formats
(Most typical)

YUV 4:2:0 8-bits.


Similar to 4:2:2, but the difference being that the UV
components are also vertically subsampled by 2.
This format contains interleaved data, which
means, odd lines contain only Y elements, and
even lines contain packed UYVY elements, similarly
to 4:2:2 subsampling.
NV12 (YUV 4:2:0 8-bits, planar)

Exact same subsampling as above format, but the
difference being that one array contains only Y
components, and a second array contains packed
UV components, 2x2 subsampled.

10. Image sensor core
Typical block diagram of an image sensor core
components:
AMP
Gain
control
10-bit
ADC
Digital gain
Image
array
Black level
calibration
Column
Sample/hold
Row select

...

11. Image sensor core functions





Analog gain: Amplifies constant voltage perpixel for entire array. Done before ADC.
Black level: Level of pixel voltage at the darkest
(black) part of the incoming image.
Digital gain: Adds a constant number to the
digitally converted pixel value, after going
through the ADC.
Skipping: Certain pixels rows/columns are
avoided (skipped) on sampling.
Binning: Averaging groups of neighborhood
pixel charges.

12. Image sensor interfaces

Overall, there's at least 2 communication links:


Control: Typically I2C, but not necessarly the case.
This is used by a master IC to initialize/program the
sensor for desired operation and data output
configuration.
Data: This can be Parallel or Serial.
Sensor
Data
Master IC
Control

13. Image sensor interfaces - Control

I2C: Inter-Integrated Circuit


Multi-master serial single-ended computer bus,
used to attach low-speed peripherals to a host IC.
Uses 2 open-drain lines, w/pull-up resistors:


SCL (Serial CLock).
SDA (Serial DAta).

14. Image sensor interfaces - I2C

I2C: Inter-Integrated Circuit



START and STOP conditions to claim/release bus control.
First byte: 7-bit device address space, 1 bit used to indicate Read
(HI) or Write (LOW).
ACK bit pulled low by slave after each byte sent, used to
handshake transmission between master and slave ICs. If ACK bit
is high, master should stop transmission.

15. Image sensor interfaces - Parallel
Parallel interface (a.k.a. Digital Video Port)

Data signals: 1 digital output pin per bit.

Sync signals:



PCLK: Pixel Clock.
HSYNC: Horizontal Sync.
VSYNC: Vertical Sync.
Image data
Vertical Blanking
Horizontal Blanking

D[9:0]
Sensor
PCLK
HSYNC
VSYNC
Master IC

16. Image sensor interfaces - CSI2

MIPI CSI2 (Camera Serial Interface 2).

Data signals: Up to 4 differential lanes (+/- signals).

Clock signals: 1 differential lane (+/- signals).

Speeds up to 1 Gbit/s per lane (500 MHz)
CLK (+/-)
DAT0 (+/-)
Sensor
DAT1 (+/-)
DAT2 (+/-)
DAT3 (+/-)
Master IC

17. Image sensor interfaces - CSI2

CSI2 Receiver components:

18. Image sensor interfaces - CSI2

D-PHY layer:

19. Image sensor interfaces - CSI2

CSI2 Protocol Layer:

20. Image sensor interfaces - CSI2

Data Lane Merger

21. Lens

22. Lens


Could be either a simple convex lens, or a
compound lens made up of a number of lens
elements, used to correct optical aberrations,
and keep good image sharpness on a specific
object distance from the camera.
There are fixed-focus lens, and lens whose
position can be adjusted by a VCM (voice coil
motor).

23. Lens


For moving lens, position can be either
controlled as part of the image sensor registers,
or as a separate I2C device in the bus.
This is usually done as a result of the Auto
Focus algorithm, after analyzing the captured
images. It can also be due to manual control, if
desired.

24. Flash

25. Flash



Used in photography to compensate natural
ambient light, and illuminate a dark scene.
Syncronized with start of capture image
command, with adjusted flashing duration, and
intensity.
Usually triggered with GPIOs and configured
with I2C commands.

26. Thanks!

27. Extra slides

28. Image sensor interfaces - Data

MIPI CSI (Camera Serial Interface).

Data signals: 1 differential lane (+/- signals).

Clock signals: 1 differential lane (+/- signals).

Speeds between 1->208 Mbps
CLK (+/-)
Sensor
DAT (+/-)
Master IC

29. Image sensor interfaces - Data

SMIA CCP2 (Compact Camera Port 2).

Data signals: 1 differential lane (+/- signals).

Clock signals: 1 differential lane (+/- signals).
Class
Data transfer capacity
(sustain data rate)
Class 0
<208 Mbit/s
Data/Clock
Class 1
208 Mbit/s to 416 Mbit/s
Data/Strobe
Class 2
416 Mbit/s to 650 Mbit/s
Data/Strobe
CLK, or
STROBE
(+/-)
Signaling method
Sensor
DAT (+/-)
Master IC