Any good IP core will conform to the specification it claims to support. But for video and image IP, matching a standard is not the same as producing the degree of image quality required for demanding applications. High quality output requires clever, high-quality development in all the crucial steps of a core's operation. In this talk from SoCIP China 2011, we look at key quality factors using JPEG 2000 compression and video deinterlacing as examples. Visit http://www.cast-inc.com/ip-cores/images/jpeg2k-e/ or http://www.cast-inc.com/ip-cores/video/vdint-ma/ to learn more.

1. Pretty as a Picture:Assessing Quality inImage/Video IP Nikos ZervasVice President, CAST, Inc.

2. SoCIP 2011 2 Video and Image Quality Quality can be crucial, especially in applications with high-resolution displays or analytics. Customers automatically judge by the quality, they see, regardless of the end application.

3. Compression and Quality Myths Supporting an industry support defines a core’s quality. A particular set of features automatically defines quality. I can judge quality with vendor-provided tests. Realities Standards typically describe a decoder, but quality is mainly determined by the encoder. Within conformance to a standard, developers have great freedom in choosing algorithms and implementation details. Image or video tests can be tuned to do well with particular encoders/decoders, and may not reflect your media content. SoCIP 2011 3

4. Compression and Quality Standards do not mandate quality. There are potential pitfalls with every compression technology. MPEG2 JPEG JPEG 2000H.264 LJPEG JPEG-LS … Let’s look at one: JPEG 2000. SoCIP 2011 4

5. JPEG 2000 — Pitfalls JPEG 2000 offers the best lossless and great lossy compression efficiency. True only when all encoding switches are turned off. JPEG 2000 offers the best lossless and great lossy compression. True for Variable Bit Rate (VBR). But under Constant Bit Rate (CBR), quality depends on Rate Control efficiency. JPEG 2000 offers Region of Interest (ROI) for selective quality. True, but the implementation of ROI may make ROI unusable. JPEG 2000 minimizes blocking artifacts. True, but tiling artifacts can be present when you get more than one tile per image. SoCIP 2011 5

6. JPEG 2000 — Encoding Switches Parallel Mode and Bypass Simplify hardware design. Destroy JPEG 2000 coding Efficiency. 6 SoCIP 2011 -3dB -1dB

7. JPEG 2000 — Region of Interest slide 7 SoCIP 2011 Requires higher quality for the region of interest (ROI). But you still care about the quality of the background. Example: Aerial photography with ROI compressed at 50:1 Some encoders only do this: But others can do this:

8. JPEG 2000 — Rate Control Not defined by the standard, but needed for many applications. Determines the quality you will get at a specific rate (compression ratio). Its accuracy defines the buffering/storage requirements and latency on your system. SoCIP 2011 slide 8 Your SIP provider should be able to provide comparative data with respect to the de facto reference software (kakadu)

9. Image Processing and Quality There are no standards for most image processing algorithms. There are potential pitfalls with every image processing algorithm. Image Scaling White BalanceBad Pixel CorrectionDe-interlacing … Let’s look at one: De-interlacing. SoCIP 2011 9

10. De-interlacing – Pitfalls Quality is preserved by minimizing motion artifacts True, but detail needs to be preserved too. Motion detection is the best de-interlacing method Not really. Motion detection algorithms can be easily fooled by small motion, image noise etc SoCIP 2011 10 DSP techniques provide great quaiity There is no continuous time signal in video, so DSP techniques just fail Treats all pixels equally

11. De-interlacing- Quality Detail without Motion Artifacts is preserved when filtering adopts not only to motion (temporal) but also to spatial variations e.g. edges should be treated differently than flat areas Content Adaptive algorithms deliver the highest quality results SoCIP 2011 11 DSP Content Adaptive Motion Detection

12. Considerations for SIP Selection How easy is it to integrate my image/video Semiconductor IP? Streaming versus SoC bus interfaces. External memory bandwidth and tolerance to memory latencies. Run-time programmability. System-level buffering requirements. How much external processing does it require? For example: some H.264 SIPs work as accelerators rather than complete stand-alone solutions. SoCIP 2011 12

13. Example: Camera SoC 13 SoCIP 2011 Good Pixel data streamed from sensor to output; Host controls the SoC. Easy SoC bus and memory arbitration, low software complexity. Bad Pixel data transferred over SoC bus; Host controls the SoC. Challenging SoC bus and memory arbitration, moderate complexity. Ugly Pixel data transferred over SoC bus; Host performs some processing and controls the SoC. SoC bus and memory arbitration becomes difficult, High complexity. 3 – 4 weeks to integrate, Low risk 2 – ??? Months to integrate, High risk

14. Takeaways Evaluate your video and image IP for quality before making a purchase decision. Ask for software modelor reference design, or ask your vendors to encode your own clips using your settings. Compare results using your perception, reference implementations, and stream analysis tools. Consider integration complexity. Cores that are difficult to integrate can end up costing you several times more than the core license itself. Failures in system-level integration can delay your product development, cripple your product and make it non-competitive. SoCIP 2011 14