This presentation outlines the ways in which still images and video can be categorised to boost findability.

1. Indexing Still and Moving Images
Ian Davis
Global Taxonomy Delivery Manager
Dow Jones Consulting Services
© Copyright 2008 Dow Jones and Company | 1

2. Indexing Still and Moving Images
Still image indexing
Moving image indexing
Two content types can often be treated very
similarly
Two content types also bring unique challenges
© Copyright 2008 Dow Jones and Company | 2

3. Who uses still images?
 Newspaper and magazine publishers
 Book publishers
 Internet or Intranet sites
 Music labels
 Advertising agencies
 Design houses
 Academics
 Corporates
 TV companies
© Copyright 2008 Dow Jones and Company | 3

4. What do People Want?
Access to images based on attributes:
e.g. images photographed at a specific time.
e.g. images photographed at a specific place.
e.g. images created using a certain photographic
process.
© Copyright 2008 Dow Jones and Company | 4

5. What do People Want?
Access to images based on specific depictions:
e.g. 'sun loungers on a tropical beach‘
e.g. 'peacock, full tail display and head’
e.g. 'individuals or couples in modern, clean, bright
kitchens or living rooms - ideally to show blur of
motion.'
© Copyright 2008 Dow Jones and Company | 5

6. What do People Want?
Access to images based on conceptual terms:
e.g. 'eerie images‘
e.g. 'exciting, speedy images’
e.g. 'images illustrating individuality'
© Copyright 2008 Dow Jones and Company | 6

7. What do People Want?
Access to images based on basic inherent content:
 Colours
 Textures
 Shapes
© Copyright 2008 Dow Jones and Company | 7

8. Fitting the system to the users
 Image indexing should focus on the images and
also on the people who will be looking for them.
How images are be indexed should relate closely
to this.
© Copyright 2008 Dow Jones and Company | 8

9. Fitting the system to the users
Editorial users often appreciate:
 Full captions of two to three sentences
 Detailed factual information
Ad and design users often:
 Care little for concrete facts
 Like to browse for ideas
 Like to search using concepts
© Copyright 2008 Dow Jones and Company | 9

10. Indexing Strategy
Choose from:
 Manual approach
 Automatic approach
 Combined approach
© Copyright 2008 Dow Jones and Company | 10

11. Fitting the system to the users
 Understand client needs
 Construct an approach to meet these needs
 Choose from the available techniques and
technologies
 Consider textual or CBIR, or even textual and CBIR
© Copyright 2008 Dow Jones and Company | 11

12. Methods of Image Classification and Retrieval
Semantic content – textual classification and
retrieval
 Human analysis of images
 Metadata development
 Thesauri or ontology supported classification and
retrieval
Content based image retrieval (CBIR)
 Automatic analysis of low level image attributes
© Copyright 2008 Dow Jones and Company | 12

13. Manual Image Indexing
Meta-date Coverage – layers of information
 Technical image information
 Basic image attributes
 Image content information
 Image context/aboutness information
© Copyright 2008 Dow Jones and Company | 13

14. Manual Image Indexing
Technical image information:
 Unique scanning numbers
 Scanner type
 Processor name
 Processing activities
 Batch numbers
 Sizes of image files
© Copyright 2008 Dow Jones and Company | 14

15. Manual Image Indexing
Basic Image Attributes:
 Photographer name
 Photography Types
 Location photographed
 Date photographed
 Work type
 Points of view
© Copyright 2008 Dow Jones and Company | 15

16. Manual Image Indexing
Image Content Information:
 Actions and Events
 Animals and Plants
 People: roles, occupations, ethnicity, attributes,
and anatomy
 Topography
 Built works
 Things
© Copyright 2008 Dow Jones and Company | 16

17. Manual Image Indexing
Image context/aboutness information:
 Emotions
 Abstract concepts
© Copyright 2008 Dow Jones and Company | 17

18. Automatic Image Indexing
Inherent in image pixels:
 Colours
 Textures
 Simple shapes
© Copyright 2008 Dow Jones and Company | 18

19. Video Indexing
Layers of information
 Indexing the whole piece
 Indexing the scene
 Indexing the frame
Types of information
 Image information
 Textual information
 Audio information
© Copyright 2008 Dow Jones and Company | 19

20. Video Indexing
 Dealing with video includes all the challenges of
still image indexing
 Plus the technical issues of capturing textual
data about the content
 And the issue of automatically identifying scenes
and frames
 And the issue of capturing, indexing and linking
audio soundtracks to these scenes and frames.
© Copyright 2008 Dow Jones and Company | 20

21. Video Indexing
 Much initial textual and audio data can be
captured and indexed automatically
 System can be trained to identify scene breaks
and key frames can be selected
 Focus is often on improving and monitoring the
automated aspects and prioritising what work
needs to be done manually – usually focusing on
key frames/scenes in a piece
© Copyright 2008 Dow Jones and Company | 21

22. Thank You!
ian.davis@dowjones.com
© Copyright 2008 Dow Jones and Company | 22