For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2021/07/object-detection-and-dataset-labeling-using-colors-of-manufactured-objects-a-presentation-from-basf/ Ian Childers, Head of Technology for Functional Coatings—Object Recognition at BASF, presents the “Object Detection and Dataset Labeling Using Colors of Manufactured Objects” tutorial at the May 2021 Embedded Vision Summit. This talk introduces a new method for object detection for consumer goods and other applications based on measuring an object’s illumination invariant fluorescent chroma. Chroma is a measure of the colorfulness of an object relative to a similarly illuminated white object. Reflective chroma changes with the illumination, making it a poor choice for object detection in different environments. Fluorescent chroma, however, is nearly illumination invariant, making it an ideal input for object detection algorithms if the fluoresced light can be separated from the reflected light. Childers describes simple designs of illuminators, cameras, and filters to achieve that separation, and shows the classification accuracy of the system to be 95-100% under different lighting conditions using the nearest neighbors algorithm with neighborhood component analysis. BASF has designed hundreds of unique colors and coatings for consumer goods and other items that can be used in this application and the system has many advantages compared to CNN object detection systems, as it does not require extensive training sets or an unoccluded view of the object.

1. © 2021 BASF
Object Detection and Dataset
Labeling Using Colors of
Manufactured Objects
Ian Childers, PhD
Head of Technology
BASF Surface Solutions,
Functional Coatings - Object Recognition

2. © 2021 BASF
Coatings – We are the Leader In Color Design
2
Global color collection Annual trend book
Global design support Innovative color concepts
Key success factors:
◼ Trend research
Excellent trend research and visualization
◼ Global color collection
Translation of trend research into annual global
color collection
◼ Innovative color concepts
New inspiring color concepts with future-
oriented approach
◼ Global design support
Close to the customer all around the world

3. © 2021 BASF
Object Detection with CNNs
• Convolutional neural networks learn
features and how the presence and
arrangement of the features correlate to
classes
• Large numbers of items, almost
infinite arrangements and views,
different illumination
• Lots of great work at solving these
issues
Remaining Problems
1. High quality, labelled training image sets are time consuming and expensive
2. AI predictions are less useful when the use case requires more certainty
Figure from: Chen, Y.; Chen, R.; Liu, M.; Xiao, A.; Wu, D.; Zhao, S. Indoor Visual
Positioning Aided by CNN-Based Image Retrieval: Training-Free, 3D Modeling-
Free. Sensors 2018, 18, 2692. https://doi.org/10.3390/s18082692
3

4. © 2021 BASF
Object Detection Edge Cases
Ultimate problem is the lack of a universal ground truth
Partial
occlusions
Flexible
shapes
Pose
dependency
Logos on
other items
Barcode issues
Recycling Bin Top View
Real life example
4

5. © 2021 BASF
Color and “Ground Truth”
• Color seems like an obvious choice for a
ground truth candidate
• Unfortunately, an object’s perceived
color is strongly dependent upon
illumination
• We really want an easy measurement
an object’s reflectance curve- it’s
inherent colorfulness
5

6. © 2021 BASF
Fluorescent Chroma
• Fluorescence is the absorption of light at one
energy and emission of light at another energy
(ex. blue→red)
• Magnitude of emission changes but shape
(chroma) does not under different
illumination
• Ideal “ground truth” IF you can separate
from reflectance
• BASF has developed fluorescent coatings and
camera/lighting designs to extract the
fluorescence chroma and use it for object
detection, classification, and segmentation
6

7. © 2021 BASF
Examples of Fluorescent Objects
7
Similar colors, different color IDs
3D shapes
• Commercial articles
• Plastics
• Printed packages
• Fabrics
• Printable labels
• Post-production
solution (similar to an
added price sticker)
• Custom coatings
7

8. © 2021 BASF
System Design
8
Cameras
LEDs
Bandpass
Filters
System designed in
partnership with
Basic idea: block reflective light
from reaching camera

9. © 2021 BASF
System Operation
9
Capture speed Classification demo

10. © 2021 BASF
Classification Model and Accuracy
• Can use many models for classification,
but nearest neighbors with
neighborhood component analysis works
well
• Fluorescence segmentation is part of
preprocessing
• Best results with all channels,
fluorescence doing majority of work
• Some LEDs are better than others,
both cameras are better than either
one
10
# LEDs Camera # Channel
Pixel
Accuracy
Accuracy
Rank
Object
Accuracy
6 1,2 all 0.957 1 1
4 1,2 all 0.946 4 1
2 1,2 all 0.885 8 0.996
2 2 all 0.869 9 0.989
2 1 all 0.766 17 0.95
8 1,2 fluorescence 0.864 10 0.966
10 1,2 reflectance 0.62 20 0.765
Reflective Image
Fluorescent
Segmentation
280 classes, different lighting conditions for
training and testing

11. © 2021 BASF
Use Cases and Partnerships
11
Collect
images
Determine
labelling
criteria
Manually
label
images
Train
CNN
Deploy
Update &
retrain
Deploy
As is: Manual image labelling slows whole pipeline
Color
Detect
articles
Collect
images
Label with
Color
Detect
Train
CNN
Deploy
Update &
retrain
To be: Color Detect automatically labels images, increasing workflow rate and retraining speed
Deploy
Visual
AI
Training
Retail
Get in touch (colordetect@basf.com)
if you think Color Detect technology
could help your business or project!
Packaged
Goods
Commerce

12. © 2021 BASF
Conclusions
• Fluorescent chroma can be separated
from reflectance
• Fluorescent chroma can be used to
improve object classification and
segmentation model performance
• Applications in retail, visual AI model
training, and more
12
Thank you!

13. © 2021 BASF
Resource Slide
13
2021 Embedded Vision Summit
Color based Object Detection System
for Visual AI Applications Demo
https://www.colordetect.basf.com
https://www.edge-ai-
vision.com/companies/basf/