Infuse your apps, websites and bots with intelligent algorithms to see, hear, speak, understand and interpret your user needs through natural methods of communication. Azure Cognitive Services are APIs, SDKs, and services available to help developers build intelligent applications without having direct AI or data science skills or knowledge.

1. Please check that you have your Azure
subscription ready for the workshop ☺
azure.microsoft.com/free
Students:
aka.ms/azure4students

3. Introduction to
Azure Cognitive Services
Roman Sudarikov
Software Engineer @Microsoft
Luis Beltran
Microsoft MVP AI, Developer Technologies
Łukasz Foks
Technologies CEE Azure
Developer Product Marketing
Manager @ Microsoft
Microsoft Prague, September 22nd, 2019
roman-sudarikov-1508a358
luisantoniobeltran
luis@luisbeltran.mx
lukaszfoks

5. Microsoft AI Platform

6. Azure Cognitive Services
Applying AI to your business
Computer Vision
Face/Emotion Recognition
OCR/Handwriting
Custom Vision
Video Indexer
Content Moderator
Text-to-Speech
Speech-to-Text
Translator
Custom Speech
Language Understanding
PII Detection
Text Translator
Text Analytics
QnA Maker
Bing Custom Search
Bing Visual Search

13. Face API

14. Face Detection

15. Emotion Detection

17. Computer Vision API
Content of Image:
Categories v0: [{ “name”: “animal”, “score”: 0.9765625 }]
V1: [{ "name": "grass", "confidence": 0.9999992847442627 },
{ "name": "outdoor", "confidence": 0.9999072551727295 },
{ "name": "cow", "confidence": 0.99954754114151 },
{ "name": "field", "confidence": 0.9976195693016052 },
{ "name": "brown", "confidence": 0.988935649394989 },
{ "name": "animal", "confidence": 0.97904372215271 },
{ "name": "standing", "confidence": 0.9632768630981445 },
{ "name": "mammal", "confidence": 0.9366017580032349,
"hint": "animal" },
{ "name": "wire", "confidence": 0.8946959376335144 },
{ "name": "green", "confidence": 0.8844101428985596 },
{ "name": "pasture", "confidence": 0.8332059383392334 },
{ "name": "bovine", "confidence": 0.5618471503257751,
"hint": "animal" },
{ "name": "grassy", "confidence": 0.48627158999443054 },
{ "name": "lush", "confidence": 0.1874018907546997 },
{ "name": "staring", "confidence": 0.165890634059906 }]
Describe
0.975 "a brown cow standing on top of a lush green field“
0.974 “a cow standing on top of a lush green field”
0.965 “a large brown cow standing on top of a lush green field”

18. Old OCR New OCR
OCR

20. Vision Speech Language
Natural Language Processing
Intent: PlayCall
Knowledge
Here are the top results:
The purpose of Customer Life-cycle Management (CLM)
is to maximize both customer retention and .... Predictive
trend analysis provides business visibility.
Oct 28, 2015 – Here are FIVE key trends in 2014 that
would help marketers in rolling ... Of late, marketers are
looking at customer lifecycle management (CLM)
Jan 5, 2016 – The top 10 customer service trends for
2016 that .... North American Consumer
Search
Here is what I found:
It also investigates the top three expected Fraud
Detection and Prevention programs, in terms of
demand in key markets…
First, let’s point out that there is not one
absolute answer—there are “pros” and “cons” to
each. Those who favor in-house…
Michael heads fraud prevention tool. Online and
mobile shopping are expected to continue
growing apace…
A variety of real-world applications

21. Apps powered by Azure Cognitive Services

24. Call to Action
• Azure Cognitive Services
aka.ms/cognitive-services
• Documentation:
https://docs.microsoft.com/en-us/azure/cognitive-services/
• GitHub Samples:
https://github.com/Azure-Samples/cognitive-services-REST-api-samples
• Container Support:
https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-
container-support

25. Thank you for your attention!
Roman Sudarikov
Software Engineer @Microsoft
Luis Beltran
Microsoft MVP AI, Developer
Technologies
Łukasz Foks
Technologies CEE Azure
Developer Product Marketing
Manager @ Microsoft
Microsoft Prague, September 22nd, 2019
roman-sudarikov-1508a358 luisantoniobeltran
luis@luisbeltran.mx
lukaszfoks

26. Last call: Please check that you have your
Azure subscription ready for the workshop ☺
azure.microsoft.com/free
Students:
aka.ms/azure4students

27. Deep Dive into
Azure Custom Vision
Roman Sudarikov
Software Engineer @Microsoft
Luis Beltran
Microsoft MVP AI, Developer Technologies
Łukasz Foks
Technologies CEE Azure
Developer Product Marketing
Manager @ Microsoft
Microsoft Prague, September 22nd, 2019
roman-sudarikov-1508a358
luisantoniobeltran
luis@luisbeltran.mx
lukaszfoks

28. 28
Prepare Data
Image Classification
Build & Train
Run
Model definition & training
Model Evaluation
Deploy the model - web service, Dockers Container or IoT EdgeScore the model

30. What is it?
Custom Vision Service is an easy-to-use tool for prototyping,
improving, and deploying a custom image classifier to a cloud service,
without any background in computer vision or deep learning required.
Model

31. Object Detection

32. Building a Classifier
• Create a project
• Select a domain
• Add images
• Assign tags to images
• Train the classifier
• Evaluate the classifier

33. Classifiers and Projects
A classifier is a model you
build using Custom Vision
Service, by using a few
training images.
Each classifier you build is
in its own project.
Classifier = Project

34. Domains
When you create a
project, you select a
domain for that project.
The domain optimizes a
classifier for a specific
type of object in your
images.
• Food
Optimized for dishes you would see on a restaurant
menu.
• Landmark
Optimized for recognizable landmarks, both natural
and artificial.
• Retail
Optimized for classifying images in a shopping
catalog or shopping website.
• Adult
Optimized to better define between adult content
and non-adult content.

35. Training Images
To create a high precision
classifier, Custom Vision
Service needs several
training images.
A training image is a
photograph of the image
you want Custom Vision
Service to classify.

36. Iteration
Every time you Train
or re-train your
classifier, you create
a new iteration of
your model.

37. Testing a Model
After you train your
model, you can quickly
test it using a locally
stored image or an
online image.
The test uses the most
recently trained
iteration.

38. Important Terms
Precision
When you classify an image,
how likely is your classifier to
correctly classify the image?
Recall
Out of all images that should
have been classified correctly,
how many did your classifier
identify correctly?

39. Using the Prediction API
After a successful training,
the Custom Vision Service
can be accessed via an
endpoint that references the
Project Identifier, as long as
the appropriate Prediction
Key is passed in the request
header.

40. Prediction API REST Concepts
All actions related to the
Custom Vision Service are
accessed via standard REST-
based methods, such as
GET and POST against an
API endpoint, making it
simple to use the Prediction
API on any platform or with
any programming language.

41. Train in the Cloud, Run Anywhere
Train in Custom Vision Service Deploy & Run Anywhere

42. Commonly Used APIs
PredictionTraining
• Createimages
• Tagimages
• Createprojects
• Manageprojects
• Manageiterations
• Createtags
• Getaccountinformation
• Trainaproject
• Predictimages
• Predictandsaveimages
• PredictimageURLs
• PredictandsaveimageURLs

43. Improving a Classifier
The best way to have a quality classifier is to add
more varied tagged images (different backgrounds,
angles, object size, groups of photos, and variants of
types.)
Always to train your classifier after you have added
more images. Include images that are representative
of what your classifier will encounter in the real
world.
Photos in context are better than photos of objects in
front of neutral backgrounds, for example.

44. Best Practices for using Custom Vision
• Use at least 30 images for each tag
• Images should be the focus of the picture
• Use sufficiently diverse images and backgrounds (ex: cats with red
background and dogs with blue background)
• Train with images that are similar in {quality, resolution, lighting, etc.} to
the images that will be used in prod
• Supports Microsoft accounts (MSA) and AAD

45. Computer Vision Scenario Examples
➢ Additional Scenarios
➢ Classify user submitted images to website
➢ Identifying elements – object counting, animal identification and lots more.
➢ Hazard detection/industrial safety – adding custom rules to videos

47. Call to Action
• Azure Custom Vision
https://www.customvision.ai/
• Documentation:
https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-
service/

48. Thank you for your attention!
Roman Sudarikov
Software Engineer @Microsoft
Luis Beltran
Microsoft MVP AI, Developer
Technologies
Łukasz Foks
Technologies CEE Azure
Developer Product Marketing
Manager @ Microsoft
Microsoft Prague, September 22nd, 2019
roman-sudarikov-1508a358 luisantoniobeltran
luis@luisbeltran.mx
lukaszfoks