This document discusses building a serverless image recognition chatbot for WhatsApp using Azure services. It begins with an overview of chatbots and their advantages. It then discusses using LUIS for natural language processing, Computer Vision API for image analysis, and Twilio API to integrate with WhatsApp. Live demos are provided of these Azure services, including using Azure Functions for serverless image processing. The document concludes with a final integrated demo of the chatbot and a call to action.

1. Serverless Image Recognition in
WhatsApp chatbot with Azure AI
Luis Beltrán
Microsoft MVP | Zlín, Czech Republic
https://www.linkedin.com/company/pdcconf @PDCConf https://www.facebook.com/pdcconf

2. Thank you to all our generous sponsors
Supported by Powered by Organized by
Sponsored by

3. Luis Beltrán
Serverless Image Recognition in
WhatsApp chatbot with Azure AI
@darkicebeam
/luisantoniobeltran
Powered By
September 16th & 17th
Online Event
International Conference
Speaker

4. Agenda
• Chatbots
• LUIS: Language Understanding Intelligent
Service
• Computer Vision
• Twilio API for WhatsApp
• Azure Functions
• Demo

5. What is a chatbot?
• An AI-based software de IA that
establishes a conversation with a human
using natural language.
• It is considered as one of the most
advanced human-machine interactions.

6. How does a chatbot work?
Chatbot: What is a Chatbot? Why are Chatbots Important?
https://expertsystem.com/chatbot/

7. Advantages of chatbots
• Chatbots can combine millions of data from different sources.
• Chatbots can be available 24/7/365.
• Chatbots can be integrated into (almost) any platform.
• Chatbots can improve the user experience and keep human effort to a
minimum.

8. Data
exploration /
collection
Simple and
repetitive
solutions
If it is simple and repetitive, it
can be coded. Bots can speed
up the service process

10. LUIS: Language Understanding Intelligent Service
• Natural Language Processing
• A cloud-based service for
understanding text.
• Simple to use and integrate
into your software
developments.
• No prior AI experience is
required.
https://www.luis.ai/

12. Demo 1 – LUIS

17. Computer Vision
• Computer vision is one of the core
areas in Artificial Intelligence.
• It focuses on creating solutions
that allow apps to "see" the world
and make sense of it through AI
capabilities.

18. Object detection
Brand identification
Face detection
OCR – Optical Character Recognition

19. Image Analysis
• The identification of “tags”
that catalog the content of an
image.
• The generation of descriptive
subtitles that summarize the
scene shown in the image.

20. Computer Vision API

21. Demo 2 – Computer Vision

22. What is WhatsApp?
• A free messaging application available for
smartphones (Android / iOS), web (WhatsApp Web),
and desktop (Windows / Mac).
• Capabilities:
• Sending text messages, photos, videos, documents, voice
messages and stickers,
• Making and receiving audio and video calls.
• Allows you to manage chat groups.
• Offline messages
• Two versions are available: Messenger & Business

24. WhatsApp Business API
• Boost communication with your
customers around the world so that
you can contact them through
WhatsApp in a simple, safe and
reliable way.
• It requires contracting with a global
business solutions provider,
available on Facebook's partner
directory.
https://www.whatsapp.com/business/api

25. https://www.facebook.com/business/partner-directory/search?platforms=whatsapp&solution_type=messaging

26. Twilio
• Twilio is a cloud Communications Platform as a Service (CPaaS) that
allows you to build applications through APIs and SDKs available for
several languages and platforms (PHP, Ruby, C # /. NET).
• Some features that developers can integrate into their apps with Twilio
include:
• Text messages
• Voice messages
• Video calls
• WhatsApp messages

27. Twilio API for WhatsApp
• Access to the WhatsApp Business API.
• Simple to use and implement.
• Test and design your app in a sandbox
environment.
• Currently in Preview version
• Available on various platforms and programming
languages
https://www.twilio.com/whatsapp
https://www.twilio.com/docs/whatsapp/api

31. Demo 3 – Twilio Dashboard

32. Code Events + data
Azure Functions
Azure Functions

33. Serverless computing
Event-driven / Instant scale Micro-billing
Abstraction of servers
$
Focused on business logic Faster time to market
Reduced DevOps

34. Your code runs when a
(previously-specified)
event occurs
Blob Storage
Cosmos DB
Event Hub
HTTP
Queues
Service Bus
Timer
Webhook
File
Table
Excel
OneDrive
Email
Mobile app
Notification
Más…
Photo taken and sent r
Upload it to
blob storage
PP
r
Produce scalated images

35. public static void Run(byte[] image, string filename,
Stream outputBlob, TraceWriter log)
{
log.Info($"Processing image: {filename}");
var imageBuilder = ImageResizer.ImageBuilder.Current;
imageBuilder.Build(
image, outputBlob,
new ResizeSettings(640, 400, FitMode.Max, null), false);
}
{
"bindings": [
{
"name": "image",
"type": "blobTrigger",
"direction": "in",
"path": "card-input/{filename}.jpg",
"connection": "AzureWebJobsStorage"
},
{
"type": "blob",
"name": "outputBlob",
"path": "card-output/{filename}.jpg",
"connection": "AzureWebJobsStorage",
"direction": "out"
}
]
}

38. Demo 4 – Azure Functions

40. Final Demo – Integration

41. 1
2
3
4a
6
4b
5a
5b
7

42. Call to Action
Developing a serverless WhatsApp chatbot
https://dev.to/icebeam7/developing-a-serverless-whatsapp-chatbot-4o72
Enviando mensajes de WhatsApp con Xamarin y Twilio
https://www.luisbeltran.mx/2018/10/15/enviando-mensajes-de-whatsapp-con-xamarin-y-twilio/
How to Use Image Recognition on Twilio WhatsApp API
https://www.twilio.com/blog/image-recognition-clarifai-twilio-whatsapp-api
Sending and Receiving Media with WhatsApp Messaging on Twilio
https://support.twilio.com/hc/en-us/articles/360017961894-Sending-and-Receiving-Media-with-
WhatsApp-Messaging-on-Twilio

43. Towards a responsible AI
• Transparency
• Clarity
• Transfer to a human
• Cultural norms
• Trustworthy
• Privacy
• Data security
• Accessibility
• Responsibility

45. Thank you!