1. Using the power of
Generative AI at scale
Maxim Salnikov
Microsoft

2. • Building on web platform since 90s
• Organizing developer communities and technical conferences
• Speaking, training, blogging: Webdev, Cloud, OpenAI
Helping developers to succeed with the Cloud & AI technology
Maxim.Salnikov@microsoft.com
Let’s connect and chat!

3. Of AI used by the enterprises over the past several years
Of executives view AI and machine
learning as priorities that they should
be incorporating into their product
lines and businesses

4. Redmond WA Montreal QB
New York NY
Boston MA
Cambridge, UK
India
Beijing, China
Shanghai, China
Global research
centers
Researchers
employed
worldwide
AI-related
patents
AI research
papers
published
To human parity
on vision,
speech, and
language

5. 2016
2017
2018
2019
2020
2021
2021

6. ML Platform
Customizable AI Models
Cognitive Services
Scenario-Based Services
Applied AI Services
Application Platform
AI Builder
Applications Partner Solutions
Power BI Power Apps Power Automate Power Virtual Agents
Azure Machine Learning
Vision Speech Language Decision
Azure OpenAI
Service
Immersive Reader
Form Recognizer
Bot Service Video Indexer Metrics Advisor
Cognitive Search
Developers &
Data Scientists
Business
Users

8. Ensure that artificial
general intelligence (AGI)
benefits humanity
Empower every person and
organization on the planet
to achieve more
GPT-3.5 and GPT-4
Text
ChatGPT
Conversation
Codex
Code
DALL·E 2
Images

9. Prompt
Write a tagline for
an ice cream shop.
Prompt
Table customers, columns =
[CustomerId, FirstName,
LastName, Company, Address,
City, State, Country,
PostalCode]
Create a SQL query for all
customers in Texas named Jane
query =
Prompt
A ball of fire with vibrant
colors to show the speed of
innovation at our media and
entertainment company
Response
We serve up smiles
with every scoop!
Response
SELECT *
FROM customers
WHERE State = 'TX' AND
FirstName = 'Jane'
Response
Prompt
I’m having trouble getting
my Xbox to turn on.
Response
There are a few things you
can try to troubleshoot
this issue … …
Prompt
Thanks! That worked. What
games do you recommend for
my 14-year-old?
Response
Here are a few games that you
might consider: …
GPT-3.5 and GPT-4 ChatGPT Codex DALL·E 2

10. GPT-3.5 & 4 (preview) ChatGPT (preview)
Codex DALL·E 2 (preview)

11. Fairness

12. Azure OpenAI
Content
generation
Call center analytics:
automatically generate
responses to customer inquiries
Generate personalized UI
for your website
Summarization
Call center analytics:
summary of customer
support conversation logs
Subject matter expert document:
summarization (e.g. Financial
reporting, analyst articles)
Social media trends
summarization
Code
generation
Convert natural language to
SQL (or vice versa) for
telemetry data
Convert natural language to
query proprietary data models
Code documentation
Semantic
search
Search reviews for a specific
product/service
Information discovery and
knowledge mining

13. Azure OpenAI
Finance
Automatically generate
financial summaries
Create natural language
responses for financial queries
IT
Convert natural language to
SQL (or vice versa) for
telemetry data
Convert natural language to
query proprietary data
models
Auto generate code
documentation
Human
resources
Automate recruitment process
by screening resumes and
identifying the most qualified
candidates
Analyze employee sentiment
(e.g., email, etc) providing
insights into company culture
Automate routine tasks
during the onboarding
process, such as sending
welcome emails and
scheduling orientation
sessions
Marketing
Generate idea for content
across different formats (blog
posts, articles, videos, and
social media)
Copywriting: generate ideas,
creating outlines, and writing
dialogue and product
descriptions
Create personalized and
targeted email campaigns to
promote products or services
to a specific audience

14. Azure OpenAI
Finance
Intelligent Compliance & Risk
Management
Retail Healthcare
Personalised Customer Care
Cost Savings through
Optimised Engagement
Innovation w/ Better /
Alternative Treatment
Comms & Media
Hyper-personalisation
Innovative Content Creation
Intelligent Contact Center
Personalised Customer Care
Cost Savings through
Optimised Engagement
Hyper-personalisation
Innovative Marketing and
Campaigning
Intelligent Contact Center

15. Example
Discovery/
exploration
Virtual agents communicate in clear,
conversational language in
customer’s native tongue
Customers get detailed answers to
policy and brand inquiries in
human language
After customer calls, agents receive
automatically generated notes,
recommendations, and call highlights
Agent can use auto generated
emails to expeditiously follow
up and close sales
Agent can use auto generated visuals
to craft customized offers and
marketing material
Buying/using
Following purchase, customer gets
automated and hyper-personalized
onboarding docs
Customers can easily understand
what they have bought as
complex contract language is
presented in lay terms
Customer can initiate claims
digitally (Digital FNOL) using
non-technical terms
Prior to customer appts, agents can
generate script to drive conversation,
based on customer profile and
conversation history
Agent can auto generate emails
and replies in Outlook to
collect/inform customers on
important issues like claims
Asking
In chat, insurer provides an
expanded knowledge base to answer
customer inquiries
Agents can automate follow up
activities and track, measure, and
evaluate the inquiries they receive
Agents/reps can retrieve
knowledge base responses to
customer queries in lay terms
Engaging
During milestones, outreach to
customer is automated and
relevant to their risk aversion
and protection needs
Alert agents on important
events (birthdays, weddings)
and generate automated emails
to drive engagement
Scale
Customer
Agent

16. CarMax produced 11 years'
worth of car summaries in a
matter of months.
Using Azure OpenAI Service
reduces their marketing
freelancer spend.
It also enables them to
provide new information to
their customers faster than
their competitors.
“With Azure OpenAI Service,
we are able to create content
that empowers our customers
so they can be informed
before they make a decision,”
‒ Shamim Mohammad, CarMax executive
vice president and chief information and
technology officer. “
“Azure OpenAI Service is
definitely playing a big part in
our SEO improvement.”
‒ Sean Goetz, Director, Application Systems

17. Strabag SE builds a risk management solution
to improve efficiency
Challenge
Strabag specializes in construction and related services. The company was
seeking a solution to centralize and better leverage their data in order to
optimize their operations.
Solution
With Microsoft’s Intelligent Data Platform serving as the foundation of Strabag’s
Data Science Hub, they were able to leverage Azure OpenAI Service to create a
solution that allows it to assess the potential impacts of a project in its earliest
stages by comparing it against all its realized projects – therefore minimizing the
risks.
“With this solution, enabled by Azure OpenAI Service, we know when the
weather will prevent us using a crane on-site. Moreover, by the aid of
Generative Design our real estate development unit knows with a few clicks
how much maximum gross floor area they can develop on a site. This kind of
information is vital when you have projects worth two million euros with a
margin of 4-5%.”
Manufacturing Austria
Corporate
(10,000+ employees)

18. The solution to the size and fit problem is knowing the personal preferences of
your customers on each item-model-level

19. • Skjold.ai protects all traffic
• Check suspicious messages
• Cybercrime is growing
• Criminals adopt new technology
• Instant feedback if the
messages is considered safe
or could be fraud
• Explain red flags and
educate the user
• Integrated in the iPhone
SMS app
• Available for iPhone and
Android
AI-powered scan of suspicious
messages
Read more
www.skjold.ai/demo
Mobile first protection against cyber crime built with Azure OpenAI

21. 1. A secret step (on the last slide)
2. Request access by submitting a form on azure.microsoft.com
3. Send your first request to Completions API
4. Experiment with models, prompts, parameters
5. Try different options to use your own data
6. Build a product and share with the world! 🎉

22. Prompt—Text input that
provides some context to the
engine on what is expecting.
Completion—Output that
GPT-3 generates based on
the prompt.
some context
Token — partial or full words
processed and produced by
the GPT models

23. “Classic” GPT-3 models
Previous models were text-in and text-out
(i.e., they accepted a prompt string and returned a completion to
append to the prompt).
Answer questions from the context below.
Context:
A neutron star is the collapsed core of a massive
supergiant star, which had a total mass of between 10
and 25 solar masses, possibly more if the star was
especially metal-rich.
Q: What is a neutron star?
A:
The ChatGPT model
The ChatGPT model is conversation-in and message-out.
(i.e., it expects a prompt string that is formatted in a specific chat-
like transcript format and returns a completion that represents a
model-written message in the chat)
<|im_start|>system
Assistant is an AI Chatbot designed to answer questions
from the context provided below.
Context:
A neutron star is the collapsed core of a massive
supergiant star, which had a total mass of between 10
and 25 solar masses, possibly more if the star was
especially metal-rich.
<|im_end|>
<|im_start|>user
What is a neutron star?
<|im_end|>
<|im_start|>assistant

24. Context
Write a
tagline for
our ice cream
shop.
Prompt Response
Scoops of
heaven in the
heart of
Phoenix!
## This is a conversational agent whose code name is Dana:
- Dana is a conversational agent at Gourmet Ice Cream, Inc.
- Gourmet Ice Cream’s marketing team uses Dana to help them be more
effective at their jobs.
- Dana understands Gourmet Ice Cream’s unique product catalog, store
locations, and the company’s strategic goal to continue to go upmarket
## On safety:
- Dana should moderate the responses to be safe, free of harm
and non-controversial.
## On Dana’s ability to gather and present information:
- Dana’s responses connect to the Product Catalog DB, Store Locator DB,
and Microsoft 365 it has access to through the Microsoft Cloud,
providing great CONTEXT
## On Dana’s profile and general capabilities:
- Dana’s responses should be informational and logical
- Dana’s logic and reasoning should be rigorous, intelligent and defensible

25. Extract precise answers from your unstructured documents with Azure OpenAI + Cognitive Search: article + repo

26. 1. Give clearer instructions
2. Split complex tasks into simpler subtasks
3. Structure the instruction to keep the model on task
4. Prompt the model to explain before answering
5. Ask for justifications of many possible answers, and then synthesize
6. Generate many outputs, and then use the model to pick the best one
7. Fine-tune custom models to maximize performance

27. Azure OpenAI
Service

28. Fine-Tuning
Domain Data
Small Set of Labeled
Data
• minimum of several
thousand examples
• Maximum of 2.5M tokens
or 80-100mb size
Fine-Tuned GPT-3 Model
Perform any domain-specific
NLP tasks
Model parameters adjusted
(Weights and Bias)
Gradient Update
high-dimensional
vector space
(Embeddings)
GPT-3
Fine-tuning
• Higher quality results than prompt
design
• Ability to train on more examples than
can fit in a prompt
• Token savings due to shorter prompts
• Lower latency requests

30. ◉ Learn more in the Azure OpenAI WorkshopI
◉ See examples in the Open AI Cookbook
◉ Start experimenting and developing your MVP
◉ Get support and help
◉ How-to guides: Generate embeddings and Fine-tune
◉ Complete embeddings and document search tutorial.
◉ How your/your customer data is processed, used and stored in Azure OpenAI:
Data, privacy and security
◉ Review the Enterprise Data with ChatGPT tech blog, and work through the
accompanying GitHub repository.
◉ Become familiar with the APIs
◉ Understand “What is Azure OpenAI?”: compare Azure OpenAI and
OpenAI. Review key concepts.
◉ Work through Introduction to Azure OpenAI Service training
module
◉ Explore key Responsible AI guidelines and principles
◉ Create an Azure subscription
◉ Apply for access to the Azure OpenAI Service:
https://aka.ms/oai/access )
◉ Watch the video: An Introduction to Azure OpenAI
◉ Get Familiarized with the product content: Azure OpenAI Page
◉ Review how other customers are using Azure AI

31. Maxim.Salnikov@microsoft.com
Let’s connect and chat!