Global AI Night Cleveland.pptx
•Transferir como PPTX, PDF•
0 gostou•160 visualizações
Slides of my presentation at Global AI Night Cleveland about Azure Video Indexer and video/audio processing to extract metadata
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Global AI Night Cleveland.pptx
Semelhante a Global AI Night Cleveland.pptx (20)
Mais de Luis Beltran
Mais de Luis Beltran (20)
Último
Último (20)
Global AI Night Cleveland.pptx
- 1. Azure Video Indexer: Advanced metadata extraction from video and audio content Luis Beltrán @darkicebeam Cleveland
- 3. Computer Vision Face/Emotion Recognition OCR/Handwriting Custom Vision Video Indexer Text-to-Speech Speech-to-Text Translator Custom Speech Language Understanding Text Translator Text Analytics QnA Maker Bing Custom Search Bing Visual Search Bing Autosuggest Bing Image Search Bing News Search microsoft.com/cognitive Vision Voice Language Web Search Bringing AI within reach of every developer—without requiring machine-learning expertise! Azure Cognitive Services
- 4. Custom Vision Customize image recognition and tailor it to your business needs Computer Vision API Analyze content in images and videos Face API Detect and identify people and emotions in images Form Recognizer Extract text, key-value pairs, and tables in documents Computer Vision
- 5. Imagine if you could… • Automatically create all of the metadata (spoken text, written text, faces, places, objects etc) for any video, including from their archives. • As a call center operator, scan for frustrated customers or count time spent on hold. • Collaboratively upload & edit videos in real-time, search to find key moments, and speed up time to publish new content • Enable your viewers to search by Name, Search by Keyword, Search by Shot Location, key objects • Distribute playable clips of live video content internally or externally without long download or encode times • Create automated summaries or highlight reels of their video content based on scene detection, specific people, and motion within the video • Translate your content into other languages for global and diverse audience reach.
- 6. Video AI is the key to solve these challenges • Improve Content Discoverability • Increase Content Value • Personalize the Viewing Experience • Uncover Hidden Content Insights • Reduce Manual Labor • Increase Revenue / Drive Viewing • Targeted Advertising • Predictive modeling & recommendations
- 7. • Azure Video Indexer is a service that extracts valuable insights from media. • It uses machine learning models that can be further customized and trained. • The video insights include face identification, text recognition, object labels, scene segmentations… • Additional insights are extracted from audio, such as transcription and emotion detection. • You can use these results to improve search, extract clips, create thumbnails, and more, thus enhancing user engagement. • It is available as a service and as a platform. Video Indexer Video Indexer Analyze the visual and audio channels in a video, and index its content.
- 8. Subscribe to the API • To use Video Indexer, you need to create a subscription. • You can sign up for a trial: • Up to 600 minutes of free indexing using the Video Indexer Portal • Up to 2400 free minutes using the API. • If you sign up for a trial, you will have a subscription created automatically. videoindexer.ai
- 9. Video Indexer Portal • Upload videos • From URL • From File • Most common media formats are supported (MOV, WMV, MPG, AVI, MP4…) • Features are indexed
- 10. Video Insights
- 11. Identifying people in the video
- 12. Getting a full transcript + Translating it to another language
- 13. Searching content inside the video
- 14. Get your Account id
- 17. 3 simple steps
- 18. Get your Subscription key
- 19. Exploring the Video Indexer Developer Portal: Getting an Account Access Token
- 20. Send the request using the API console
- 21. Get the code in your favorite language
- 22. Exploring the Video Indexer Developer Portal: Searching videos
- 23. Send the request
- 24. Demo
- 27. Video Indexer Pricing Pricing for Media Services: https://azure.microsoft.com/en-us/pricing/details/media-services/
- 28. Call to Action • Video Indexer https://www.videoindexer.ai/ • Video Indexer API Portal https://api-portal.videoindexer.ai/ • Extract insights from videos with the Video Indexer Service https://docs.microsoft.com/en-us/learn/modules/extract-insights- from-videos-with-video-indexer-service/
- 29. Thank you for your attention! Luis Beltrán Tomás Bata University in Zlín Tecnológico Nacional de México en Celaya luis@luisbeltran.mx luisbeltran.mx @darkicebeam About Me: https://about.me/luis-beltran
Notas do Editor
- Your goal is to help travelers search and filter to find videos uploaded by others. Proper indexing of media and extracting insights is a challenge. The Video Indexer can extract insights from posted videos to help build a stunning platform with easy to implement features.
- Azure Cognitive Services is a suite of services and APIs backed by machine learning that enables developers to incorporate intelligent features such as facial recognition in photos and videos, sentiment analysis in text, and language understanding into their applications.
- Video Indexer is one of the newest members of the Cognitive Services family. Its purpose is to transform raw video content into content that is searchable, discoverable, and more engaging to the user. Want to generate a video transcript, index words spoken in the video or written on a whiteboard, or create a list of keywords from topics discussed in the video? Video Indexer can do all this and more. It can even find individuals in the video, and sometimes tell who they are. Video Indexer is both a service and an API. The service is accessed through a Web portal. It allows you to upload videos and examine the information generated from them. The Video Indexer API is a REST API that does everything the portal does, and also allows you to access the information that is generated when videos are indexed.
- Open the Video Indexer portal in your browser and select Get Started. Then select Sign in with your work account (AAD) to sign in with a work or school account, or Sign in with your personal Microsoft account to use a personal Microsoft account. Answer Yes if prompted to let this app access your info. Select the Upload button. Then select enter a file url to upload a video from a URL. Paste the following URL into the URL field, and enter "Overview of the Microsoft AI School" as the video name. Then select Upload to begin the upload. https://topcs.blob.core.windows.net/public/Machine-Learning-in-IoT-solutions_high.mp4 Machine Learning in IoT Solutions When indexing is complete, you will receive an e-mail notification for each video with a link to the video and a short description of what was found in it, such as people, topics, and keywords. Wait for all three videos to finish indexing, and then proceed to the next exercise.
- In Video Indexer, insights are aggregated views of the knowledge extracted from a video, such as faces, keywords, and sentiment. For example, you can see the faces of people appearing in the video, as well as time ranges and percentages for each face shown. Video Indexer cross-references the faces that it finds against a database of thousands of famous people and automatically identifies them. You can see for yourself by opening the "Microsoft in Education" video in the portal. Microsoft CEO Satya Nadella appears in that video, and Video Indexer recognizes him. Video Indexer automatically generates video transcripts based on its built-in speech and speaker recognition services. It even provides facilities for editing the information that it generated so you can correct errors in transcripts, put names to faces that weren't recognized, and more. The "Insights" tab shows people featured in the video, keywords generated from the video, topics identified in the video, brands featured in the video, and even emotions found in the video. You can select Play next for any of these items and cycle through the corresponding points in the video. In this example, Video indexer found two people in the video. It was unable to identify them because they don't appear in its database of famous people. However, you can lend a helping hand by identifying them yourself. Enable editing by selecting the Edit icon in the upper-right corner. Then select the pencil icon next to "Unknown #1" and enter "Sonya Koptyev" as the person's name. Finish up by pressing Enter to save the change. Repeat this step for the "Unknown #2" in the video. This person's name is "Seth Juarez". Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.” Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term. Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
- The "Insights" tab shows people featured in the video, keywords generated from the video, topics identified in the video, brands featured in the video, and even emotions found in the video. You can select Play next for any of these items and cycle through the corresponding points in the video. In this example, Video indexer found two people in the video. It was unable to identify them because they don't appear in its database of famous people. However, you can lend a helping hand by identifying them yourself. Enable editing by selecting the Edit icon in the upper-right corner. Then select the pencil icon next to "Unknown #1" and enter "Sonya Koptyev" as the person's name. Finish up by pressing Enter to save the change. Repeat this step for the "Unknown #2" in the video. This person's name is "Seth Juarez". Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.” Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term. Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
- Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.” Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term. Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
- Once a video is indexed, you can search its contents. Type "suggestion" into the search box at the top of the page and press Enter. Confirm that the search results include four instances in which the word "suggestion" was found in the video. Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term.
- Video Indexer provides regarding each video that it indexes, and this information is available not only in the portal, but through the Video Indexer API.
- The Video Indexer portal provides a window into the videos that you index and lets you see a wealth of information extracted from them. But the real power of Video Indexer lies in the Video Indexer API, which lets you submit videos for indexing programmatically and access the results using a REST API. In Exercise 4, you will build an app that uses this API to expose content in the videos you indexed in Exercise 1. But to call the API, you must first subscribe to it and obtain an API key that is transmitted in each request. In this exercise, you will create a Video Indexer API subscription and retrieve the API key created for it.
- Open the Video Indexer API portal in your browser and select SIGN IN in the top-right corner. Sign in with your Microsoft account — the same one you used to sign in to the Video Indexer portal. Answer Yes if prompted to let this app access your info. Select Products, and then select Authorization. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Show button next to the Primary Key. Copy the API key to the clipboard, and then select Hide to hide it again. Now that you have an API key, you can write apps that call the Video Indexer API. The API key travels in an HTTP header in each request. Without a valid API key, the Video Indexer API fails requests placed to it. It is the API's way of ensuring that the caller is authorized.
- Open the Video Indexer API portal in your browser and select SIGN IN in the top-right corner. Sign in with your Microsoft account — the same one you used to sign in to the Video Indexer portal. Answer Yes if prompted to let this app access your info. Select Products, and then select Authorization. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Show button next to the Primary Key. Copy the API key to the clipboard, and then select Hide to hide it again. Now that you have an API key, you can write apps that call the Video Indexer API. The API key travels in an HTTP header in each request. Without a valid API key, the Video Indexer API fails requests placed to it. It is the API's way of ensuring that the caller is authorized.
- There is no cost associated with the AI part because it doesn't require an Azure subscription.
- The Video Indexer API is a rich one that includes methods for uploading videos for indexing, searching indexed videos, retrieving and modifying transcripts, monitoring the processing state as a video is being indexed, and more. One of the more powerful methods is Get Video Index, which returns the indexed content of a video containing the same kind of detailed information found in the Video Indexer portal after a video is indexed. Video Explorer uses the Search Videos method, which is just one of more than 30 methods featured in the Video Indexer API. You could leverage additional APIs to make Video Explorer richer and more interactive. For example, you could allow users to search for people that appear in a video, or search specifically for text that is extracted via OCR. You could even use the Get Video Player Widget URL method to embed a video player in the app. Feel free to use these APIs to expand Video Explorer and customize it to fit your needs, and learn more about Video Indexer in the process.