Open Source Lisbon - Francisco Teixeira Presentation

•

0 gostou•59 visualizações

Patrícia Costa

"In data we trust, a smarter future to build Azure Databricks" - Francisco Teixeira

Tecnologia

Shape the future with the
power of data and Azure
Databricks

WATER
LEVEL
FULL
pH
NEUTRAL
14
0
SMART COLLAR
Retuned home at 5:07pm
3 NEW
MESSAGES
Re: Medical Education
26° INDOOR
TEMPERATURE
FAVORITE STATION
Jazz Classics
MORNING COFFEE
Normally wakes up at 7:00am
PURCHASED ON AMAZON
Reminder: Shall I reorder
more detergent?
THE DIGITALERAIS HERE
What does this mean to you?

Artificial
Intelligenc
e
3D Printers
1% of 2007 Cost
Smartphones
2% of 2007 Cost
Sensors
.3% of 2007 Cost
Solar
.5% of 1984 Cost
DNA Sequencing
.01% of 2007 Cost
Industrial Robots
3% of 2007 Cost
Mixed Reality
1% of 2007 cost
Drones
1% of 2007 cost
Cloud
Transformative
Technologies
Democratized

Digital companies - faster and disrupt
furtherVALUATION $25b $50b $75b $100b $150b $200b $250b
Y
E
A
R
S
IN
B
U
SI
N
E
S
S
$984b$508b
$862b
$385b
50
75
100
175
25
200
150
125

Streami
ng
Recommendati
ons
Analyti
cs
$5b movie
studio
Mail DVD
Delivery
What did
Netflix do
right?
Cross-device
Intelligent
cloud
Data
Analytics
Agile
Cross-device
Multi-sense
AI
Intelligent
cloud
Agile
Cross-
device
Multi-sense
Data
AgileAnalytics
Agile
Intelligent cloud
AI
Data
A
I
Intelligent
cloud
Agile
Data
Agile
Multi-
senseAnalytics
Cross-device
2004 2006 2008 2010 2012 2014 2016
#NETFLIX
#5billionMovieStudio
2002
v
v

Azure Databricks
Powered by Apache Spark

• Open-source data processing engine built around speed, ease of use, and sophisticated analytics
• In memory engine that is up to 100 times faster than Hadoop
• Largestopen-source data projectwith 1000+ contributors
• Highly extensible with support for Scala, Java and Python alongside Spark SQL, GraphX,
Streaming and Machine Learning Library (Mllib)
Why Spark?

What is Azure Databricks?
A fast, easy and collaborative Apache® Spark™ based analytics platform optimized for Azure
Best of Databricks Best of Microsoft
Designed in collaboration with the founders of Apache Spark
One-click set up; streamlined workflows
Interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.
Native integration with Azure services (Power BI, SQL DW, Cosmos DB, Blob Storage)
Enterprise-grade Azure security (Active Directory integration, compliance, enterprise-grade SLAs)

Optimized Databricks Runtime Engine
DATABRICKS I/
O
SERVERLESS
Collaborative Workspace
Cloud
storage
Data warehouses
Hadoop storage
IoT / streaming
data
Rest APIs
Machine learning
models
BI tools
Data exports
Data warehouses
Azure Databricks
Enhance Productivity
Deploy Production Jobs &Workflows
APACHE
SPARK
MULTI-STAGE
PIPELINES
DATA
ENGINEER
JOB SCHEDULER NOTIFICATION &
LOGS
DATA
SCIENTIST
BUSINESS
ANALYST
Build on secure &trusted cloud Scale without limits
Azure Databricks

Modern Big Data Warehouse
Business / customapps
(Structured)
Logs, files and media
(unstructured)
Azure storage
Polybase
Azure SQL Data Warehouse
Data factory
Data factory
Azure Databricks
(Spark)
Analytical dashboards
Model &ServePrep &TrainStoreIngest Intelligence

Advanced Analytics on Big Data
Web &mobile appsAzure Databricks
(SparkMllib,
SparkR, SparklyR)
Azure Cosmos DB
Business / customapps
(Structured)
Logs, files and media
(unstructured)
Azure storage
Polybase
Azure SQL Data Warehouse
Data factory
Data factory
Analytical dashboards
Model &ServePrep &TrainStoreIngest Intelligence

Real-time analytics on Big Data
Unstructured data
Azure storage
Polybase
Azure SQL Data Warehouse
Azure HDInsight
(Kafka)
Azure Databricks
(Spark)
Analytical dashboards
Model &ServePrep &TrainStoreIngest Intelligence

Engage Microsoft experts for a workshop to help identify
high impact scenarios
Already using Azure? try Azure Databricks now or
create a free Azure account to start using Azure Databricks
Learn more about Azure Databricks
www.azure.com/databricks
How to get started

Thank you!
Francisco Teixeira
@fratei
aka.ms/francisco.teixeira

Mais conteúdo relacionado

Último

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Histor y of HAM Radio presentation slide

vu2urc

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

Developing An App To Navigate The Roads of Brazil

V3cube

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Destaque

How to Prepare For a Successful Job Search for 2024

Albert Qian

A report by thenetworkone and Kurio. The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).

Social Media Marketing Trends 2024 // The Global Indie Insights

Kurio // The Social Media Age(ncy)

The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands. It’s important that you’re ready to implement new strategies in 2024. Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024. You’ll learn: - The latest trends in AI and automation, and what this means for an evolving paid search ecosystem. - New developments in privacy and data regulation. - Emerging ad formats that are expected to make an impact next year. Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape. If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.

Trends In Paid Search: Navigating The Digital Landscape In 2024

Search Engine Journal

From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world. With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively. The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage. Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience. See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers See the original article on Forbes here: http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b

5 Public speaking tips from TED - Visualized summary

SpeakerHub

Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world. Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance. For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age. Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.

ChatGPT and the Future of Work - Clark Boyd

Clark Boyd

Getting into the tech field. what next

Tessa Mero

Google's Just Not That Into You: Understanding Core Updates & Search Intent

Lily Ray

How to have difficult conversations

Rajiv Jayarajah, MAppComm, ACC

Introduction to Data Science

Christy Abraham Joy

Time Management & Productivity - Best Practices

Vit Horky

The six step guide to practical project management If you think managing projects is too difficult, think again. We’ve stripped back project management processes to the basics – to make it quicker and easier, without sacrificing the vital ingredients for success. “If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.” Dr Andrew Makar, Tactical Project Management

The six step guide to practical project management

MindGenius

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...

RachelPearson36

During this webinar, Anand Bagmar demonstrates how AI tools such as ChatGPT can be applied to various stages of the software development life cycle (SDLC) using an eCommerce application case study. Find the on-demand recording and more info at https://applitools.info/b59 Key takeaways: • Learn how to use ChatGPT to add AI power to your testing and test automation • Understand the limitations of the technology and where human expertise is crucial • Gain insight into different AI-based tools • Adopt AI-based tools to stay relevant and optimize work for developers and testers * ChatGPT and OpenAI belong to OpenAI, L.L.C.

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...

Applitools

12 Ways to Increase Your Influence at Work

GetSmarter

ChatGPT webinar slides

Alireza Esmikhani

More than Just Lines on a Map: Best Practices for U.S Bike Routes

Project for Public Spaces & National Center for Biking and Walking

Has your project been caught in a storm of deadlines, clashing requirements, and the need to change course halfway through? If yes, then check out how the administration team navigated through all of this, relocating 160 people from 3 countries and opening 2 offices during the most turbulent time in the last 20 years. Belka Games’ Chief Administrative Officer, Katerina Rudko, will share universal approaches and life hacks that can help your project survive unstable periods when there seem to be too many tasks and a lack of time and people.

Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...

DevGAMM Conference

Barbie - Brand Strategy Presentation

Erica Santiago

According to the latest State of the American Manager report from Gallup, employees who have regular meetings with their managers are almost three times as likely to be engaged as those who don’t. These regular check-ins keep managers and employees in sync and aligned. Want to see better manager/employee relationships in your organisation? Then make an all-in commitment to 1:1 meetings. Not sure how? You’ve come to the right place. In this webinar with Jamie Resker, Founder and Practice Leader for Employee Performance Solutions (EPS), and Teala Wilson, Talent Management Consultant at Saba Software, you’ll get the inside track on how to hold effective 1:1 meetings, including tips for getting managers on board. • Go beyond discussing the status of everyday work to higher level topics, including recognition, performance, development, and career aspirations • Learn how to decide meeting frequency, what to cover, as well as roles and responsibilities of the manager and employee • Understand how managers can build trust and make it comfortable for employees to provide upward feedback • Unite your organisation with a unified approach to 1:1 meetings Join us for this 1-hour webinar to get practical tips for building better manager-employee relationships with intention and purpose. About the Speakers Jamie Resker - Founder and Practice Leader for Employee Performance Solutions (EPS) Jamie Resker, Practice Leader and Founder of Employee Performance Solutions, is a recognized innovator in performance management. She is the originator of the-the Performance Continuum Feedback Method® and Conversations to Optimize Employee Performance training program; tools and training that reshape communications between managers and employees to drive and align performance. Jamie is on the faculty for the Northeast Human Resources Association, is a contributor to Halogen Software's Talent Space Blog, and is an editorial advisory board member for HR Examiner. Teala Wilson - Senior Consultant, Strategic Services, Saba Software Teala is a Talent Management Consultant at Halogen Software, now a part of Saba Software. She has worked with teams on a national and global level supporting human resources in areas such as performance management, recruitment, employee benefit programs, training and talent development, workforce planning and internal communications. Teala also has a personal passion for visual arts and design. Want to learn more? Join us for an upcoming Product Tour! http://bit.ly/2yitfqu

Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well

Saba Software

This video by Simplilearn will explain to you Introduction to C Programming Language. Introduction to C Programming Language Tutorial For Beginners will explain to you the C language's history, C's importance, its features, real-world applications, and some of its advantages and disadvantages. 00:00 Introduction to C 1:42-History of C language Dennis Ritchie, a computer scientist, could identify the gaps and tap out the best features from both B and BCPL languages to invent a new hybrid. Hence, C was born in 1972 at Bell Laboratories. A remarkably simple and highly readable programming language resulted in groundbreaking advancements in the IT industry. 2:48-Importance and unraveling the powerful capabilities of C, The widespread use of C started to take over the IT industry. Unraveling the potential of C, the designers began to discover new possibilities that led them to focus on the big picture. 3:56-C's cutting-edge features The designers at Bell Laboratories ensured that their programming language solved the issues with B and BCPL and the ones they had foreseen. 6:35-The popular real-world applications of C -UNIX operating system -google file system -Mozilla -Graphical user interface 8:30-The advantages and disadvantages of C 10:34-The popular IT companies and their domains that employ C · MasterCard · IBM · Flipkart · Dell · Twitter · GitHub and twitch 11:09-First c program. 🔥 Explore our FREE courses with completion certificates: https://www.simplilearn.com/skillup-f... ✅Subscribe to our Channel to learn more about the top Technologies: https://bit.ly/2VT4WtH ⏩ Check out the C++ Programming training videos: https://www.youtube.com/playlist?list... #IntroductiontoCProgrammingLanguage #CLanguage #CProgramming #CProgram #CProgrammingLanguage #LearnCProgramming #HowToCodeInCForBeginners #CTutorialForBeginners #LearnCProgramming #Simplilearn Dennis Ritchie, a computer scientist, was able to identify the gaps and tap out the best features from both B and BCPL languages to invent a new hybrid. Hence, C was born in 1972 at Bell Laboratories. A remarkably simple and highly readable programming language resulted in groundbreaking advancements in the IT industry. ✅What is C++ Programming? C++ is an enhanced and extended version of C programming language, developed by Bjarne Stroustrup in 1979 as part of his Ph.D. project. Bjarne developed what he called ‘C with Classes’ (later renamed C++) because he felt limited by the existing programming languages that were not ideal for large scale projects. He used C to build what he wanted because C was already a general-purpose language that was efficient and fast in its operations. ✅C++ Career Prospects: With just C++ programming expertise, you will have excellent job opportunities, salaries, and career prospects. However, for a career based on programming languages such as Java and Python (which are in more demand than C++) or for careers based on front-end, back-end, and full-stack

Introduction to C Programming Language

Simplilearn

Destaque (20)

How to Prepare For a Successful Job Search for 2024

Social Media Marketing Trends 2024 // The Global Indie Insights

Trends In Paid Search: Navigating The Digital Landscape In 2024

5 Public speaking tips from TED - Visualized summary

ChatGPT and the Future of Work - Clark Boyd

Getting into the tech field. what next

Google's Just Not That Into You: Understanding Core Updates & Search Intent

How to have difficult conversations

Introduction to Data Science

Time Management & Productivity - Best Practices

The six step guide to practical project management

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...

12 Ways to Increase Your Influence at Work

ChatGPT webinar slides

More than Just Lines on a Map: Best Practices for U.S Bike Routes

Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...

Barbie - Brand Strategy Presentation

Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well

Introduction to C Programming Language

Open Source Lisbon - Francisco Teixeira Presentation

1. Shape the future with the power of data and Azure Databricks

2. WATER LEVEL FULL pH NEUTRAL 14 0 SMART COLLAR Retuned home at 5:07pm 3 NEW MESSAGES Re: Medical Education 26° INDOOR TEMPERATURE FAVORITE STATION Jazz Classics MORNING COFFEE Normally wakes up at 7:00am PURCHASED ON AMAZON Reminder: Shall I reorder more detergent? THE DIGITALERAIS HERE What does this mean to you?

3. Artificial Intelligenc e 3D Printers 1% of 2007 Cost Smartphones 2% of 2007 Cost Sensors .3% of 2007 Cost Solar .5% of 1984 Cost DNA Sequencing .01% of 2007 Cost Industrial Robots 3% of 2007 Cost Mixed Reality 1% of 2007 cost Drones 1% of 2007 cost Cloud Transformative Technologies Democratized

4. Digital companies - faster and disrupt furtherVALUATION $25b $50b $75b $100b $150b $200b $250b Y E A R S IN B U SI N E S S $984b$508b $862b $385b 50 75 100 175 25 200 150 125

5. Streami ng Recommendati ons Analyti cs $5b movie studio Mail DVD Delivery What did Netflix do right? Cross-device Intelligent cloud Data Analytics Agile Cross-device Multi-sense AI Intelligent cloud Agile Cross- device Multi-sense Data AgileAnalytics Agile Intelligent cloud AI Data A I Intelligent cloud Agile Data Agile Multi- senseAnalytics Cross-device 2004 2006 2008 2010 2012 2014 2016 #NETFLIX #5billionMovieStudio 2002 v v

6. Azure Databricks Powered by Apache Spark

7. S P A R K : A B R I E F H I S T O R Y

8. • Open-source data processing engine built around speed, ease of use, and sophisticated analytics • In memory engine that is up to 100 times faster than Hadoop • Largestopen-source data projectwith 1000+ contributors • Highly extensible with support for Scala, Java and Python alongside Spark SQL, GraphX, Streaming and Machine Learning Library (Mllib) Why Spark?

9. What is Azure Databricks? A fast, easy and collaborative Apache® Spark™ based analytics platform optimized for Azure Best of Databricks Best of Microsoft Designed in collaboration with the founders of Apache Spark One-click set up; streamlined workflows Interactive workspace that enables collaboration between data scientists, data engineers, and business analysts. Native integration with Azure services (Power BI, SQL DW, Cosmos DB, Blob Storage) Enterprise-grade Azure security (Active Directory integration, compliance, enterprise-grade SLAs)

10. Optimized Databricks Runtime Engine DATABRICKS I/ O SERVERLESS Collaborative Workspace Cloud storage Data warehouses Hadoop storage IoT / streaming data Rest APIs Machine learning models BI tools Data exports Data warehouses Azure Databricks Enhance Productivity Deploy Production Jobs &Workflows APACHE SPARK MULTI-STAGE PIPELINES DATA ENGINEER JOB SCHEDULER NOTIFICATION & LOGS DATA SCIENTIST BUSINESS ANALYST Build on secure &trusted cloud Scale without limits Azure Databricks

11. Use Cases

12. Modern Big Data Warehouse Business / customapps (Structured) Logs, files and media (unstructured) Azure storage Polybase Azure SQL Data Warehouse Data factory Data factory Azure Databricks (Spark) Analytical dashboards Model &ServePrep &TrainStoreIngest Intelligence

13. Advanced Analytics on Big Data Web &mobile appsAzure Databricks (SparkMllib, SparkR, SparklyR) Azure Cosmos DB Business / customapps (Structured) Logs, files and media (unstructured) Azure storage Polybase Azure SQL Data Warehouse Data factory Data factory Analytical dashboards Model &ServePrep &TrainStoreIngest Intelligence

14. Real-time analytics on Big Data Unstructured data Azure storage Polybase Azure SQL Data Warehouse Azure HDInsight (Kafka) Azure Databricks (Spark) Analytical dashboards Model &ServePrep &TrainStoreIngest Intelligence

15. Demo

16. Engage Microsoft experts for a workshop to help identify high impact scenarios Already using Azure? try Azure Databricks now or create a free Azure account to start using Azure Databricks Learn more about Azure Databricks www.azure.com/databricks How to get started

17. Thank you! Francisco Teixeira @fratei aka.ms/francisco.teixeira

Open Source Lisbon - Francisco Teixeira Presentation

Recomendados

Recomendados

Mais conteúdo relacionado

Último

Último (20)

Destaque

Destaque (20)

Open Source Lisbon - Francisco Teixeira Presentation