Data-Driven @ Netflix

•

26 gostaram•8,044 visualizações

Michelle Ufford is a Principal Architect at Netflix who leads their Data Engineering and Analytics team. Netflix has over 86 million members who watch over 125 million hours of content daily on over 1000 supported devices. The data team manages a 40 petabyte data warehouse with 4 petabyte daily reads and 300 terabyte daily writes, processing over 700 billion events. The data is used to predict content value, optimize the user experience, analyze news and PR, monitor global service delivery, and power experimentation.

Tecnologia

Data-Driven @ Netflix
Michelle Ufford
Principal Architect
Data Engineering & Analytics

Michelle Ufford
Highlights
● Principal Architect at Netflix
Data Engineering & Analytics
● Prev. Engineering Manager at GoDaddy
Data Platform
● Microsoft Data Platform MVP
● 10+ years building web-scale analytics &
data engineering infrastructure
● advises on Big Data topics
Microsoft, Hortonworks, Teradata, etc.
Gratuitous picture of my kids

The business numbers.
86.7 million
members
1000+ devices
supported
125+ million
hours watched
launched
19 years ago
every. day.

Any device. Anywhere.*
* Well, almost anywhere.

The data numbers.
4 petabyte
DW reads
300 terabyte
DW writes
40 petabyte
data warehouse
700+ billion
events written

Predicting Value for
Licensed Content.
Feature Engineering Predictive Models License Terms Content Efficiency

Predicting Value for
Licensed Content.
● past performance of similar content on Netflix
● broadcast & Box Office performance
● talent (writers, actors, directors, etc.)
● critic & user reviews
● awards & accolades
Feature Engineering Predictive Models License Terms Content Efficiency

Predicting Value for
Licensed Content.
● terms (length, exclusivity, etc.)
● bid amount
● negotiations
Feature Engineering Predictive Models License Terms Content Efficiency

Predicting Value for
Licensed Content.
● value / cost
● if efficient, license
Feature Engineering Predictive Models License Terms Content Efficiency

“ last year our original content overall
was some of our most efficient content.
”

“ We are building a studio in the cloud
and pioneering new approaches to movie
production, optimizing pitches, production
schedules, subtitling, and digital asset
management for our Original content. ”

There are 86 million different
versions of Netflix.

billboard
rows, row order
titles, title order
title artwork

Public Relations.
Analytics of news.Analytics is news.

Content Delivery.
Monitoring a global service.

YouTube video of Vizceral demo
https://youtu.be/JctsPpgEsVs

data access
AWS
S3
Big Data Platform
Amazon
Redshift
data processing
fast storage data viz
METACA
T
data services
events data
operational data
elastic storage Apache Pig

Data Engineering & Analytics
MarketingProduct PlaybackContent Finance
105 talented engineers & analysts
data viz engineers
analytics engineers
data engineers
Big Data
Platform
analysts

Experimentation Platform
Batch &
Ad Hoc
Analysis

Thank you
for attending!
Michelle Ufford
linkedin.com/in/mufford
@sqlfool
Data @
Netflix
@NetflixData
hadoopsie.com techblog.netflix.com
tinyurl.com/NetflixData

Mais conteúdo relacionado

Mais procurados

Cloud-native Semantic Layer on Data Lake

Databricks

The purpose of this presentation is providing an overview of the main approaches in using big data: data focus vs. business analytics focus. The following topics will be covered: - Why getting data should not be a starting point in business analytics, and why more data not always result in more accurate predictions - The simulation analytics methodology in comparison to machine learning and data science approach - Examples of two business cases: (i) Healthcare: Pediatric Triage in a Severe Pandemic-Maximizing Population Survival by Establishing Admission Thresholds (ii) Banking & Finance: Analysis of the staffing and utilization of a team of mutual fund analysts for timely producing ‘buy-sell’ reports

DATA ANALYTICS FOR SOLVING BUSINESS PROBLEMS

Alexander Kolker

Data Driven Design

Tanya M.

We present you content-ready big data characteristics and process PowerPoint presentation that can be used to present content management techniques. It can be presented by IT consulting and analytics firms to their clients or company’s management. This relational database management PPT design comprises of 53 slides including introduction, facts, how big is big data, market forecast, sources, 3Vs and 5Vs small Vs big data, objective, technologies, workflow, four phases, types, information analytics process, impact, benefits, future, opportunities and challenges etc. Our data transformation PowerPoint templates are apt to present various topics such as information management concepts and technologies, transforming facts with intelligence, data analysis framework, data mining, technology platforms, data transfer and visualization, content management, Internet of things, data storage and analysis, information infrastructure, datasets, technology and cloud computing. Download big data characteristics and process PPT graphics to make an impressive presentation. Develop greater goodwill with our Big Data Characteristics And Process PowerPoint Presentation Slides. Folks feel friendlier towards you.

Big Data Characteristics And Process PowerPoint Presentation Slides

SlideTeam

Azure data analytics platform - A reference architecture

Rajesh Kumar

Intro to Delta Lake

Databricks

1. The basic definition of Data, Analytics, and Data Analytics 2. Definition: Data: Data is a set of values of qualitative or quantitative variables. It is information in the raw or unorganized form. It may be a fact, figure, characters, symbols etc Analytics: Analytics is the discovery, interpretation, and communication of meaningful patterns in data and applying those patterns towards effective decision making. Data Analytics: Data analytics refers to qualitative and quantitative techniques and processes used to enhance productivity and business gain. 3.Types of analytics: Predictive Analytics (What could happen?) Prescriptive Analytics (What should we do) Descriptive Analytics (What has happened?) 4.Why Data analytics? Data Analytics is needed in Business to Consumer applications (B2C) 5.The process of Data analytics: Data requirements, Data collection, Data processing, Data cleaning, Exploratory data analysis, Modeling and algorithms, Data product, Communication 6.The scope of Data Analytics: Bright future of data analytics, many professionals and students are interested in a career in data analytics. 7.Importance of data analytics:1. Predict customer trends and behaviors Analyze, 2 interpret and deliver data in meaningful ways 3.Increase business productivity 4.Drive effective decision-making 8.why become a data analyst? talented gaps of skill candidates, good salaries for freshers, great future growth path 9. What recruiters look for in applicants: Problem-Solving Skills, Analytical Mind, Maths and Statistic Skills, Communication (both oral and written), Teamwork Abilities 10. Skill is required for Data analytics? 1.) Analytical Skills 2.) Numeracy Skills 3.) Technical and Computer Skills 4.) Attention to Details 5.) Business Skills 6.) Communication Skills 11. Data analytics tools 1.SAS: SAS (Statistical Analysis System) is a software suite developed by SAS Institute. sas language can be defined as a programming language in the computing field. This language is generally used for the purpose of statistical analysis. The language has the ability to read data from databases and common spreadsheets. 2. R: R is a programming language and software environment for statistical analysis, graphics representation and reporting.R is freely available under the GNU General Public License, and pre-compiled binary versions are provided for various operating systems like Linux, Windows, and Mac. 3.PYTHON: Python is a popular programming language Python is a powerful, flexible, open-sources language that is easy to use, and has a powerful library for data manipulation and analysis. 4.TABLEAU: Tableau Software is a software company that produces interactive data visualization products focused on business intelligence.

Data analytics presentation- Management career institute

PoojaPatidar11

Watch full webinar here: https://bit.ly/3CMqS0E Today, businesses have more data and data types combined with more complex ecosystems than they have ever had before. Examples include on-premise data marts, data warehouses, data lakes, applications, spreadsheets, IoT data, sensor data, unstructured, etc. combined with cloud data ecosystems like Snowflake, Big Query, Azure Synapse, Amazon S3, Redshift, Databricks, SaaS apps, such as Salesforce, Oracle, Service Now, Workday, and on and on. Data, Analytics, Data Science and Architecture teams are struggling to provide the business users with the right data as quickly and efficiently as possible to quickly enable Analytics, Dashboards, BI, Reports, etc. Unfortunately, many enterprises seek to meet this pressing need by utilizing antiquated and legacy 40+ year-old approaches. There is a better way. Proven by thousands of other companies. As Forrester so astutely reported in their recent Total Economic Impact Study, companies who employed Data Virtualization reported a “65% decrease in data delivery times over ETL” and an “83% reduction in time to new revenue.” Join us for this very educational webinar to learn firsthand from Denodo Technologies and Fusion Alliance how: - Data Virtualization helps your company save time and money by eliminating superfluous ETL pipelines and data replication. - Data Virtualization can become the cornerstone of your modern data approach to deliver data faster and more efficiently than old legacy approaches at enterprise scale. - How quickly and easily, Data Virtualization can scale, even in the most complex environments, to create a universal abstraction semantic model(s) for all of your cloud, on premise, structured, unstructured and hybrid data - Data Mesh and Data Fabric architecture patterns for maximum reuse - Other customers have used, and are using, Data Virtualization to tackle their toughest data integration and data delivery challenges - Fusion Alliance can help you define a data strategy tailored to your organization’s needs and requirements, and how they can help you achieve success and enable your business with self-service capabilities

Modernizing Integration with Data Virtualization

Denodo

"Combining Databricks, the unified analytics platform with Snowflake, the data warehouse built for the cloud is a powerful combo. Databricks offers the ability to process large amounts of data reliably, including developing scalable AI projects. Snowflake offers the elasticity of a cloud-based data warehouse that centralizes the access to data. Databricks brings the unparalleled utility of being based on a mature distributed big data processing and AI-enabled tool to the table, capable of integrating with nearly every technology, from message queues (e.g. Kafka) to databases (e.g. Snowflake) to object stores (e.g. S3) and AI tools (e.g. Tensorflow). Key Takeaways: How Databricks & Snowflake work; Why they're so powerful; How Databricks + Snowflake symbiotically catalyze analytics and AI initiatives"

Databricks + Snowflake: Catalyzing Data and AI Initiatives

Databricks

Today’s data-driven companies have a choice to make – where do we store our data? As the move to the cloud continues to be a driving factor, the choice becomes either the data warehouse (Snowflake et al) or the data lake (AWS S3 et al). There are pro’s and con’s for each approach. While the data warehouse will give you strong data management with analytics, they don’t do well with semi-structured and unstructured data with tightly coupled storage and compute, not to mention expensive vendor lock-in. On the other hand, data lakes allow you to store all kinds of data and are extremely affordable, but they’re only meant for storage and by themselves provide no direct value to an organization. Enter the Open Data Lakehouse, the next evolution of the data stack that gives you the openness and flexibility of the data lake with the key aspects of the data warehouse like management and transaction support. In this webinar, you’ll hear from Ali LeClerc who will discuss the data landscape and why many companies are moving to an open data lakehouse. Ali will share more perspective on how you should think about what fits best based on your use case and workloads, and how some real world customers are using Presto, a SQL query engine, to bring analytics to the data lakehouse.

Data Warehouse or Data Lake, Which Do I Choose?

DATAVERSITY

Big Data Analytics Strategy and Roadmap

Srinath Perera

Creating a clearly articulated data strategy—a roadmap of technology-driven capability investments prioritized to deliver value—helps ensure from the get-go that you are focusing on the right things, so that your work with data has a business impact. In this presentation, the experts at Silicon Valley Data Science share their approach for crafting an actionable and flexible data strategy to maximize business value.

8 Steps to Creating a Data Strategy

Silicon Valley Data Science

Big data analytics for telecom operators final use cases 0712-2014_prof_m erdas

Prof Dr Mehmed ERDAS

In this report, my focus was mainly on analysing the data and various factors affecting the trend of Movies and Shows available on Netflix. Data Visualization was a primary aim and was implemented using Plotly. Plotly's Python graphing library makes interactive, publication-quality graphs. This report describes the basic exploratory data analysis and the trend of movies and TV Shows on the different factors and some more visualization

Report: EDA of TV shows & movies available on Netflix

AnkitBirla5

The introductory morning session will discuss big data challenges and provide an overview of the AWS Big Data Platform. We will also cover: • How AWS customers leverage the platform to manage massive volumes of data from a variety of sources while containing costs. • Reference architectures for popular use cases, including: connected devices (IoT), log streaming, real-time intelligence, and analytics. • The AWS big data portfolio of services, including Amazon S3, Kinesis, DynamoDB, Elastic MapReduce (EMR) and Redshift. • The latest relational database engine, Amazon Aurora - a MySQL-compatible, highly-available relational database engine which provides up to five times better performance than MySQL at a price one-tenth the cost of a commercial database. • Amazon Machine Learning – the latest big data service from AWS provides visualization tools and wizards that guide you through the process of creating machine learning (ML) models without having to learn complex ML algorithms and technology.

The AWS Big Data Platform – Overview

Amazon Web Services

Developing a Data Strategy for your organization can seem like a daunting task – but it’s worth the effort. Getting your Data Strategy right can provide significant value, as data drives many of the key initiatives in today’s marketplace, from digital transformation to marketing, customer centricity, population health, and more. This webinar will help demystify Data Strategy and its relationship to Data Architecture and will provide concrete, practical ways to get started.

Building a Data Strategy – Practical Steps for Aligning with Business Goals

DATAVERSITY

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

Paris Data Engineers !

Many had dubbed 2020 as the decade of data. This is indeed an era of data zeitgeist. From code-centric software development 1.0, we are entering software development 2.0, a data-centric and data-driven approach, where data plays a central theme in our everyday lives. As the volume and variety of data garnered from myriad data sources continue to grow at an astronomical scale and as cloud computing offers cheap computing and data storage resources at scale, the data platforms have to match in their abilities to process, analyze, and visualize at scale and speed and with ease — this involves data paradigm shifts in processing and storing and in providing programming frameworks to developers to access and work with these data platforms. In this talk, we will survey some emerging technologies that address the challenges of data at scale, how these tools help data scientists and machine learning developers with their data tasks, why they scale, and how they facilitate the future data scientists to start quickly. In particular, we will examine in detail two open-source tools MLflow (for machine learning life cycle development) and Delta Lake (for reliable storage for structured and unstructured data). Other emerging tools such as Koalas help data scientists to do exploratory data analysis at scale in a language and framework they are familiar with as well as emerging data + AI trends in 2021. You will understand the challenges of machine learning model development at scale, why you need reliable and scalable storage, and what other open source tools are at your disposal to do data science and machine learning at scale.

The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...

Databricks

Cloudera - The Modern Platform for Analytics

Cloudera, Inc.

Lakehouse in Azure

Sergio Zenatti Filho

Mais procurados (20)

Cloud-native Semantic Layer on Data Lake

DATA ANALYTICS FOR SOLVING BUSINESS PROBLEMS

Data Driven Design

Big Data Characteristics And Process PowerPoint Presentation Slides

Azure data analytics platform - A reference architecture

Intro to Delta Lake

Data analytics presentation- Management career institute

Modernizing Integration with Data Virtualization

Databricks + Snowflake: Catalyzing Data and AI Initiatives

Data Warehouse or Data Lake, Which Do I Choose?

Big Data Analytics Strategy and Roadmap

8 Steps to Creating a Data Strategy

Big data analytics for telecom operators final use cases 0712-2014_prof_m erdas

Report: EDA of TV shows & movies available on Netflix

The AWS Big Data Platform – Overview

Building a Data Strategy – Practical Steps for Aligning with Business Goals

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...

Cloudera - The Modern Platform for Analytics

Lakehouse in Azure

Semelhante a Data-Driven @ Netflix

Abstract:- Netflix has a growing presence in Hollywood, with technical teams working on everything from high-speed video editing pipelines to machine learning methods for categorizing films. Data is foundational across these efforts and in this talk Josh will take a tour through why we invest so much in data about content, what data engineering challenges we tackle, and the style in which we do it.

Data engineering at the interface of art and analytics: the why, what, and ho...

Data Con LA

AI Solutions with Macnica.ai - AI Expo 2018 Tokyo Japan

Avkash Chauhan

This session will begin with an introduction to non-relational (NoSQL) databases and compare them with relational (SQL) databases. We will also explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service. Learn the fundamentals of DynamoDB and see the new DynamoDB console first-hand as we discuss common use cases and benefits of this high-performance key-value and JSON document store.

Getting Started with Amazon DynamoDB

Amazon Web Services

BigDL-enabled Deep Learning analysis of photos attached to property listings in Multiple Listings Services database allowed us to extract image features and identify similar-looking properties. We leveraged this information to in real-time property search application to improve the relevancy of user search results. Imagine identifying a property listing photo you like and having the system suggest other listings you should also review. Traditional real-estate MLS (multiple-listings services) search methods rely on SQL-type queries to search and serve real-estate listings results. However, using BigDL in conjunction with MLSLinstings standard APIs allows users to include photos as search parameters in real-time, based both on image similarities and semantic feature search. The information extracted from listing’s images is used to improve the relevancy of the search results. To enable this use-case, we implemented several CNNs using BigDL framework on Microsoft’s Azure hosted Apache Spark: – Image feature extraction and tagging. Extracts features from real estate images and classifies them according to Real Estates Standards Organization rules, such as overall house style, interior and exterior attributes, etc. – Image similarity network which allows for comparing images that belong to different properties based on their extracted features and create a similarity score to be used in search results. We’ll discuss the above networks in details as well as run a live demo of real-estate search results. Key takeaways: a) Why invest into Spark BigDL from the start. b) Why choose cloud-based solution from the start. c) Choice of Scala vs Python.

Using BigDL on Apache Spark to Improve the MLS Real Estate Search Experience ...

Databricks

[Public] 7 archetipi della tecnologia moderna [italy]

Nicolas Bortolotti

This was presented by the Yongsheng Wu, head of big data and ML platform at Pinterest, at the Alluxio bay area meetup. Yongsheng shares Pinterest's journey to build a fast and scalable big data and ML platform in AWS for Pinterest to handle the requests and complexity in data at scale. In this talk, he will cover different aspects from the requirements of the platform, the challenges encountered, the technologies chosen, and the tradeoffs that were made.

Pinterest - Big Data Machine Learning Platform at Pinterest

Alluxio, Inc.

Slides from JupyterCon 2018 in NYC on 8/23/2018. Notebooks have moved beyond a niche solution at Netflix; they are now the critical path for how everyone runs jobs against the company’s data platform. From creating original content to delivering bufferless streaming, Netflix relies on notebooks to inform decisions and fuel experiments across the company. Netflix also uses notebooks to power its machine learning infrastructure and run over 150,000 jobs against its 100 PB cloud-based data warehouse every day. The goal is to deliver a compelling notebooks experience that simplifies end-to-end workflows for every type of user. To enable this, Netflix is investing deeply in notebook infrastructure and open source projects such as nteract. In this talk, Michelle Ufford and Kyle Kelley share interesting ways Netflix uses data and some of the big bets the company is making on notebooks. Topics will include architecture, kernels, UIs, and Netflix’s open source collaborations with projects such as Jupyter, nteract, pandas, and Spark.

Notebooks @ Netflix: From analytics to engineering with Jupyter notebooks

Michelle Ufford

Big Data LDN 2017: How Big Data Insights Become Easily Accessible With Workfl...

Matt Stubbs

When you have to say ""YES"" to everything, what does that really mean? The Boston Consulting Group (BCG) is a global management consulting firm that advises industry leading companies on value creation strategies, innovation, transformation, supply chain management and much more. BCG provides recommendations and analytics that are custom tailored to the needs of each client. To do this BCG is transforming what used to be presentations and manual models into software that is prototyped and distributed to their client to run on their infrastructure - in effect changing the way BCG delivers value. However the productization process poses unique challenges and opportunities. Their clients have every type of infrastructure and application stack within their IT environments, how can BCG ensure that the custom built analytics applications will operate well at scale in their client's production environment - especially when it is an environment that they don't control? The bespoke nature of the BCG business led the team to embark on an engineering led journey to containerization with Docker. Attend this session to learn more about the approach, challenges and how BCG is enabling transformation with Docker Enterprise Edition.

The Complexity to "Yes" in Analytics Software and the Possibilities with Dock...

Docker, Inc.

At H2O.ai we see a world where all software will incorporate AI, and we’re focused on bringing AI to business through software. H2O.ai is the maker behind H2O, the leading open source machine and deep learning platform for smarter applications and data products. H2O operationalizes data science by developing and deploying algorithms and models for R, Python and the Sparkling Water API for Spark. In this webinar, you will learn about the scalable H2O core platform and the distributed algorithms it supports. H2O integrates seamlessly with the R and the Python environments. We will show you how to leverage the power of H2O algorithms in R, Python and H2O Flow interface. Come with an open mind and some high level knowledge of machine learning, and you will take away a stream of knowledge for your next ML/DL project. Amy Wang is a math hacker at H2O, as well as the Sales Engineering Lead. She graduated from Hunter College in NYC with a Masters in Applied Mathematics and Statistics with a heavy concentration on numerical analysis and financial mathematics. Her interest in applicable math eventually lead her to big data and finding the appropriate mediums for data analysis. Desmond is a Senior Director of Marketing at H2O.ai. In his 15+ years of career in Enterprise Software, Desmond worked in Distributed Systems, Storage, Virtualization, MPP databases, Streaming Analytics Platform, and most recently Machine Learning. He obtained his Master’s degree in Computer Science from Stanford University and MBA degree from UC Berkeley, Haas School of Business.

Start Getting Your Feet Wet in Open Source Machine and Deep Learning

Ian Gomez

There was a time when the Enterprise Data Warehouse (EDW) was the only way to provide a 360-degree analytical view of the business. In recent years many organizations have deployed disparate analytics alternatives to the EDW, including: cloud data warehouses, machine learning frameworks, graph databases, geospatial tools, and other technologies. Often these new deployments have resulted in the creation of analytical silos that are too complex to integrate, seriously limiting global insights and innovation. Join guest speaker, 451 Research’s Jim Curtis and Pivotal’s Jacque Istok for an interactive discussion about some of the overarching trends affecting the data warehousing market, as well as how to build a next generation data platform to accelerate business innovation. During this webinar you will learn: - The significance of a multi-cloud, infrastructure-agnostic analytics - What is working and what isn’t, when it comes to analytics integration - The importance of seamlessly integrating all your analytics in one platform - How to innovate faster, taking advantage of open source and agile software Speakers: James Curtis, Senior Analyst, Data Platforms & Analytics, 451 Research & Jacque Istok, Head of Data, Pivotal

Analytical Innovation: How to Build the Next Generation Data Platform

VMware Tanzu

Applying R in BI and Real Time applications EARL London 2015

Lou Bajuk

Today’s complex data is not only big, but also semi-structured and densely connected. In this session we’ll look at how size, structure and connectedness have converged to transform the data landscape. We’ll then go on to look at some of the new opportunities for creating end-user value that have emerged in a world of connected data, illustrated with practical examples drawn from the telecommunications, social media and logistics sectors.

New Opportunities for Connected Data - Emil Eifrem @ GraphConnect Boston + Ch...

Neo4j

Big Data LDN 2017: Big Impact with Big Data

Matt Stubbs

Applying the R Language to BI and Real Time Applications

Lou Bajuk

******** Abstract: ******** Not long ago the question was whether your organization had big data. Did you have the volume, the velocity, the technology. Now those basics are largely given for most of the people attending this event. The path to success is still fuzzy, however, with so many technologies to choose from – and so many ways to use them. This presentation triangulates in a holistic manner on the modern business dilemma: how can we leverage technology to improve revenue, profit, market share, and numerous other success criteria. That said, this is not about the analytics or KPIs -- although it is about measurable improvement. It’s about lining up the right technologies and using them in effective, proven ways to maximize Return on Investment (ROI). Since the slant here is holistic, we’ll show how to blend infrastructure, tools, methods, and talent to avoid and constantly trim technical debt… and to produce success stories that are consistently repeatable, not a byproduct of individual heroics.

Maximize Big Data ROI via Best of Breed Patterns and Practices

Jeff Bertman

TIBCO Jaspersoft® empowers millions of people every day to make faster business decisions by bringing them timely data via applications and business processes. Get the answers you need, with Jaspersoft. See how you can improve your reporting and analytics solution and get access to actionable data. Join us for one hour and watch how Jaspersoft can transform your business with reporting and analytics. Topics Covered: -Provide a general product overview of Jaspersoft BI -Showcase the broad capabilities of Jaspersoft including: dashboards, data visualization, ad-hoc reporting and production reporting -Demonstrate the user experience from an end-user perspective as well as a BI Builder -Conduct a Q & A session

JASPERSOFT LIVE DEMO - NAM

TIBCO Jaspersoft

Comparison of Data Preparation vs. Data Wrangling Programming Languages, Frameworks and Tools in Machine Learning / Deep Learning Projects. A key task to create appropriate analytic models in machine learning or deep learning is the integration and preparation of data sets from various sources like files, databases, big data storages, sensors or social networks. This step can take up to 80% of the whole project. This session compares different alternative techniques to prepare data, including extract-transform-load (ETL) batch processing (like Talend, Pentaho), streaming analytics ingestion (like Apache Storm, Flink, Apex, TIBCO StreamBase, IBM Streams, Software AG Apama), and data wrangling (DataWrangler, Trifacta) within visual analytics. Various options and their trade-offs are shown in live demos using different advanced analytics technologies and open source frameworks such as R, Python, Apache Hadoop, Spark, KNIME or RapidMiner. The session also discusses how this is related to visual analytics tools (like TIBCO Spotfire), and best practices for how the data scientist and business user should work together to build good analytic models. Key takeaways for the audience: - Learn various options for preparing data sets to build analytic models - Understand the pros and cons and the targeted persona for each option - See different technologies and open source frameworks for data preparation - Understand the relation to visual analytics and streaming analytics, and how these concepts are actually leveraged to build the analytic model after data preparation Video Recording / Screencast of this Slide Deck: https://youtu.be/2MR5UynQocs

Data Preparation vs. Inline Data Wrangling in Data Science and Machine Learning

Kai Wähner

The Briefing Room with Dean Abbott and Tableau Software Live Webcast July 23, 2013 http://www.insideanalysis.com Today’s desire for analytics extends well beyond the traditional domain of Business Intelligence. That’s partly because business users are realizing the value of mixing and matching all kinds of data, from all kinds of sources. One emerging market driver is Cloud-based data, and the desire companies have to analyze this data cohesively with their on-premise data sets. Register for this episode of The Briefing Room to learn from Analyst Dean Abbott, who will explain how the ability to access data in the cloud can play a critical role for generating business value from analytics. He’ll be briefed by Ellie Fields of Tableau Software who will tout Tableau’s latest release, which includes native connectors to cloud-based applications like Salesforce.com, Amazon Redshift, Google Analytics and BigQuery. She’ll also demonstrate how Tableau can combine cloud data with other data sources, including spreadsheets, databases, cubes and even Big Data.

Bridging the Gap: Analyzing Data in and Below the Cloud

Inside Analysis

I presented this at a 2014 Tableau Conference session with Albert Wong. Netflix relies on data to make decisions ranging from buying and recommending content, to improving the streaming experience on devices. This presentation shares our Big Data analytics architecture and the tools used to make data accessible throughout our business, focusing on how Tableau fits into our organization and why it aligns well with our culture.

DATA @ NFLX (Tableau Conference 2014 Presentation)

Blake Irvine

Semelhante a Data-Driven @ Netflix (20)

Data engineering at the interface of art and analytics: the why, what, and ho...

AI Solutions with Macnica.ai - AI Expo 2018 Tokyo Japan

Getting Started with Amazon DynamoDB

Using BigDL on Apache Spark to Improve the MLS Real Estate Search Experience ...

[Public] 7 archetipi della tecnologia moderna [italy]

Pinterest - Big Data Machine Learning Platform at Pinterest

Notebooks @ Netflix: From analytics to engineering with Jupyter notebooks

Big Data LDN 2017: How Big Data Insights Become Easily Accessible With Workfl...

The Complexity to "Yes" in Analytics Software and the Possibilities with Dock...

Start Getting Your Feet Wet in Open Source Machine and Deep Learning

Analytical Innovation: How to Build the Next Generation Data Platform

Applying R in BI and Real Time applications EARL London 2015

New Opportunities for Connected Data - Emil Eifrem @ GraphConnect Boston + Ch...

Big Data LDN 2017: Big Impact with Big Data

Applying the R Language to BI and Real Time Applications

Maximize Big Data ROI via Best of Breed Patterns and Practices

JASPERSOFT LIVE DEMO - NAM

Data Preparation vs. Inline Data Wrangling in Data Science and Machine Learning

Bridging the Gap: Analyzing Data in and Below the Cloud

DATA @ NFLX (Tableau Conference 2014 Presentation)

Último

Scaling API-first – The story of a global engineering organization

Radu Cotescu

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

How to convert PDF to text with Nanonets

naman860154

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Civil Lines Women Seeking Men

Delhi Call girls

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Presentation on how to chat with PDF using ChatGPT code interpreter

naman860154

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Delhi Call girls

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

Data-Driven @ Netflix

1. Data-Driven @ Netflix Michelle Ufford Principal Architect Data Engineering & Analytics

2. Michelle Ufford Highlights ● Principal Architect at Netflix Data Engineering & Analytics ● Prev. Engineering Manager at GoDaddy Data Platform ● Microsoft Data Platform MVP ● 10+ years building web-scale analytics & data engineering infrastructure ● advises on Big Data topics Microsoft, Hortonworks, Teradata, etc. Gratuitous picture of my kids

3. By the Numbers.

4. The business numbers. 86.7 million members 1000+ devices supported 125+ million hours watched launched 19 years ago every. day.

5. Any device. Anywhere.* * Well, almost anywhere.

6. The data numbers. 4 petabyte DW reads 300 terabyte DW writes 40 petabyte data warehouse 700+ billion events written

7. Data in Action.

8. Content. What should we license?

9. Predicting Value for Licensed Content. Feature Engineering Predictive Models License Terms Content Efficiency

10. Predicting Value for Licensed Content. ● past performance of similar content on Netflix ● broadcast & Box Office performance ● talent (writers, actors, directors, etc.) ● critic & user reviews ● awards & accolades Feature Engineering Predictive Models License Terms Content Efficiency

11. Predicting Value for Licensed Content. Feature Engineering Predictive Models License Terms Content Efficiency

12. Predicting Value for Licensed Content. ● terms (length, exclusivity, etc.) ● bid amount ● negotiations Feature Engineering Predictive Models License Terms Content Efficiency

13. Predicting Value for Licensed Content. ● value / cost ● if efficient, license Feature Engineering Predictive Models License Terms Content Efficiency

14. “ last year our original content overall was some of our most efficient content. ”

15. “ We are building a studio in the cloud and pioneering new approaches to movie production, optimizing pitches, production schedules, subtitling, and digital asset management for our Original content. ”

16.

17. What should we license?create?

18. Product UX. Data. Driven. Experience.

19.

20.

21.

22.

23.

24. There are 86 million different versions of Netflix.

25. billboard rows, row order titles, title order title artwork

26. Public Relations. Analytics of news.Analytics is news.

27.

28.

29. Content Delivery. Monitoring a global service.

30.

31. YouTube video of Vizceral demo https://youtu.be/JctsPpgEsVs

32. Behind the Scenes.

33. data access AWS S3 Big Data Platform Amazon Redshift data processing fast storage data viz METACA T data services events data operational data elastic storage Apache Pig

34. Philosophy.

35.

36. Freedom & Responsibility.

37.

38. Context, Not Control.

39.

40.

41. Highly Aligned & Loosely Coupled.

42. Big Data Platform

43. Data Engineering & Analytics MarketingProduct PlaybackContent Finance 105 talented engineers & analysts data viz engineers analytics engineers data engineers Big Data Platform analysts

44. Results, Not Opinions.

45.

46.

47.

48. Experimentation Platform Batch & Ad Hoc Analysis

49. Questions?

50. Thank you for attending! Michelle Ufford linkedin.com/in/mufford @sqlfool Data @ Netflix @NetflixData hadoopsie.com techblog.netflix.com tinyurl.com/NetflixData

Notas do Editor

Abstract: Netflix is the quintessential data-driven company. It’s 83 million members stream more than 125 million hours in over 190 countries every day and generate more than 700 billion events in the process. In this session, we’ll share how data is used to make informed decisions across the entire business — from content acquisition to content delivery, and everything in between. We’ll look at how Netflix successfully employs a scalable cloud-based data platform to support a constant deluge of data and a small army of data analysts, engineers, and scientists. We’ll discuss the advanced analytical capabilities that are enabled through modern data technologies. Lastly, we’ll explore some of the architectural & operational principals that enable Netflix to so effectively make use of its data.
Obligatory “why should you listen to me talk?” slide
Numbers as of Q3 2016
During CES 2016 this January, ‘flipped the switch’ making Netflix available in 130+ new countries. Netflix is presently available in over 190 countries worldwide.
What content should we license? How much should we bid? How should we value exclusivity? How should we measure content performance?
Originals content: 2015 - 450 hours 2016 - 600 hours 2017 - 1000 hours
Netflix website: circa 2012
Netflix website: circa 2013
Netflix website: circa 2014
Netflix website: circa 2015
Netflix website: circa 2016
Vizceral Open-Source Project: https://github.com/netflix/vizceral http://techblog.netflix.com/2016/08/vizceral-open-source.html http://techblog.netflix.com/2015/10/flux-new-approach-to-system-intuition.html
Genie – federated job execution engine Metacat – federated metadata service Kragle – python APIs
15m views on SlideShare
Minimize rules Make smart choices Take ownership
Avoid prescriptive requirements Give visibility
Set context (strategy, goals) Communicate only as much as needed
At Netflix, we use the scientific method We’re often right at predicting behavior – for people exactly like us Most people aren’t like us

Data-Driven @ Netflix

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Data-Driven @ Netflix

Semelhante a Data-Driven @ Netflix (20)

Último

Último (20)

Data-Driven @ Netflix

Notas do Editor