Playing with Xbox Data with Dale Lane

HostedbyConfluentSeguir

2 de Jun de 2023•0 gostou•393 visualizações

Playing with Xbox Data with Dale Lane

2 de Jun de 2023•0 gostou•393 visualizações

Baixar para ler offline

Tecnologia

"A fun introduction to the world of Kafka Connect and Kafka Streams by using it to process data from Xbox Live. Gaming is a social activity, so Xbox includes a social aspect. Details about what games you play and when you’re playing are shared with your friends on the Xbox Live service. Achievements that you earn and complete in games are shared with your friends. This makes for a fun source of data, that we can play with using Kafka Connect and Kafka Streams. 1) Getting the data into Kafka — I’ll show how I’m using Kafka Connect to surface data from Xbox REST APIs as streams of events on Kafka topics. 2) Processing the events — I’ll show how I’m using Kafka Streams to process the Xbox events as they’re emitted - such as cleaning up some of the noisy events with filters, enriching events with additional context using transformations, and creating gamerscore leaderboards using aggregations. As a project, it is just a bit of fun. But it is a useful exercise for illustrating how to get started with Kafka Connect and Kafka Streams, and a way to demonstrate the principles using a real source of real events, rather than using fake or generated events. Plus it is a good excuse for me to spend more time gaming, and justifying it as work!"

HostedbyConfluentSeguir

Mais conteúdo relacionado

Similar a Playing with Xbox Data with Dale Lane

Akka Streams And Kafka Streams: Where Microservices Meet Fast DataLightbend6.5K visualizações•74 slides

A 7 architecture sustaining line liveLINE Corporation13.6K visualizações•56 slides

What is Apache Kafka and What is an Event Streaming Platform?confluent2.8K visualizações•63 slides

Moving from Big Data to Fast Data? Here's How To Pick The Right Streaming EngineLightbend4.5K visualizações•35 slides

Apache Spark Structured Streaming + Apache Kafka = ♡Bartosz Konieczny505 visualizações•34 slides

Big Data LDN 2018: STREAMING DATA MICROSERVICES WITH AKKA STREAMS, KAFKA STRE...Matt Stubbs374 visualizações•51 slides

Similar a Playing with Xbox Data with Dale Lane(20)

Akka Streams And Kafka Streams: Where Microservices Meet Fast Data

Lightbend•6.5K visualizações

A 7 architecture sustaining line live

LINE Corporation•13.6K visualizações

What is Apache Kafka and What is an Event Streaming Platform?

confluent•2.8K visualizações

Moving from Big Data to Fast Data? Here's How To Pick The Right Streaming Engine

Lightbend•4.5K visualizações

Apache Spark Structured Streaming + Apache Kafka = ♡

Bartosz Konieczny•505 visualizações

Big Data LDN 2018: STREAMING DATA MICROSERVICES WITH AKKA STREAMS, KAFKA STRE...

Matt Stubbs•374 visualizações

KSQL Deep Dive - The Open Source Streaming Engine for Apache Kafka

Kai Wähner•4.8K visualizações

Battle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDB

Jesse Wolgamott•9.7K visualizações

Paris jug ksql - 2018-06-28

Florent Ramiere•713 visualizações

Kick Your Database to the Curb

Bill Bejeck•44 visualizações

扩展世界上最大的图片Blog社区

yiditushe•581 visualizações

Fotolog: Scaling the World's Largest Photo Blogging Community

farhan "Frank" mashraqi•2.7K visualizações

Application Development and Data Modeling on Amazon DynamoDB

Amazon Web Services Japan•2.7K visualizações

MLOps with a Feature Store: Filling the Gap in ML Infrastructure

Data Science Milan•474 visualizações

KSQL – An Open Source Streaming Engine for Apache Kafka

Kai Wähner•2.5K visualizações

Building Stream Processing Applications with Apache Kafka Using KSQL (Robin M...

confluent•2.1K visualizações

Tableau Architecture

Kishore Chaganti•1.6K visualizações

Hands On With Spark: Creating A Fast Data Pipeline With Structured Streaming ...

Lightbend•18.9K visualizações

Leveraging Kafka Eco-System For Redis Streams: Sripathi Krishnan

Redis Labs•276 visualizações

Riviera Jug - 20/03/2018 - KSQL

Florent Ramiere•224 visualizações

Mais de HostedbyConfluent

Chill, Distill, No Overkill: Best Practices to Stress Test Kafka with Siva Ku...HostedbyConfluent35 visualizações•53 slides

Manage Consistent Configurations Across Multiple Kafka Environments with Naga...HostedbyConfluent15 visualizações•8 slides

The Cost of Kafka’s High Availability on Cloud with Geetha AnneHostedbyConfluent17 visualizações•13 slides

Case-Study: Building Real-Time Applications at Scale-Cyclist Crash Detection ...HostedbyConfluent13 visualizações•37 slides

Migrating Your System to the Different Kafka Platforms Available wtih Bhavesh...HostedbyConfluent17 visualizações•20 slides

Eliminating the Double Write Problem in Apache Kafka Using the Outbox Pattern...HostedbyConfluent155 visualizações•13 slides

Mais de HostedbyConfluent(20)

Chill, Distill, No Overkill: Best Practices to Stress Test Kafka with Siva Ku...

HostedbyConfluent•35 visualizações

Manage Consistent Configurations Across Multiple Kafka Environments with Naga...

HostedbyConfluent•15 visualizações

The Cost of Kafka’s High Availability on Cloud with Geetha Anne

HostedbyConfluent•17 visualizações

Case-Study: Building Real-Time Applications at Scale-Cyclist Crash Detection ...

HostedbyConfluent•13 visualizações

Migrating Your System to the Different Kafka Platforms Available wtih Bhavesh...

HostedbyConfluent•17 visualizações

Eliminating the Double Write Problem in Apache Kafka Using the Outbox Pattern...

HostedbyConfluent•155 visualizações

Safeguarding - Protecting Your Kafka from Misbehaving Clients with Tom Scott

HostedbyConfluent•152 visualizações

Intelligent, Automatic Restarts for Unhealthy Kafka Consumers on Kubernetes w...

HostedbyConfluent•186 visualizações

Bench, a Framework for Benchmarking Kafka Using K8s and OpenMessaging Benchma...

HostedbyConfluent•187 visualizações

Balance Kafka Cluster with Zero Data Movement with Haochen Li & Yaodong Yang

HostedbyConfluent•517 visualizações

Memory Matters: Drift Detection with a Low Memory Footprint for ML Models on ...

HostedbyConfluent•162 visualizações

Deep Dive into Kafka Connect Protocol with Catalin Pop

HostedbyConfluent•154 visualizações

Testing SMTs? Testcontainers to the Rescue! with Fábio Sequeira & Mafalda Santos

HostedbyConfluent•105 visualizações

What to do if Your Kafka Streams App Gets OOMKilled? with Andrey Serebryanskiy

HostedbyConfluent•489 visualizações

Apache Flink on Kafka: Reliable Data Pipelines Everyone Can Code with Ela Demir

HostedbyConfluent•201 visualizações

Learnings From Shipping 1000+ Streaming Data Pipelines To Production with Hak...

HostedbyConfluent•112 visualizações

Unveiling the Inner Workings of Apache Kafka® with Flamegraphs with Christo L...

HostedbyConfluent•343 visualizações

Storage Capacity Management on Multi-tenant Kafka Cluster with Nurettin Omeroglu

HostedbyConfluent•239 visualizações

Powering Consistent, High-throughput, Real-time Distributed Calculation Engin...

HostedbyConfluent•454 visualizações

Don’t Let Kafka Be A Cluster: Kafka Chaos Experimentation with Justin Fetherolf

HostedbyConfluent•99 visualizações

Último

Improving Employee Experiences on Cisco RoomOS Devices, Webex, and Microsoft ...ThousandEyes77 visualizações•25 slides

The Ultimate Administrator’s Guide to HCL Nomad Webpanagenda58 visualizações•71 slides

Roottoo Innovation V24_CP.pdfroottooinnovation33 visualizações•13 slides

Accelerating Data Science through Feature Platform, Transformers, and GenAIFeatureByte158 visualizações•46 slides

Privacy in the era of quantum computersSpeck&Tech22 visualizações•37 slides

How to use the Cataloguing Code Ethics at your day job : a hands-on workshop ...CILIP MDG38 visualizações•13 slides