Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (20)
Semelhante a Machine Learning + Analytics
Semelhante a Machine Learning + Analytics (20)
Mais de Splunk
Mais de Splunk (20)
Último
Último (20)
Machine Learning + Analytics
- 4. 4 Humans are good at learning, but we get lost in volume and details…
- 5. 5 Why do we need Machine Learning? - Improve Decision Making - Forecast or Predict KPIs - Alert on Deviation - Uncover hidden trends or relationships All of this requires Diverse Data from across Many Silos. Lots of Unstructured, Real Time Data.
- 6. 6 Run the Business in Real-time Data From the Past Real-time Data Statistical Forecast T – a few days T + a few days Security Operations Center IT Operations Center Business Operations Center Predictive (Models) Descriptive (BI Tools, Data Lakes) Grey space
- 8. 8 ML 101: What is Machine Learning? What: “Field of study that gives computers the ability to learn without being explicitly programmed” – A. Samuel, 1959 How: Generalizing (learning) from examples (data) Simple ML workflow: – EXPLORE data – FIT models based on data – APPLY models in production – VALIDATE models – REPEAT
- 9. 9 Learning From Data [Prediction] • When we see thick clouds and an overcast sky, we predict that it’s likely going to rain [Estimation/ Regression] • Estimate how much an apartment costs based on its location, condition and prices of properties in that neighborhood [Classification/ Clustering] • Determine the gender of a person based on her/his features, hair style and the way s/he dresses [Anomaly Detection] • Identify the odd one out [Reinforcement Learning] • If I made a mistake this time, can I do better next time? All of us have had some experience in learning. But… what’s behind our experience? How do we translate that knowledge to code?
- 10. 10 Major Types of Machine Learning 1. Supervised Learning: generalizing from labeled data
- 11. 11 Major Types of Machine Learning 2. Unsupervised Learning: generalizing from unlabeled data ?
- 12. 12 3. Reinforcement Learning: • System is rewarded (or punished) based on the outcomes it generates • Action leads to a change in the state of the world and generates an error score Major Types of Machine Learning
- 16. 16 Splunk Machine Learning Toolkit Assistants: Guide model building, testing, & deploying for common objectives Showcases: Interactive examples for typical IT, security, business, IoT use cases Algorithms: 25+ standard algorithms available prepackaged with the toolkit SPL ML Commands: New commands to fit, test and operationalize models Python for Scientific Computing Library: 300+ open source algorithms available for use Build custom analytics for any use case Extends Splunk platform functions and provides a guided modeling environment
- 19. 19 Summary: The ML Process Problem: <Stuff in the world> causes big time & money expense. Value Hypothesis Solution: Build ML model to forecast <possible incidents>, act pre-emptively & learn Operationalize 1. Get all the rellevant data to the problem; Explore the data 2. Select and Fit an algorithem on the data, generating a model 3. Apply & Validate models until predictions solve the problem 4. Surface the model to X Ops, who consume the model to solve the problem
- 24. 24 Splunk: Data Fabric 2 OT Industrial Assets IT Consumer and Mobile Devices Real Time IT users Analysts Business Users Ad Hoc Search Custom Dashboards Monitor and Alert Reports/ Analyze Clickstreams HadoopDevices Networks GPS/ Cellular Online Shopping Carts Servers Applications Analysts Business Users Data Warehouses Structured Data Sources CRM ERP HR Billing Product Finance DB Connect Look-ups ODBC SDK API
- 27. 27 ML Is All Around You! Recall: EXPLORE > FIT > APPLY > VALIDATE > REPEAT • Face detection: find faces in images • Spam filtering: identify SPAM messages • Shopping Recommendations: predict what customers would like to buy • Fraud detection: identify credit card transactions which may be fraudulent in nature • Weather forecast: predict whether or not it will rain tomorrow; estimate daily max/min
- 31. 31 Reliable website updates Proactive website monitoring leads to reduced downtime 3 “Splunk ML helps us rapidly improve end-user experience by ranking issue severity which helps us determine root causes faster thus reducing MTTR and improving SLA” • Very frequent code and config updates (1000+ daily) can cause site issues • Find errors in server pools, then prioritize actions and predict root cause • Custom outlier detection built using ML Toolkit Outlier assistant • Built by Splunk Architect with no Data Science background ML Use Case Technical overview
- 33. 33 Next Steps with Splunk ML • Reach out to your Tech Team! We can help architect ML workflows. • Lots of ML commands in Core Splunk (predict, anomalydetection, stats) • ML Toolkit & Showcase – available and free, ready to use • Splunk ITSI: Applied ML for ITOA use cases – Manage 1000s of KPIs & alerts – Adaptive Thresholding & Anomaly Detection • Splunk UBA: Applied ML for Security – Unsupervised learning of Users & Entities – Surfaces Anomalies & Threats • ML Customer Advisory Program: – Connect with Product & Engineering teams - mlprogram@splunk.com
- 34. 34 What Else? • Get the Machine Learning Toolkit from Splunkbase • Go watch Machine Learning Videos on Splunk Youtube Channel http://tiny.cc/splunkmlvideos • Go watch the Machine Learnings talks from Conf 2016: – Advanced Machine Learning in SPL with the Machine Learning Toolkit by Jacob Leverich – Extending SPL with Custom Search Commands and the Splunk SDK for Python by Jacob Leverich • Early Adopter And Customer Advisory Program : mlprogram@splunk.com • Field ML Architects : Andrew Stein (astein@), Brian Nash (bnash@)
- 36. 36 What’s New since our 0.9 Beta Release (last year’s .conf)? 3 • New name and abbreviation ;-) • No event limits (removal of 50K limit on fitting models) • Configurable resource caps via mlspl.conf • Search head clustering support • Distributed / streaming apply • Scheduled fit • New algorithms (next slide) – Feature engineering and selection – Stochastic gradient descent (e.g.) – ARIMA • Multi-algorithm support across Assistants • Scatterplot matrix viz • Alerting • Tooltips • In-app tours • Cluster Numeric Events assistant • Videos videos videos for each assistant across IT, Security, IoT and Business Analytics • ML-SPL Cheat Sheet