Winning the metrics battle

•

3 gostaram•5,024 visualizações

The slides from a presentation at Velocity Europe 2012 talk about how the Guardian does metrics an monitoring. The original proposal is at http://velocityconf.com/velocityeu2012/public/schedule/detail/26576 and there is also an article about it at http://www.guardian.co.uk/info/developer-blog/2012/oct/04/winning-the-metrics-battle

Tecnologia

Winning the metrics battle
(ﬁnally)
Simon Hildrew Nick Satterly
Infrastructure Developer Monitoring Engineer
The Guardian The Guardian

Total metrics

180,000

50,000

1,400 2,800

http://www.ﬂickr.com/photos/ghostsigns/6676069121

5 minutes

every 15
seconds

http://www.ﬂickr.com/photos/millynet/134071210

Physical screens Screensaver hacks
20

15

10

5

0

http://www.ﬂickr.com/photos/chrisjames_taylor/5454315456

our approach
Side project ➡ Prioritise
Incremental upgrade ➡ Understand the real problem
Use off the shelf tool ➡ Question the tools
Pragmatic solution ➡ Be ambitious
Done in a year ➡ Keep learning

drowning in work

http://www.ﬂickr.com/photos/iampeas/246738971

a dedicated monitoring and
metrics engineer

metrics were not helping us
solve production outages

but... difﬁcult to instrument applications

T.T. Detect
+
T.T. Fix = T.T. Diagnose
+
T.T. Resolve

inaccessible tools

http://www.ﬂickr.com/photos/kdashy/2678539087

inconsistent data

http://www.ﬂickr.com/photos/sybrenstuvel/2468506922

hypothesising & arguing
easier than measuring

http://www.ﬂickr.com/photos/nouqraz/200049988

The ‘right’ thing
• measure everything
• measure frequently
• measure each data point once
• input and output must be open

Brute force?

http://www.ﬂickr.com/photos/epublicist/3546059144

The safe option?

http://www.ﬂickr.com/photos/alicebartlett/2361209195

Unintuitive?

http://www.ﬂickr.com/photos/merlijnhoek/2841785343

Imposing a ﬂawed model?
http://www.ﬂickr.com/photos/evansville/8953838/

Too difﬁcult / no progress?
http://www.ﬂickr.com/photos/ginja_andy/4165849136/

Nagios

• the “IBM” of monitoring tools

• compromise over quantity and frequency of checks

• < insert your criticism of nagios here >

Zabbix

• metric collection tightly coupled to monitoring tool

• confusing UI with poor visualisation

• needed brute force to make limited API work

http://www.ﬂickr.com/photos/mugley/2961131550

Throw work away

http://www.ﬂickr.com/photos/sk8geek/7358702704

Get as far as you can

screens users
db? alerting?

Etsy dashboard
message queue

graphite SNMP? syslog?

FITB ganglia api?

network hosts applications

Develop missing pieces

http://www.ﬂickr.com/photos/kalexanderson/5969012589

screens users
mongodb alerta elastic
search

Etsy dashboard
message queue

syslog SNMP
graphite ganglia alerts
alerts alerts

FITB ganglia ganglia-api

network hosts applications

Guardian Management
https://github.com/guardian/guardian-management

Ganglia API
https://github.com/guardian/ganglia-api

rescale image???

Alerta
https://github.com/guardian/alerta

Current stack
• Ganglia • Guardian management
https://github.com/guardian/guardian-management

• FITB • Guardian ganglia-api
https://github.com/guardian/ganglia-api
• Graphite
• Guardian alerta
• Etsy dashboards https://github.com/guardian/alerta

➡ Prioritise
➡ Understand the real problem
➡ Question the tools
➡ Be ambitious
➡ Keep learning

Thank you
http://github.com/guardian
http://gu.com/p/3ap5f
Simon Hildrew Nick Satterly
@sihil @nicksatterly
simon.hildrew@guardian.co.uk nick.satterly@guardian.co.uk

Mais conteúdo relacionado

Semelhante a Winning the metrics battle

Monitoring the #DevOps way

Theo Schlossnagle

Every day, millions of advertising campaigns are happening around the world. As campaign owners, measuring the ongoing campaign effectiveness (e.g “how many distinct users saw my online ad VS how many distinct users saw my online ad, clicked it and purchased my product?”) is super important. However, this task (often referred to as “funnel analysis”) is not an easy task, especially if the chronological order of events matters. One way to mitigate this challenge is combining Apache Druid and Apache DataSketches, to provide fast analytics on large volumes of data. However, while that combination can answer some of these questions, it still can’t answer the question “how many distinct users viewed the brand’s homepage FIRST and THEN viewed product X page?” In this talk, we will discuss how we combine Spark, Druid and DataSketches to answer such questions at scale.

Funnel Analysis with Apache Spark and Druid

Databricks

Crossing the Production Barrier: Development at Scale

jgoulah

Google Cloud: Next'19 Extended Hanoi

GCPUserGroupVietnam

Move out from AppEngine, and Python PaaS alternatives

tzang ms

Google Wave: Ripple or Tsunami for Research

Cameron Neylon

InfoSec analysts are all somewhat familiar with Honeypots. When they are given the proper attention, care and feeding, they produce invaluable information and can be a critical asset when it comes to defending the network. This intel has been primarily used by security researchers and organizations with advanced defensive capabilities to study their adversaries and learn from their actions. But what about the rest of us? Honeypots are a lot of work to configure, maintain, and monitor, right? Not exactly; when deployed and monitored properly, Honeypots and Honey Tokens are a simple way to alert on anomalous activity inside the network. But how can an organization that is not focused on research gain valuable threat intelligence using Honeypots and actively defend their network using indicators generated from an internal Honeynet? The answer is Honeypots for Active Defense. There are currently many open source security tool distributions that come pre-loaded with Honeypots among other useful tools, however the Honeypot software is often not deployed in an effective manner. This session will discuss techniques to leverage Honeypots in ways that will not overburden the security team with massive logs to sift through and focuses efforts on correlating active threat data observed in the Honeypots with the production environment. When deploying Honeypots effectively, this can give security analysts one additional mechanism to tip them off to nefarious activity within their network before they become the next headline.

Honeypots for Active Defense

Greg Foss

This slide deck goes into detail about the main concepts in Hyperledger Fabric - peers, orderer, certificate authority, and channels. We talk about chaincode, and go into details about modeling a network which involves putting cars on an auction. I end with a demo of the IBM Blockchain Platform, and give you links to get a free 1-month trial of the Starter Plan Blockchain service on the IBM Blockchain Platform.

Hyperleger Fabric Workshop - Denver Blockchain Week

Horea Porutiu

Performance - a challenging craft

Fabian Lange

Build and Host Real-world Machine Learning Services from Scratch @ pycontw2019

Chun-Yu Tseng

Ultimate Git Workflow - Seoul 2015

Atlassian 대한민국

According to the very recent research by global technology company Nuix, most of the hackers (88%) can compromise the system in 12 hours and exfiltrate in another 12 hours (81% respondents). Is the situation really so bad? And what about Security Operation Centres, could not they make the bad guys stop? Let’s have a quick look into the hacker’s arsenal and examine SOC defences. Modern cyber warfare is really brilliant, introducing such weapons as gargantuan size DDoS attacks & infected smart devices, defences augmented with “next generation” security devices, Big Data Analytics and Threat Intelligence. Finally, the defenders have new soldier on the battlefield, the Space Marine of cybersecurity - Threat Hunter. What can possibly go wrong? Why are the breaches still happening? Is SOC really failing miserably this war? Let’s try to find the answers.

CONFidence 2017: Hackers vs SOC - 12 hours to break in, 250 days to detect (G...

PROIDEA

Co Speaker: Cheryl Biswas Talk Description: How about this: a blue team talk given by red teamers. But here’s our rationale - your best defence right now is a strategic offence. The rules of the game have changed and we need to get defence up to speed. We’ll show you what the key elements are in a good defence strategy; what you can and need to be using to full advantage. We’ll talk about the new “buzzwords” and how they apply: visibility; patterns; big data. There’s a whole lotta data to wrangle, and you aren’t seeing the whole picture if you aren’t doing things right. Threat intel is about getting the big picture as it applies to you. You’ll learn the importance of context and prioritization so that you can manipulate intel feeds to do your bidding. And then we’ll take things further and talk about hunting the adversary, using an update on proven methodologies. We’ll show you how to understand your data, correlate threats and pin point attacks. Attendees will leave with a new understanding of the resources they have on hand, and how to leverage those into an Adaptive Proactive Defense Strategy.

Blue team reboot - HackFest

Haydn Johnson

In this talk we’ll present the technology behind the Fully Automated Store by Checkout Technologies. The actual version of the store is a result of the work of 12 engineers that spans the areas from hardware and design to the ultimate deep learning architectures. Will be also discussed the challenges and lessons learnt during this adventure and what it means to deploy the system which has an AI engine in its core. Creation of the dataset and the invention of the specific metrics that is capable to measure the accuracy of the entire system will be discussed.

How to fully automate a store.pptx

Igor Moiseev

Introduzione alle metodologie di sviluppo agile

Stefano Valle

Accessibility and web innovation. (no notes)

Christian Heilmann

Using Blockchain to Increase Supply Chain Transparency

Horea Porutiu

Adoption of AI: The Great Opportunities for Everyone

Kan Ouivirach, Ph.D.

AB Testing, Ads and other 3rd party tags - London WebPerf - March 2018

Andy Davies

A micro frontend architecture is an approach to developing web applications as a composition of small frontend apps. Instead of writing a large monolith frontend application, the application can be broken down into domain-specific micro frontends, which are self-contained and can be developed and deployed independently. Each micro frontend can be owned by verticalised domain specific teams. Micro frontends do have advantages and disadvantages and they are not suitable in many cases. When micro frontends is a potential solution, make sure to use a domain driven top-down approach, with no big design upfronts. Keep it simple! Frontend Developer Love Amsterdam - 30 January 2018

Micro Frontends

Spyros Ioakeimidis

Semelhante a Winning the metrics battle (20)

Monitoring the #DevOps way

Funnel Analysis with Apache Spark and Druid

Crossing the Production Barrier: Development at Scale

Google Cloud: Next'19 Extended Hanoi

Move out from AppEngine, and Python PaaS alternatives

Google Wave: Ripple or Tsunami for Research

Honeypots for Active Defense

Hyperleger Fabric Workshop - Denver Blockchain Week

Performance - a challenging craft

Build and Host Real-world Machine Learning Services from Scratch @ pycontw2019

Ultimate Git Workflow - Seoul 2015

CONFidence 2017: Hackers vs SOC - 12 hours to break in, 250 days to detect (G...

Blue team reboot - HackFest

How to fully automate a store.pptx

Introduzione alle metodologie di sviluppo agile

Accessibility and web innovation. (no notes)

Using Blockchain to Increase Supply Chain Transparency

Adoption of AI: The Great Opportunities for Everyone

AB Testing, Ads and other 3rd party tags - London WebPerf - March 2018

Micro Frontends

Último

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Civil Lines Women Seeking Men

Delhi Call girls

08448380779 Call Girls In Friends Colony Women Seeking Men

Delhi Call girls

Presentation on how to chat with PDF using ChatGPT code interpreter

naman860154

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Evaluating the top large language models.pdf

ChristopherTHyatt

How to convert PDF to text with Nanonets

naman860154

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

Scaling API-first – The story of a global engineering organization

Radu Cotescu

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

Winning the metrics battle

1. Winning the metrics battle (ﬁnally)

2. Winning the metrics battle (ﬁnally) Simon Hildrew Nick Satterly Infrastructure Developer Monitoring Engineer The Guardian The Guardian

4. The metrics battleﬁeld

5. Total metrics 180,000 50,000 1,400 2,800

6. http://www.ﬂickr.com/photos/ghostsigns/6676069121 5 minutes every 15 seconds http://www.ﬂickr.com/photos/millynet/134071210

7. developer dashboards

8. Physical screens Screensaver hacks 20 15 10 5 0

9. dev hack

10. business dashboards

11. metrics + dashboards = culture change

12. http://www.ﬂickr.com/photos/chrisjames_taylor/5454315456

13. our approach Side project ➡ Prioritise Incremental upgrade ➡ Understand the real problem Use off the shelf tool ➡ Question the tools Pragmatic solution ➡ Be ambitious Done in a year ➡ Keep learning

14. Prioritise

15. drowning in work http://www.ﬂickr.com/photos/iampeas/246738971

16. a dedicated monitoring and metrics engineer

17. Understand the real problem

18. Urgent issue - current tool end of life

19. The story so far...

20. metrics were not helping us solve production outages

21. ballooning number of applications

22. but... difﬁcult to instrument applications

23. T.T. Detect + T.T. Fix = T.T. Diagnose + T.T. Resolve

24. inaccessible tools http://www.ﬂickr.com/photos/kdashy/2678539087

25. inconsistent data http://www.ﬂickr.com/photos/sybrenstuvel/2468506922

26. hypothesising & arguing easier than measuring http://www.ﬂickr.com/photos/nouqraz/200049988

27. The ‘right’ thing • measure everything • measure frequently • measure each data point once • input and output must be open

28. Question the tools

29. Brute force? http://www.ﬂickr.com/photos/epublicist/3546059144

30. The safe option? http://www.ﬂickr.com/photos/alicebartlett/2361209195

31. Unintuitive? http://www.ﬂickr.com/photos/merlijnhoek/2841785343

32. Imposing a ﬂawed model? http://www.ﬂickr.com/photos/evansville/8953838/

33. Too difﬁcult / no progress? http://www.ﬂickr.com/photos/ginja_andy/4165849136/

34. Nagios • the “IBM” of monitoring tools • compromise over quantity and frequency of checks • < insert your criticism of nagios here >

35. Zabbix • metric collection tightly coupled to monitoring tool • confusing UI with poor visualisation • needed brute force to make limited API work

36. The ‘right’ thing • measure everything • measure frequently • measure each data point once • input and output must be open

37.

38. don’t compromise

39. Be ambitious

40. http://www.ﬂickr.com/photos/mugley/2961131550 Throw work away

41. Draw your dream

42. http://www.ﬂickr.com/photos/sk8geek/7358702704 Get as far as you can

43. screens users db? alerting? Etsy dashboard message queue graphite SNMP? syslog? FITB ganglia api? network hosts applications

44. Develop missing pieces http://www.ﬂickr.com/photos/kalexanderson/5969012589

45. screens users mongodb alerta elastic search Etsy dashboard message queue syslog SNMP graphite ganglia alerts alerts alerts FITB ganglia ganglia-api network hosts applications

46. Guardian Management https://github.com/guardian/guardian-management

47. Ganglia API https://github.com/guardian/ganglia-api

48. rescale image??? Alerta https://github.com/guardian/alerta

49. Current stack • Ganglia • Guardian management https://github.com/guardian/guardian-management • FITB • Guardian ganglia-api https://github.com/guardian/ganglia-api • Graphite • Guardian alerta • Etsy dashboards https://github.com/guardian/alerta

50. Keep learning

51. we are not there yet

52. Watch the cultural changes

53. detecting

54. diagnosis

55. diagnosis

56. performance testing

57. conﬁrmation

58. #monitoringsucks

59. ➡ Prioritise ➡ Understand the real problem ➡ Question the tools ➡ Be ambitious ➡ Keep learning

60. tools can change culture

61. Thank you http://github.com/guardian http://gu.com/p/3ap5f Simon Hildrew Nick Satterly @sihil @nicksatterly simon.hildrew@guardian.co.uk nick.satterly@guardian.co.uk

Winning the metrics battle

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Winning the metrics battle

Semelhante a Winning the metrics battle (20)

Último

Último (20)

Winning the metrics battle