Basic Ops Skills and Tools for Beginners

•

4 gostaram•1,806 visualizações

Your datastore, MongoDB, is likely critical to your project; if you're new to MongoDB or Ops and have to keep things running in production this is the talk for you. We will cover Ops with MongoDB from the ground up; which parts of servers you should monitor and why, and show how to take and test backups. Also, we'll run through different tools you can use for installing, running, and monitoring MongoDB. After this you should be comfortable with the basics of running MongoDB in production, as well as have a head start on what you should do next.

Tecnologia

rainforest @rainforestqa
Ops Skills and Tools for
Beginners

Russell Smith
!
English guy
Mongo DB Master
Ex Ops consultant
Ruby / PHP developer
YC S12 Alum
Using mongo in Prod since 1.6/1.8

@rainforestqarainforest
Rainforest QA
!
Insanely simple testing
checkout: www.rainforestqa.com

@rainforestqarainforest
Setting up servers / services

@rainforestqarainforest
Supporting Engineers

@rainforestqarainforest
Keeping stuff running

@rainforestqarainforest
Avoiding disasters

@rainforestqarainforest
Best practices
Ubuntu is simplest
Install from Mongo’s repo
Chose the right replica set size: 3 nodes
Never use 32 bits

@rainforestqarainforest
Alternatives
MongoHQ
MonogLab
ObjectRocket
MMS + AWS (via automation)

@rainforestqarainforest
Server components

@rainforestqarainforest
Storage
magnetic, ssd
metric: disk io

@rainforestqarainforest
Memory / RAM
fastest general store
metrics; free, swap usage

@rainforestqarainforest
Network
allows communication
use 1gbit or higher (avoid 100mbit)
metrics; latency, throughput

@rainforestqarainforest
CPU
more is better
mostly used
metrics; % usage

@rainforestqarainforest
Monitoring & Diagnostics

@rainforestqarainforest
Top things to monitor
Failover events
Repl Lag
Connections
Lock %
Disk usage

@rainforestqarainforest
Hosted is Simplest

MMS
Hosted, free for monitoring
!
Also can do
Backups
Automation

Scout
Monitors mongo, servers and many
others (Plugins!)
!
non-free

Server Density
Monitors mongo, servers and many
others (Plugins!)
!
non-free

@rainforestqarainforest
Opensource options
Ganglia
Munin
Nagios
Zabbix

Linux tools to learn
mongostat / mongotop
top
iostat
vmstat
netstat

@rainforestqarainforest
Mongo shell
rs.conf()
rs.status()
db.serverStatus()
db.stats()
db.currentOp()

@rainforestqarainforest
Backup options
mongodump
from the ﬁle system
LVM / EBS
MMS

@rainforestqarainforest
TEST YOUR BACKUPS

Missing indexes!
Running out of disk space
Untested backups
Sharding too late / wrong key
un-replicated / even set size

@rainforestqarainforest
Conclusion
do backups
monitor
read the docs

rainforest @rainforestqa
Questions?
@rainforestqa
@rhs

Mais conteúdo relacionado

Mais procurados

Xtrabackup and FTWRL

Raghavendra Prabhu

The challenge within telemetry in real-time systems is that you need as many sources of telemetry as possible (Throughput, latency, Errors, CPU, and many more... ) but you can't pay for extra overhead when our users are expecting sub-ms ops that scale to millions of transactions per second. In this talk, we'll describe how we're using and improving several OSS data structures to incorporate telemetry features at scale, and showcase why they do matter on scenarios in which we have Performance/Security/Ops issues.

Data Structures for High Resolution, Real-time Telemetry at Scale

ScyllaDB

Pulsar is used by a portfolio of products at Splunk for stream processing of different types of data, including metrics and logs. In this talk, Karthik Ramasamy will share how Splunk helped a flagship customer scale a Pulsar deployment to handle 10 PB/day in a single cluster. He will talk about the journey, the challenges faced, and the trade-offs made to scale Pulsar and operate it reliably and stably in Google Cloud Platform (GCP).

Scaling Apache Pulsar to 10 Petabytes/Day

ScyllaDB

Due to the exhaustion of IPv4 free address space, the use of IPv6 on the Internet is gradually increasing. All Windows operating systems since Windows Vista have IPv6 enabled by default. IPv6 brings a series of improvements compared to IPV4, but these improvements are also put a double-edged sword. Recently, we have been focusing on "IPv6" attack research and found that in the IPV6 environment, there are many attack points, such as Iptables will fail, use IPV6 to bypass the Web defense strategy and abuse IPV6-specific protocols for man-in-the-middle attacks, and Other attack ideas! In this speech, I will disclose the attack methods and ideas I have found for IPV6, and will also release tools for IPV6 attacks.

[CB19] New threats are already around you, the IPV6 attack must be understood...

CODE BLUE

IPLOG? A beginner's IDS for the WIN!

Nathan Gibbs

10 Basic Sysadmin Commands for Developers & Operators

Rosemary Wang

Nmap 9 truth "Nothing to say any more"

abend_cve_9999_0001

WordPress Security - A Top Down Approach

Brecht Ryckaert

As a k8s cluster-admin your app teams have a certain expectation of your cluster to be available to deploy services at any time without problems. While there is no shortage on metrics in k8s its important to have the right metrics to alert on issues and giving you enough data to react to potential availability issues. Prometheus has become a standard and sheds light on the inner behaviour of Kubernetes clusters and workloads. Lots of KPIs (CPU, IO, network. Etc) in our On-Premise environment are less precise when we start to work in a Cloud environment. Ebpf is the perfect technology that fulfills that requirement as it gives us information down to the kernel level. In 2018 Cloudflare shared an opensource project to expose custom ebpf metrics in Prometheus. Join this session and learn about: • What is ebpf? • What type of metrics we can collect? • How to expose those metrics in a K8s environment. This session will try to deliver a step-by-step guide on how to take advantage of the ebpf exporter.

Using eBPF to Measure the k8s Cluster Health

ScyllaDB

Yakiniku on the Cloud

Takao Funami

Modern systems are large, and complicated, and it is often difficult to account precisely where CPU cycles are spent in production. Once you begin measuring, you will find all sorts of strange surprises - like cleaning out strange objects from an attic that has accumulated stuff for decades. This talk discusses surprising places where we found CPU waste in real-world production environments: From Kubelet consuming multiple percent of whole-cluster CPU, via popular machine learning libraries spending their time juggling exceptions instead of classifying, to EC2 time sources being much slower than necessary. CPU cycles are being lost in surprising places, and often it isn't in your own code.

Where Did All These Cycles Go?

ScyllaDB

Buffer overflow for Beginners

Ajin Abraham

Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...

ScyllaDB

Let’s Fix Logging Once and for All

ScyllaDB

Nmap2Nessus Presentation Slides at Black Hat Asia Arsenal 2015

Keith Lee

Automatizando tarefas com grunt - O que era chato não é mais

Anderson Aguiar

Performance

Christophe Marchal

Thread Safe Interprocess Shared Memory in Java (in 7 mins)

Peter Lawrey

Mais procurados (18)

Xtrabackup and FTWRL

Data Structures for High Resolution, Real-time Telemetry at Scale

Scaling Apache Pulsar to 10 Petabytes/Day

[CB19] New threats are already around you, the IPV6 attack must be understood...

IPLOG? A beginner's IDS for the WIN!

10 Basic Sysadmin Commands for Developers & Operators

Nmap 9 truth "Nothing to say any more"

WordPress Security - A Top Down Approach

Using eBPF to Measure the k8s Cluster Health

Yakiniku on the Cloud

Where Did All These Cycles Go?

Buffer overflow for Beginners

Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...

Let’s Fix Logging Once and for All

Nmap2Nessus Presentation Slides at Black Hat Asia Arsenal 2015

Automatizando tarefas com grunt - O que era chato não é mais

Performance

Thread Safe Interprocess Shared Memory in Java (in 7 mins)

Semelhante a Basic Ops Skills and Tools for Beginners

In this episode, we will take a close look at 2 different approaches to high-throughput/low-latency data stores, developed by Netflix. The first, EVCache, is a battle-tested distributed memcached-backed data store, optimized for the cloud. You will also hear about the road ahead for EVCache it evolves into an L1/L2 cache over RAM and SSDs. The second, Dynomite, is a framework to make any non-distributed data-store, distributed. Netflix's first implementation of Dynomite is based on Redis. Come learn about the products' features and hear from Thomson and Reuters, Diego Pacheco from Ilegra and other third party speakers, internal and external to Netflix, on how these products fit in their stack and roadmap.

Netflix Open Source Meetup Season 4 Episode 2

aspyker

Low cost multi-sensor IDS system

Robert Schrack

Jstorm introduction-0.9.6

longda feng

Techiques for Writing Embedded Code_Internet of Things

alengadan

High performance json- postgre sql vs. mongodb

Wei Shan Ang

Capistrano and SystemD

Amoniac OÜ

Capistrano && SystemD

Aleksandr Simonov

Embeded System

Dhaval09

Cloud native deployment has become one of the major trends for large scale Big Data analytics. Compared to on-premise data center, cloud offers much stronger scalability and higher elasticity to Big Data applications. However, cloud is also considered to be less performance than on-premise alternatives due to virtualization and cluster resource disaggregation. We present a new cloud native Spark application architecture backed by persistent memory technology. The key ingredient of this architecture is a novel acceleration engine that uses Intel's 3DXPoint technology as external memory. We discuss how the performance of multiple aspects of data processing can be improved using this new architecture. As a key takeaway, audience will gain understanding on the benefits of latest persistent memory technology, and how such new technology could be leveraged in cloud data processing architecture.

Elastify Cloud-Native Spark Application with Persistent Memory

Databricks

JerryScript: An ultra-lighteweight JavaScript Engine for the Internet of Things

Samsung Open Source Group

SOUG_SDM_OracleDB_V3

UniFabric

W 0300 codingfor_life-batterylifethatis

jicheng687

As the industry moves towards more cloud based and containerised solutions such as Kubernetes, monitoring tools have to keep up. These new environments are far more dynamic than the hand-maintained machines of old, requiring more sophisticated and scalable approaches. This talk will look at how Prometheus has evolved over the past 5 years to be better able to cope with these challenges, including the 2.0 release and practices that we encourage in a cloud native world.

Evolving Prometheus for the Cloud Native World (FOSDEM 2018)

Brian Brazil

Customer and business requirements are shifting constantly. Today’s powerful programming languages can keep up—but what about your database? NetApp® MongoDB solutions offer a flexible, scalable answer. Learn how NetApp storage solutions will accelerate your MongoDB Performance, reduce operational Costs and provide the highest levels of Availability and Security. These solutions provide advanced fault-recovery features and easy, in-service growth capabilities to accommodate your unpredictable, ever-changing business demands. NetApp storage is designed to help you build a high-performance, cost-efficient, and highly available analytics solution. So you can focus on adding real business value.

MongoDB Europe 2016 - Deploying MongoDB on NetApp storage

Basic Ops Skills and Tools for Beginners

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (18)

Semelhante a Basic Ops Skills and Tools for Beginners

Semelhante a Basic Ops Skills and Tools for Beginners (20)

Mais de MongoDB

Mais de MongoDB (20)

Último

Último (20)

Basic Ops Skills and Tools for Beginners