Learning how to teach Ops.
•Transferir como PPTX, PDF•
0 gostou•70 visualizações
In this talk, I'll tell you the story of how we slowly learned to teach Ops faster, better, and in a repeatable way.
Recomendados
Mais conteúdo relacionado
Semelhante a Learning how to teach Ops.
Semelhante a Learning how to teach Ops. (20)
Mais de John Contad
Mais de John Contad (9)
Último
Último (20)
Learning how to teach Ops.
- 1. Learning to Teach Ops @JohnContad @REATechBlog
- 2. It all started with a problem…
- 3. …Youth!
- 5. And so we tried dumb things. • “You should be on pager…” • “You should fix a couple of boxes...” • “We’ll synthesize a couple of problems for you...”
- 6. The problem with teaching Ops. • Ops is experiential. • “Defensive” Ops cannot be scheduled. • “Offensive” Ops needs a lot of knowledge baselines.
- 12. The Whiteboard Exercise. • Shows levels of established proficiency. • No “faking” of knowledge. • Delineates domains that need work. • Exposed what I don’t know myself.
- 20. What now?
- 21. Stumbling through Instructional Design • V.A.K. Learning Styles • The Teachback Method • Knowledge checks
- 22. Anyone can play guitar Ops • Faster pickup of Ops skills • Repeatable template for Dev/Grad onboarding • Anyone else can do it
- 23. More to come… • Metrics! Measure improvement. • How to train trainers to train. • How to scale to more people concurrently. • How to scale with limited resources.
- 24. In closing.
Notas do Editor
- Hello, everyone. My name is John Contad, and I work with REA Group – the people behind realestate.com.au. My team in charge of Global Infrastructure and Architecture – which is a unit in the company that deals with shared infrastructure, patterns of work, and empowering people to work better. For this session, I’ll be talking about our journey on how to teach IT Operations skills.
- Every solution starts with a problem. Mine started less than a year ago, and the cause of the problem is similar to what a lot of people have problems with.
- Youth! Young, smart, enthusiastic youth. To be more exact, we had a Graduate program – which pulls talented people fresh out of Uni, and gives them a chance to rotate within the business to maximize their learning. Stupidly, I put my hand up to take care of some of these…youth. Some of them, apparently, wanted to learn Ops.
- “Seems painless,” I thought – but I realized too late that it wasn’t going to be easy. “Ops” is one of those vague, nebulous terms – like “Cloud”, or “Agile”, or “DevOps”, and when they asked me about it, I really didn’t have a meaningful answer. I only knew the things I did. I only knew that I had to pass it on.
- And so we tried everything. We used the shotgun approach initially – because to me, a lot of the appeal of Ops comes from the fact that you’re trying to fix problems as quickly and cleanly as possible. It was about trying to figure things out under pressure. We put them on pager, so they’d have the experience of responding to alerts. We put them on the task of fixing tech debt. We even made some broken boxes for them to fix.
- Through this shotgun process, we started to realize that teaching Ops was going to be difficult, because of a couple of problems. One, Ops relies heavily on experiential knowledge. It’s built on things you’ve run into before – app-breaking network problems, crushing SQL issues… Two, it’s multi-disciplinary. When we tried to get them to fix a couple of issues, we realized that they needed to be able to troubleshoot problems on a lot of layers. Ten thousand things can be broken; they needed to know about each of those things. Three, defensive ops – or reactive ops, which is when you’re responding to something broken – isn’t conducive to learning. The time pressure, plus the breadth of the domain you need to know amounts to a lot of rushing. Four, offensive ops – or proactive ops, which is when you’re building things that stop things from happening or make things less error prone – requires a good baseline of information, and for you to be able to tie those things together.
- My feelings on this can be summarized by this comic strip, which I think highlights the difference between information and knowledge. Ops in particular is made of many, many shallow units of data that you tie together to make a whole. The problem with trying to pass down knowledge is that there’s too many of these blobs. There was no way someone could learn it in three months – which is the length of a rotation.
- …and this is when things got weird.
- And so we had a new mission. We were going to try and define these blobs of information that he needed to know. And how did we do that?
- Well, we started with everyone’s favorite interview question: “What happens when you go to dub-dub-dub dot google dot com? Go as deep as possible.”
- Eventually, we worked together and had this really nice high-level map. There’s a CDN, there’s DNS, there’s SSL, and there’s even a tiny-ass BGP frame over there.
- This ended up being really, really good – for many reasons. 1.) This showed things he already knew, and that we didn’t need to cover. That’s less blobs out of the map. 2.) He wasn’t able to feign knowledge – because nothing’s more naked than trying to explain something to someone. 3.) Which then highlighted – for his and my good – the parts or domains where he needed to know more. In that previous slide, for example? The SSL thing was just a bunch of question marks. 4.) This also showed what I myself didn’t know. My BGP knowledge wentas far as pushing routes, but I didn’t know how the damn thing worked. I had to pull someone in. 5.) Last, and the most important – is the “Teachback Method”. I’ll elaborate on this later.
- From here, we were able to cut his knowledge into domains. God bless Trello. We made cards for each tiny thing, then grouped them – but you gotta start somewhere, and we started with everyone’s favorite security problem: Wordpress.
- From there, we were able to focus on specific parts: Networking, basic security – all working on the same stack.
- Which then grew into this massive list. This is just a slice of what it looks like.
- With each card item, there were specific questions – but the most important part is that he should be able to explain it back to me. This took the shape of screenshots and documentation, to whiteboards – and he loved whiteboarding.
- He loved it so much, he drew a messload of it. We were able to amass a lot of these – from how load balancers work, to SAML, to RAID, to SANs, to DNS, etcetera, etcetera. This was spectacularly helpful, because it helped visualize a lot of abstracted concepts, and gave a clear picture on the components of a system.
- Once he had a good picture of how a system worked, we were able to do “verbal” troubleshooting. Each of those cards has a root cause at the back, and he had to figure out what was wrong – by telling me what he’d do, and me telling him what he’d get back. “I check disk space.” “It’s full.” And it was going marvelously well. But we had to get better, had to improve some more. So I did what anyone would do.
- So I picked up on a couple of things, which I’ll run through briefly. VAK Learning Styles stands for Visual, Auditory, Kinesthetic – each person has his/her own learning bias, and you should take it into account to optimize learning. Teachback method is when you ask someone to teach the subject to you. Much like the Whiteboarding exercise, having them explain it back to you shows gaps in knowledge. Knowledge Checks is the demonstration of knowledge from the learners’ part – and is essential as a mental “commit”. Spaced repetition is putting intervals of time between previously learned material to take advantage of psychological spacing.
- Overall, this produced a faster pickup of Ops skills for grads. We then had a repeatable workflow for Dev or Grad onboarding, which saved time because we didn’t have to do it from scratch. And the good thing is, anyone can take someone through it now.
- We have a couple more things to add: Metrics, so we have a demonstrable basis for which methods work. We need a meta-template to train trainers on how to train untrained people. And lastly, a lot of this effort comes from volunteered hours – this is not in the job description of the people involved. We have finite resources, and we need to make sure we can maximize our reach with them.
- In closing, this has been a pretty good journey. We’ve scoped out the implementation, created the architecture, and produced repeatable artifacts. We’re aiming to have metrics, A/B tests, spikes…and really, all of that is part of Ops in itself. Thank you.