Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Infra@LINE_Chris
Similar to Infra@LINE_Chris (20)
Recently uploaded
Recently uploaded (20)
Infra@LINE_Chris
- 2. Infra@LINE 1. Me 2. Server-Side 3. Infra 4. Technology 5. Demo
- 3. 自己紹介 Chris Pickel Server Development Team LINE株式会社 京都開発室 クリス・ピッケル
- 4. Me–Name • I’m a Pickel, not a Pickle Noto Color Emoji
- 5. Me–I like… > > > > > > > > > > > > > > > > Noto Color Emoji
- 6. Me–History • U. of Pennsylvania • Google New York (5y) • Asago City schools (2y) • Google Munich (2y) • LINE Kyoto (0.5y)
- 8. Server-Side Why do I prefer server-side?
- 9. Server-Side You know your environment Bjarni Juliusson / Wikimedia Commons
- 12. Server-Side Lots of open-source github.com/line top 5 stars: • line/armeria 1.7k (server/client) • line/promgen 422 (server) • line/stellite 310 (server/client) • line/centraldogma 249 (server) • line/clay 102 (android) ※ Excludes {sdk,starter}
- 13. Infra What is infrastructure? Why do I like infrastructure?
- 15. Infra • Machines • Networking • Storage • Monitoring Pixabay
- 16. Infra You know your users
- 17. Technology What technologies do we use?
- 18. Technology • Docker • Drone • Kubernetes • Rancher • OpenStack • Ansible • Prometheus • Go Harland Quarrington (OGL) / Wikimedia Commons
- 19. Tech– Docker “It works for me”
- 20. Tech– Docker “It works with my • OS (Linux, Mac, Win) • Installed software • Software versions • Network”
- 21. Tech– Docker Put it in a container • Minimal Linux system • Linux, on Mac or Win • Includes all deps • Isolated from host jumilla (CC-BY) / flickr
- 22. Tech– Docker $ docker run -ti ubuntu bash
- 23. Tech–Drone “It passes for me” US DoD / Wikimedia Commons
- 26. Tech– Kubernetes Microservices: • Work independently • Work together Pixabay
- 27. Tech– Kubernetes • Configs, not commands • Containers, not machines • Clusters, not datacenters Pixabay
- 28. Tech– Kubernetes • Minikube (local) • Google GKE • Microsoft AKS • Amazon EKS
- 33. Tech– Prometheus “Did it stop?” Jon Sullivan / Wikimedia Commons
- 34. Tech–Go Pixabay • Docker • Kubernetes • Drone • Rancher • Prometheus
- 35. Technology ◎ Docker ○ Drone ○ Kubernetes ╳ Rancher ╳ OpenStack △ Ansible △ Prometheus ○ Go Harland Quarrington (OGL) / Wikimedia Commons
- 37. Thanks Balint Földesi (CC-BY) / Flickr
- 38. $ cat serve.go time.Sleep(5 * time.Second) http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { log.Printf("%s %s %s %s", r.Method, r.URL.String(), r.RemoteAddr, r.UserAgent()) w.Write([]byte(*greeting + "n")) })
- 39. $ cat Dockerfile FROM golang:1.12-alpine RUN go build serve.go CMD ["./serve", "--listen=:80", "--greeting=Hello, Docker!"] $ docker build --tag $TAG . Successfully built 728b5f3086be Successfully tagged docker- registry.linecorp.com/christopher- pickel/hello-world:latest
- 40. $ docker run -rm --name=demo --publish 127.0.0.1:2080:80/tcp $TAG 2019/07/25 06:34:41 starting 2019/07/25 06:34:46 listening on :80 2019/07/25 06:34:52 GET / 172.17.0.1:41676 curl/7.54.0 $ curl localhost:2080 Hello, Docker!
- 41. $ docker exec -ti demo bash bash-4.4# ps PID USER TIME COMMAND 1 root 0:00 ./serve --listen=… 12 root 0:00 bash 22 root 0:00 ps bash-4.4# curl localhost Hello, Docker! 2019/07/25 06:36:27 GET / 127.0.0.1:36774 curl/7.64.0
- 42. bash-4.4# file ./serve serve: ELF 64-bit LSB executable, x86-64… bash-4.4# uname Linux bash-4.4# exit $ uname Darwin
- 43. $ cat kube.yaml containers: - name: hello image: docker-registry.linecorp.com/ch… args: ["./serve", "--listen=:80", "--greeting=Hello, Kubernetes!"] readinessProbe: httpGet: path: /healthz port: 80
- 44. $ kubectl apply -f kube.yaml namespace/demo created deployment.apps/hello created service/hello created $ curl $(minikube ip):$PORT Hello, Kubernetes! namespace/demo created deployment.apps/hello created service/hello created
- 45. $ kubectl logs hello-674f6956d-97kcb 2019/07/23 09:11:34 starting 2019/07/23 09:11:39 listening on :80 2019/07/23 09:11:44 GET /healthz 172.17.0.1:39214 kube-probe/1.15 2019/07/23 09:11:49 GET /healthz 172.17.0.1:39214 kube-probe/1.15 2019/07/23 09:11:51 GET / 172.17.0.1:63338 curl/7.54.0
- 46. $ kubectl edit deployment hello (s/replicas: 1/replicas: 10/) $ kubectl edit deployment hello (s/Hello, Kubernetes/Hello, Rollouts/) $ kubectl rollout status deployment hello Waiting for deployment "hello" rollout to finish: 9 of 10 updated replicas are available... deployment "hello" successfully rolled out
- 47. Thanks Balint Földesi (CC-BY) / Flickr
Editor's Notes
- みんなさん、こんにちは。クリスと申します。今日はLINEインフラの説明をしたいと思います。 まずは短い自己紹介をします。それから サーバーサイドについて、 インフラについて、 日常に使うテクノロジー、そして時間あれば 簡単のDocker, Kubernetesのデモンストレーション の準にします。
- 私は京都オフィスサーバー開発チームのクリス・ピッケルです。よろしくお願いします。
- 名前についてのお願いです。これを覚えて帰ってください：私はピクルスじゃなくてピッケルです。 Emoji: Noto Color Emoji (Apache license)
- 好きなものは… 絵文字です。 Emoji: Noto Color Emoji (Apache license)
- 私はアメリカのニュウヨーク州出身です。ペンシルヴェニア大学を卒業した時ニュウヨークへ帰って、グーグルニューヨークで働き始めました。 でも、何年か後、自分の環境を変えたいと思って、日本に来ました。兵庫県の小さい町の中学校で英語教師をしました。 終わった時、やはり海外で住む方がいいと思って、グーグルに帰ってミュンヘン支所で働きました。でも、ドイツで分かったことは海外で住むのじゃなくて、日本で住むのが好きだということです。それで、京都へ引っ越して、LINEで去年から働いています。 https://commons.wikimedia.org/wiki/File:Google_%22G%22_Logo.svg https://en.wikipedia.org/wiki/Asago,_Hy%C5%8Dgo#/media/File:Symbol_of_Asago_Hyogo.svg
- LINEでVerdaと言うPrivate cloudがあります。その中で、私はKubernetes関係の開発しています。 Kubernetesはサーバーサイドインフラです。このプレゼンテーションで大体これについて話します。 じゃ、すみません、今からは日本語はちょっと難しいので英語で話します。 Photo: company work
- First, I want to talk about why I like Server-Side engineering. Why do I prefer server-side?
- For me, the best thing about server-side engineering is knowing your environment. Server-side life is very simple. You choose your machine. You choose your OS. You install your software. If you want a newer machine or OS or software, you upgrade it. You’re in control. By comparison, on iOS, LINE supports two OS versions and several screen sizes. On Android, there are lots of combinations–too many to count. And the LINE app has to work on all of them. It has to work with no network. It has to keep working even if users don’t update LINE. I like server-side because I like not having to worry about all that. Photo: https://commons.wikimedia.org/wiki/File:PDP-12-Update-Uppsala.jpeg (PD)
- Also, because you know your servers, you can monitor them so you know they are working. If your server has a problem, you can find out and you can fix the problem. (probably by turning it off and turning it on again) Client-side monitoring is a lot harder. If there’s a bug in the LINE app on users’ phones, how do you find out? And how do you fix it? Apple’s review process could take a week or two. It’s a lot easier server-side. Find, fix, push, finish. Photo: https://www.pexels.com/de-de/foto/aufnahme-daten-diagramm-ecg-415779/ (CC0)
- Another great thing about server-side work is that we use lots of small focused components. Or at least, we can, and we like to. Software tends to grow and grow, but we do our best to keep things small. On the other hand, do you know every single feature in the LINE app? There are a lot of features. I sure can’t remember all of them. And they all go into one single, huge app. On the server, we can break down our software into lots of pieces. They compile fast and they’re easy to work with, by comparison. Photo: own work
- One last thing about server-side is that there’s a lot of open-source. And actually, most of LINE’s open-source creations are server-side software. Here’s a list of LINE’s most popular open-source software on GitHub. Basically, LINE client engineering benefits when the LINE app has lots of distinctive features, but server engineering benefits when the servers are standard. So, for server-side we tend to share our technology, and to use technology that others have shared. Later on, I’m going to talk about some technologies that I use in my daily work. All of the software projects I’ll discuss are open-source. So, you can go home today and try them out yourself. Photo: own work
- So, what is infrastructure? Why do I like infrastructure?
- Essentially, infra is about making building blocks for other server-side engineers. Photo: own work
- Do you want to build a server? You need a machine. You need a network. You need file storage and a database. You need monitoring. You can set everything up yourself, but it’s a lot of trouble. It’s much easier if you can go to the infra engineers, and say: “please give me all of that”. That’s infrastructure. Photo: https://pixabay.com/photos/computers-information-technology-2653005/ (CC0)
- What do I like about infra? Well, the LINE client has many millions of users, and that’s fine. But in infra, the “users” are other server-side engineers at LINE, and there are hundreds of them, not millions. And having hundreds of users means that you can actually go and talk to all of them, and say “what do you want?”, and hopefully “here you are!”. That’s what I like about infra. Photo: own work
- So, what technologies do we use in our daily work?
- Here’s a list of some of the software I might use from day to day. All of this is open-source, as I said before. So, if any of it is interesting to you, you can try it too. Photo: https://commons.wikimedia.org/wiki/File:Computer_Circuit_Board_MOD_45153619.jpg (OGL)
- So, please imagine. You have a friend. And your friend gives you some software. And the software doesn’t work. So, you ask your friend for help, and they say, “Oh, weird. It works for me.” What does that mean? Photo: own work (Ggantija, Gozo)
- Hmm, well. It means “It works on my OS, and my OS version, with all of the software I installed, and my software versions, and on my network. So, you know. Just run the software the same way, and it should work.” Which isn’t all that helpful. Photo: own work (Ggantija, Gozo)
- The idea behind Docker is that you can build containers. Inside the container, you have a small, completely self-contained Linux system with everything you need to run your software. Outside the container, it doesn’t matter. Linux, Mac, Windows, any of them can be the host machine. Because it’s self-contained, if you have a container, you know that the software will work. Photo: https://www.flickr.com/photos/jumilla/14403331148 (CC-BY)
- So yeah, Docker is really useful software, and you can use Docker to make sure that “it works for me” means “it works for everyone”. If you install Docker, you can run this command to run a local shell on Ubuntu. Even on Mac. Even on Windows. I was really surprised the first time I tried it. https://commons.wikimedia.org/wiki/File:Docker_(container_engine)_logo.svg (Apache license 2)
- Same idea. Sometimes your test fails, and your coworker says “Oh, weird. It passes for me.” Image: https://commons.wikimedia.org/wiki/File:Defense.gov_photo_essay_061021-D-7203T-002.jpg (PD)
- Drone is basically Docker for tests and continuous integration. If you don’t know continuous integration, it means running tests on everything you write. Before you merge a change, it runs the tests. After you merge a change, it runs the tests. Tag a release? It runs the tests. And, because Drone runs the tests in a Docker container, you know they will pass for everyone. Photo: https://www.flickr.com/photos/jumilla/14610034513/ (CC-BY)
- You can download Drone and run it yourself, which is what we do at LINE. Or, if your project is open-source on GitHub, you can use cloud.drone.io for free. If you’re not already using continuous integration, give it a try. Actually, there are a few CI options out there, like Travis and CircleCI and others. Drone is built on Docker, and that makes it convenient for server-side work. Travis and CircleCI are work with more platforms, so they’re convenient for client-side work. They’re all good, so you can decide what to use based on the needs of each project. Photo: own work
- OK, so, the great thing about Docker is isolation. Each of your programs works independently. But, if you run a microservice architecture with lots of small, interconnected servers, then they can’t be totally isolated. They need to work independenty AND together. Traditionally, if you had a lot of servers, you’d assign them to machines, and configure the servers to talk to each other. But machines fail, and get outdated. In the end, to run a lot of servers, you end up administering machines, and reconfiguring everything each time you replace one. It doesn’t scale. Photo: https://www.pexels.com/photo/abstract-art-circle-clockwork-414579/ (CC0)
- With Kubernetes, you can write a configuration that says “run ten servers” and let Kubernetes take care of the rest. It starts and stops servers to make sure that there are ten running. It assigns and reassigns them to machines so that your server stays running, even when the machines don’t. It gives your server an address that doesn’t change, even when the machine does. Photo: https://pixabay.com/photos/blueprint-ruler-architecture-964629/ (CC0)
- These days, it’s been getting more and more popular, and now Google, Microsoft, and Amazon all offer Kubernetes clusters as part of their cloud. If you want to try Kubernetes, you can try any of those, or run it locally. If you’re comfortable with Docker, I think it’s a good next step. Logo: https://commons.wikimedia.org/wiki/File:Kubernetes_logo.svg
- Where do our Kubernetes clusters come from? We build them ourselves with Rancher. Now Rancher is a good technology, and it’s very helpful for us. But, I don’t necessarily recommend you use it. If you want to try Kubernetes, there are lots of easier options, like the three I mentioned before. If you’re running your own cloud like LINE, then definitely, I recommend Rancher. But, most people don’t need to run their own datacenters. Photo: https://pixabay.com/photos/longhorn-cattle-ranch-livestock-754894/ (CC0) Logo: https://rancher.com/brand-guidelines/
- Speaking of which, how does LINE run its cloud? LINE uses OpenStack. So, when we use Rancher to create a Kubernetes cluster, OpenStack is how we get the machines. It’s very important to us, because it’s part of how we build Kubernetes clusters with Rancher, but I’m just a user. I don’t know much about running it. Image: https://pixabay.com/photos/ethernet-data-internet-network-1245122/ (CC0) Logo: https://www.openstack.org/brand/openstack-logo/logo-download/
- OK. So, when you start up your system, how do you start it? How do you make sure that startup was successful? Some people write a guide to follow and a checklist and then run through them by hand. Some people just remember things and don’t write them down at all. I think that it’s good to write things down. And really, the best format for setup instructions is a script that automates the setup. That’s what Ansible does. Photo: own work (Oshu, Iwate)
- If your server setup is complicated, you should definitely consider automation, and Ansible is one way to do that. For my part, I actually really like the standard old make tool. It’s simple, it’s everywhere, and it does dependencies. I think even a really simple setup benefits from automation, and make can do simple setups very well. On the other hand, if your setup is complicated—well, the best thing is to make it less complicated, but if it’s complicated, Ansible is good to use. Photo: own work Logo: https://en.wikipedia.org/wiki/File:Ansible_logo.svg
- And lastly, what if your system breaks? How do you know? Hopefully, it’s not Twitter. Prometheus is software for monitoring servers. If people are depending on your servers, you should be monitoring them. Photo: https://commons.wikimedia.org/wiki/File:Montana_16_bg_062406.jpg (PD)
- Most of our software is written in Go. Docker is Go, Kubernetes is Go, and Drone, and Rancher, and Prometheus too. So, on my team, when we write LINE internal software, we choose Go too. (OpenStack and Ansible are mostly Python) Photo: https://pixabay.com/photos/evraiki-gophers-family-rodents-2514544/ (CC0)
- So, here’s the list of software again, and some recommendations. I think everyone should try Docker. If you don’t know about containers, I think you’re missing out. Kubernetes is getting more important too, so it’s also worth trying. Drone is good. But there are also lots of good alternatives too. Depending on your needs, you might find Ansible or Prometheus relevant, but maybe not. And unless you’re running your own cloud, I wouldn’t worry so much about Rancher or OpenStack. Photo: https://commons.wikimedia.org/wiki/File:Computer_Circuit_Board_MOD_45153619.jpg (OGL)
- So, what do I spend my time on at work? Where does the time go? I usually get to work around 10 and leave around 7. Between that time, I don’t feel that I really have a standard schedule, but I tried to estimate about how much time on average I spend on different tasks. So…
- And that’s all from me. Thank you for listening. I hope I was able to give you an impression of how we work with server-side infrastructure, and that you feel encouraged to come work with us at LINE Kyoto and help with our mission of closing the distance. Photo: https://www.flickr.com/photos/balintfoeldesi/16707315815/ (CC-BY)
- And that’s all from me. Thank you for listening. I hope I was able to give you an impression of how we work with server-side infrastructure, and that you feel encouraged to come work with us at LINE Kyoto and help with our mission of closing the distance. Photo: https://www.flickr.com/photos/balintfoeldesi/16707315815/ (CC-BY)