Managing Databases with Chef - ChefConf 2015
•Transferir como PPTX, PDF•
2 gostaram•1,359 visualizações
The easy part of implementing chef is getting applications and OS configuration within the system. But how do you deal with databases? We'll talk through the patterns that are out there and give some examples of how NCR Hospitality is approaching the problem. https://youtu.be/DYSvbcFC2ck
Recomendados
Recomendados
Mais conteúdo relacionado
Mais de Chef
Mais de Chef (20)
Último
Último (20)
Managing Databases with Chef - ChefConf 2015
- 2. 2 About NCR :: Pioneered Cash Register
- 3. 3 About NCR :: Hospitality
- 4. 6/2/2015 4 NCR’s Biggest Challenge :: High Availability With A Side Of Fries
- 5. 2008: Darkness Automation and CI not enough Influence Deployment and Delivery Blog at hedge- ops.com Married to a photographer 5 About me :: Michael Hedgpeth
- 6. 6
- 7. But what about the databases? Answer was: Snowflakes 7 The Database Question
- 8. Answer: Simple and Repeatable approach 8 The Database Question
- 9. 9 Problem Scope :: Installing Databases Backup & Restore Schema Management Database Configuration
- 10. NCR Confidential 10 Problem Scope :: Installing Databases
- 15. Our new process will be consistent BUT we are in debt with our old process We’ve made schema consistency a separate initiative REDGATE is a great start to this effort 6/2/2015 15 Schema Consistency Is Separate
- 16. 16 Our Schema Management Strategy :: Don’t reinvent the wheel
- 17. 17 Our Schema Management Strategy :: Make It Chef-Friendly
- 18. 18 Requirements :: Persisted in the database
- 19. 19 Requirements :: Knows if changes are required
- 22. 22
- 23. Application: http://bit.ly/1DqxTz5 Cookbook: http://bit.ly/1NBc5BP Supermarket: http://bit.ly/1Dqy3qa 23 Introducing… The Database Upgrader
- 34. 34 Challenges :: Long-running updates
- 35. 35 Challenges :: Internal vs. External changes
- 36. 36 Database Configuration Add Linked Server Memory in SQL Server Accept RPC call
- 37. 37 Strategy Matters When Gaining Allies :: Other Nodes Preproduction Cattle, Not Pets Culturally Automated
- 38. Strategy Matters :: Other Nodes 38
- 39. Strategy Matters :: Preproduction 39
- 40. Strategy Matters :: Cattle, Not Pets 40
- 41. Strategy Matters :: Culturally Automated 41
- 43. What else? 43
- 44. 6/2/2015 44
Notas do Editor
- Who I am: and I’m Michael Hedgpeth from NCR Brief Problem Overview: Databases are a key part of our infrastructure, but nobody talks about them! I’d like to lay a foundation that will get us to start talking about it Outline: Context Requirements Example Strategy
- Context :: NCR We invented the cash register 131 years ago We are reinventing ourselves into a hardware-driven software company
- I work within the Hospitality division of NCR which mainly focuses on providing technology solutions for restaurants globally.
- Our biggest challenge is: High availability with a side of fries – What happens when we’re ringing someone up and they spill a coke on the terminal? Each transaction is vital - failover Rapidly change out physical components quickly Deal with an unsafe world
- Tell My Story: Remember 2008 when the world was going to end? We wanted to automate build and test – did so with TeamCity as our core It wasn’t enough Biggest constraint: deployment and delivery Blog plug Annie Plug
- In 2014 we wanted to automate configuration management of our hosted products in the DC and engaged Chef for a division-wide automation initiative
- Story: Daniel’s first question (Windows, PostgreSQL, hadoop) Snowflakes: frustrated me because it felt like we were supposed to figure it out on our own felt like Karate Kid
- After doing the investigation, though, the database question has Simple solutions that apply to many different situations It’s more than just snowflakes So when I decided to go to ChefConf, I wrote the organizer that this is a session that I wanted to attend, and if they weren’t providing it, I would give it.
- When people ask the database question, they’re talking about one of four things
- How do we manage virtual images for creating new database nodes? We already solve this in other ways We could use chef for this, but it’s not important to us right now
- How do we do backup and restore? – this is already solved by Red Gate Backup
- How do we do schema management? – this should be already solved, but we’ll need to focus on this some more
- How do we configure the database itself once it is installed? – this should be already solved but we’ll need to focus on this some more So we’ll talk about the last two
- A majority of the session will be about schema management
- Before we do, let’s talk about what we aren’t talking about first. We want to create a repeatable schema management solution going forward. But what do you do about the past bad behavior that has created schema inconsistency? We need to deal with that separately. We’ll use a tool like REDGATE to ferret out differences in schemas and solve them. We’re not going to use Chef to discover differences in schemas, because that’s not what Chef is great at. Schema consistency issues, when they exist, have become separate but complementary initiatives to chef.
- Our strategy for schema management is simple: Let’s not reinvent the wheel. If a team has a schema management approach that fits our requirements, we let them keep that approach. If not, we’ll make minor improvements to the approach to fit the requirements I’ll share in a few moments.
- We really want to make existing solutions Chef friendly. And this has proven to be remarkably simple.
- Our requirements are simple: We need for all actions related to schema management to be persisted within the database itself. This lets you check a database and see what has happened It lets you restore a backup and then apply an update seamlessly It’s just easier
- We need to run something that knows if a change is needed Just like chef – it applies to databases too
- We need to execute those changes
- We need to run it within a recipe in an idempotent way
- This book is awesome It breaks down reading into composable pieces And reproduces it over and over again And then my son gets a reading rainbow cake That’s pretty much what we’re going to do Except the cake is awesome automation
- Let’s look at a sample application that fits these requirements Available now on github and the chef supermarket Also look at my blog for links: hedge-ops.com
- A Version table within the database keeps track of what has run against the database We measure everything just as we do with software in the database It’s natural to persist stuff in the database!
- Create a scripts folder with sql files where the name is equal to the version number.
- The application can initialize this table within the database, and a resource within the cookbook ensures that the table is there and ready to go.
- We need to run something that knows if a change is needed
- We need to execute those changes
- We need to run it within a recipe in an idempotent way And we’re going to use John Keiser’s awesome resource cookbook!
- We need to run it within a recipe in an idempotent way
- We need to run it within a recipe in an idempotent way
- We need to run it within a recipe in an idempotent way
- Before you put it in production, know these things: It’s only for SQL Server It’s not transactional, so failures will leave you in a bad state Use it as a starting point for ideas
- Before we finish the topic of schema management, let’s talk about a couple of challenges we’ve faced when planning out schema management for all of our products. Story: flagship product, thousands of databases on hundreds of nodes – all upgraded at once – with Scott Analytics will tell us, but we need real time – we use these tools for certain things but need a tighter time resolution What to do if the internal team does a lot of changes but wants to limit the changes available to external teams? Have a major/minor version! (image – internal/external, maybe a doorway) Analytics will tell us, but we need real time – we use these tools for certain things but need a tigher time resolution
- Story: Mike manages dozens of schema changes a month in every release. How does he manage it?
- Configuration Management boils down to database engine configuration, which is limited
- At first we think that database people will be the most resistant to change That hasn’t been my experience They are my biggest allies, and here’s how I’ve made it this way
- We’re going to make the other nodes in the system better, which will make the database easier to manage.
- It’s easier to create a preproduction environment that models production. A lot of that comes down to cost of time, and we are lowering that through automation
- The servers in the infrastructure make a transition from pets that we love adore and know the names of, to cattle. This gives them more flexibility to changing their solution based on changing technology
- The servers in the infrastructure make a transition from pets that we love adore and know the names of, to cattle. This gives them more flexibility to changing their solution based on changing technology
- I got frustrated about snowflakes But now I know what it was about: Chef is a framework that combines existing solutions with new capabilities I don’t have to wait around for the right answer I can create our right answer With applications and with the database
- Let’s keep talking about this; I know there’s more to discuss.