R.I. Pienaar gives an introduction to MCollective at the San Francisco and Silicon Valley joint Puppet User Group, May 2013.
Silicon Valley PUG: http://www.meetup.com/SiPMUG/
SF PUG: http://www.meetup.com/SFPMUG/#past
2. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Who am I?
• Puppet user since 0.22.x
• Architect of MCollective
• Author of Extlookup and Hiera
• Developer at Puppet Labs London
• Blog at http://devco.net
• Tweets at @ripienaar
• Volcane on IRC
3. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
What is MCollective?
Framework for building server
orchestration and parallel job execution
systems
24. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Class Filters
$ mco package status httpd -C /apache/
Server Addressing
Nodes with Puppet Classes /apache/
applied
25. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Fact Filters
$ mco package status httpd -F country=uk
Server Addressing
Nodes with the custom fact “country”
set to “uk”
26. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Identity Filters
$ mco package status httpd -I devco.net
Server Addressing
One specific node called “devco.net”
27. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Simple Combined Fact and Class Filters
$ mco package .... -W “country=uk /apache/”
Server Addressing
Nodes in the UK with Puppet Classes
matching /apache/
28. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Compound Statements for facts, classes and data
$ mco .. -S “((country=uk and /apache/)
or customer=acme)
and puppet().config_retrieval_time > 30"
Server Addressing
UK nodes with Apache in addition to all
nodes for “customer=acme” where
Puppet compiles are slow
36. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective Agents
• Units of addressable logic
• An agent has actions - package agent has
update, status, install, uninstall etc
• Written in Ruby, actions can be Python,
PHP, Perl and others
• Deployable for interactive commands,
background daemons, SOA style services,
etc
• Optional custom user interfaces
37. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Generate a skeleton agent
$ mco plugin generate agent nrpe actions=runcommand
Created plugin directory : nrpe
Created DDL file : nrpe/agent/nrpe.ddl
Created Agent file : nrpe/agent/nrpe.rb
Writing an Agent
38. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
DDL File describes the agent for UI
generation and input validation
metadata :name => "nrpe",
:description => "NRPE Agent",
:author => "R.I.Pienaar <rip@devco.net>",
:license => "ASL2.0",
:version => "0.1",
:url => "http://devco.net",
:timeout => 10
action "runcommand", :description => "Run a preconfigured NRPE command" do
input :command,
:prompt => "Command",
:description => "NRPE command to run",
:type => :string,
:validation => 'A[a-zA-Z0-9_-]+z',
:optional => false,
:maxlength => 50
output :exitcode,
:description => "Exit Code from the Nagios plugin",
:display_as => "Exit Code",
:default => 3
.
.
.
end
Writing an Agent
39. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Agent logic
module MCollective
module Agent
class Nrpe<RPC::Agent
action "runcommand" do
reply[:exitcode] = run(get_nrpe_command(request[:command]),
:stdout => :output,
:stderr => :output,
:chomp => true)
end
def get_nrpe_command(command)
# not shown
end
end
end
end
Writing an Agent
40. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Ready to deploy...
Writing an Agent
$ mco plugin package
Created RPM and SRPM packages for mcollective-nrpe-agent
Created RPM and SRPM packages for mcollective-nrpe-common
$ ls -l *rpm
mcollective-nrpe-agent-0.1-1.noarch.rpm
mcollective-nrpe-agent-0.1-1.src.rpm
mcollective-nrpe-common-0.1-1.noarch.rpm
mcollective-nrpe-common-0.1-1.src.rpm
41. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
...but test your code first using
mcollective-test gem
Writing an Agent
describe "nrpe agent" do
describe "#runcommand" do
before do
@agent = MCollective::Test::LocalAgentTest.new("nrpe",
:agent_file => “agent/nrpe.rb”).plugin
end
it "should return correct status" do
@agent.expects(:get_nrpe_command).returns("/bin/true")
result = @agent.call(:runcommand, :command => "rspec")
result.should be_successful
result.should have_data_items(:exitcode=>0)
end
end
end
Custom matchers
and helpers
42. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Post deployment, interact using
standard RPC user interface
Writing an Agent
$ mco rpc nrpe runcommand command=check_load
Discovering hosts using the mongo method .... 28
* [ ============================================================> ]
28 / 28
devco.net Request Aborted
UNKNOWN
Summary of Exit Code:
OK : 27
WARNING : 1
UNKNOWN : 0
CRITICAL : 0
Finished processing 28 / 28 hosts in 418.42 ms
Agents can provide custom
aggregation plugins
43. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
...or web interfaces with auto-generated
user interfaces based on the DDL
Writing an Agent
44. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
...or web interfaces with auto-generated
user interfaces based on the DDL
Writing an Agent
45. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
New in MCollective 2
• Entirely rewritten messaging layer
• Asynchronous mode
• Additional non-broadcast based comms
• Reliable messaging with TTLs and Queues
• Batched mode to affect nodes in groups
• Improved RabbitMQ and ActiveMQ support
• Pluggable discovery against your own source of truth
• Data plugins for discovery, data query and ACLs
• Plugin Generators and Packaging
• Improved Security
• MS Windows Support
• DDL based pluggableValidation on clients and servers