Walkthrough of the OCCI specification - for more information, a copy of the slides & transcript go to http://www.occi-wg.org for more information and contact details.
15. Mixins as Tags Resource Resource User defined Mixin Resource User-defined group/collection Resource Resource Resource
16. But… How to prevent Racecar capabilities (define by a Mixin) added to an SUV (Which is impossible)? How do we define the Type of a resource? Since this is all dynamic – how does the client a) know what’s supported and b) when? What about hierarchies?
46. Some things we couldn‘t demo Templates (a requirement for the compute resource) support for partial / full updates Filtering mechanisms based on Attributes & Categories Tags - lets tag the compute with “my-blog” and storage with “static-content” Handling of collection of Kinds and Mixins adding, removing, updating collections Performing actions on collections Batch operation (Multipart) Return code handling (200, 201, 202, and Error codes) All the nice HTTP stuff (Versioning, Caching, security, etc etc, etc, …) …
Many of you might know this slide – Surrounding our Logo are the main topics and goals for the work of the group. We’re a truly open group which defends and creates it’s own Cloud standard. The group consist of several people from commercial companies, people from research and academia and large community projects. Next to Interoperability, which of course is one of the main reason to do standards we focus on portability. The integration keyword is especially there since we do not strive to create a standalone standard but we follow the idea of integrating other standards. You might be aware of Standards like CDMI and OVF which play nicely with OCCI. Last but not least, and I personally think people miss this point some time is – that standards are also about innovation.
The OCCI-working group started back in 2009. Since then we had steady growth and delivered several documents. Starting with a requirements and use cases document we published our first specification in the beginning of 2010 and after two rounds of public comments we where able to deliver a stable & mature specification. Point here is that we have quite a background and can build upon the things we did in the past.
The last months have been especially interesting for SDOs. Next to initiatives like the cloud-standards.org wiki we have been involved with groups like NIST and SIENA which both strive to look into Standards for Clouds. Next to this the UK-Government has recommended the OCCI specification through their G-Cloud initiative . Main reason for this was the openness of the group and specification. Also the European Grid initiative which you might know is looking into OCCI at the moment.
What you can see on this diagram is where OCCI fits in a service provider domain. We sit next to the proprietary APIs – Note that we do not want to replace them – we wan to tackle the interoperability and integration topics – not all the fine detailed highly specialized APIs which a service provider can give. So the client talks through the OCCI interface using RESTful principals and currently HTTP. Behind the APIs is the Resource Management Framework followed by the Resource. Note that I said Resource – regardless of what kind of Resources (VMs, Emails in Inboxes etc)…
After these short introductions I wanted to start the comparison of the work which has been done in either groups. As said before both groups started about the same time in the past. And it might not be surprising that we indeed share a lot of the same ideas/thoughts. Although there are of course some differences since we took different approaches to solve obstacles.
Now this is the model OCCI uses to model the infrastructure. Something which is probably not directly clear is that this is an extension to our core model – which is also not shown here – but we’ll get to that later.
But again I can color the areas of interest – and for those of you who are interested in more detail I would encourage you to look at one of our documents of the specification. It’s called ‘OCCI Infrastructure’ and describes this model in detail. With state diagrams, description of all attributes, methods and so on. Note there is not a tight coupling between Network, Compute and Storage – this looseness is given by the linking mechanism
Now if you wonder what the other two objects are doing: Those are objects which can be used to link the compute, network and storage resources together. They are later also exposed as RESTful resources btw. They can have attributes and methods on there one. The idea behind this is that we can express certain semantics. E.g. Linking a compute and storage resource means semantically to mount a disk in a virtual machine. Now the question was where to describe the ‘mount point’? And we do this via there object instances. A similar example is the link between a network and compute resource. The network resource is the network adapter (aka eth0 in unix) and the link would have an attribute with the IP.
Now exactly these network resources and links brought up a lot of discussions in the OCCI working group. To explain the problem we had I want to use the following example. Normally a Level 2 Networking resource doesn’t do the trick and you need more functionalities. Eventually you want to be able use Level 3 and 4 type network resources. But on the other hand sometimes you want to express a L2 resources? So the question was do we either create 3 different classes/types to express the different types or do we find a better solution? 3 different classes/types isn’t the best idea since they all build upon each other and a L3 resource is basically a L2 resource with additional features. And since the same issue arose with several topics (also storage) related we came up with a different idea to solve this…
So we took a approach which you might be familiar with. Mixins! Many programming languages support them and basically for OCCI it’s a way to dynamically and remove functionalities to a resource. So taking out previous example we can simple ‘extend’ a Level 2 network resource and make it a Level 3 or Level 4 resource…OCCI allows not only for static extension (via inheritance) but also dynamic extension (via inheritance by composition aka the mixin mechanism)
These UML diagrams show this concept. Notice that the mixin itself is not instantiated but it just adds attributes and methods to the original class. Note that here we add an IpNetworkMixin to the Network resource. But we could add other Mixins as well. There can be multiple Mixins bound to a resource! And this overall gives us a flexible way handle the issues we had. And remember since links are classes/objects themselves we can add mixins to links as well!
Now we’re not only using Mixins for extending/adding capabilities to a resource but also to solve some other issues. Let’s pick up the previously mentioned Resource ‘templating’. Have a look at this example: At the left side is a compute resource – so a Virtual Machine – and a Mixin is applied to it. Called ‘ubuntu’ – And you guessed right this basically means that the virtual machine is a ubuntu machine. But the ubuntumixin doesn’t help us much. We as humans understand that ubuntu is an operating system and that it has a version number. Now you see two more classes in this diagram and this brings us to our next problem. By adding so much flexibility we need to have a hierarchy. And by hierarchy I mean a hierarchy between the mixins. So if you look at the ‘ubuntu’ mixin again it shows you that it is related to a os_templatemixin. And now the os_templatemixin finally tells us that ubuntu actually is an Operating System. This is what we call in OCCI ‘OS Templating’…Also note that in this case the Mixin "contract" involves provider-specific behaviour. Now going one step further…
Now this is the same resource – it still has the ubuntuos template assign. Bus now it also has a resource template assigned. A resource template is rather easy: It’s a Mixin which defines some ‘default’ values for the resource itself. Think of it as the Amazon EC2 instance types like small, medium, large. It you look at the UML diagram you will see that it tells us that the VM is a ubuntu machine running on a 64bit system with 2 cores and 1Gb or memory.
So next to extending resource capabilities and as templates,Mixins can be used to tag resources. Mixins can not only be dynamically added or removed from a resource class/type/kind – but also they can be added by the user to the service on the fly. E.g. think of the user case that the user wants to create a group/collection of all resource belonging to one service. For example a LAMP service consisting of 3 VMs. This collection can be expressed by defining a mixin and adding this mixins to all 3 VMs…
Don’t fear – we have answer to all questions in this slide! But the question is really how do we make sure that somebody only applies Mixins which actually match the type of the resource? If you have a SUV you can’t make it an racecar by just applying a Mixin. So we need type information. But if we have a ‘type system’ – how does the Client/Customer now? And also we need hierarchies! E.g. the Level 3 network mixin can only be applied to Level 2 network resource. And in case of templates we need to make sure that they are structured correctly.
So the answer to all this questions is OCCI’s core model. All things which I presented before build upon the core model which I will explain now. This is probably the most interesting part – since this is where all the stuff comes together As mentioned before this is also a document of the OCCI specification series – and if your’re interested you can read a lot about it and all the details – which I can’t all describe in this presentation – in the document called ‘OCCI Core’
The most import class of the core model which I’m going to show you in a few slides is the ‘Category’. The Category contains a term and scheme which are together the unique identifier of the category. It has a title/description and a set of attributes. Derived from the Category are to ‘specialized’ categories. The Kind and Mixin. We already discussed the Mixin and this class just represents a mixin within the system. The Kind class will later define the concrete type of a (compute, storage, network) resource. Kinds and Mixins add actions to the Category. So Kinds and Mixin in the end will have attributes and actions. Note the self relationships on the Kind and Mixin – those related associations setup the hierarchy in an instantiated system.
So Kind and Mixin derive from Category and have Actions. The Action class is also defined by a Category since an Action can have attributes. But an Action cannot have Actions. BTW the idea of having actions and not do anything by modyfing a state is nicely described in a blog post of Tim Bray (http://www.tbray.org/ongoing/When/200x/2009/03/20/Rest-Casuistry)…Now looking at the entities we see the entities Resource and Link which have a relationship. More important is that an Entity has exactly one kind (or speaking a type). But can have multiple Mixins. Now this is all not strictly UML…
This is a more strict and complete description of the core model.
Now I showed you the Infrastructure model and from there explained the core model which is our classification system. But actually you have to see it that the Infrastructure model is an extension of the Core model. It just adds more detail and information for a particular area of interest / In this case Infrastructure services. Overall the OCCI specification is very modular – and you can see it as a building block system. Independent from the Infrastructure and Core model is a way to represent the model and all the information through an Rendering. In this case a RESTful one…
– but you could also define a SOAP based Web Service rendering for OCCI. Or one using JSON or Ovf files…
One can take the blocks you like and build something different with it. For example we worked together with the DRMAA group inside of OGF to create a Job Submission model for Clusters…As you can see the core binds together all it’s extensions…But with all this dynamicall and modular way of operating we need a way so the client can discover the capabilities of the service he is talking too…
…So we introduced a query interface – you can ask a service about all the categories which are registered in the service. This will tell you what kind of service you can use and how to use it. For example it shows you which attributes and actions are available…It is pretty dynamic in it’s own way – since clients can here add the user defined mixin which I mentioned earlier as well…Now since I already talked about the rendering and there is a URI on the screen now I wanted to introduce the last part of the specification…
So the Rendering we have strives to be a truly RESTful one. We incorporate a lot of features from RFCs which related to HTTP. We do not particularly reinvent the wheel and take the ideas like CRUD and HATEOAS pretty serious. Overall this diagram should just show you what influenced our ideas. My guess is that I don’t need to go through all the ideas and thoughts of RESTful service here. Just note that OCCI’s rendering is in fact the specification how a a client can interact with the OCCI model through a RESTful service.
For historical reason (because the group could not decide on a fight over XML vs JSON vs text) we picked up an RFC which introduced a way to express Category information in the HTTP header through a Header field. Adding to that we introduced some more fields so we could express our model in the HTTP header. Issue with the headers is that they are small and the model can become big – so we decide to use the same grammar and ways to express the information and just put it in the HTTP body. We distinguish between the two renderings through the Content-type text/occi and text/plain. When using text/occi we suspect all the content in the header and just an ‘OK’ in the body. And vis-versa. We also have complete ABNF grammars setup so writing parsers and lexers for OCCI is a peace of cake. This is the default behavior. As noted before we could replace the rendering and simple add a JSON way of expressing our models. And as a quick side note we could also ‘post’ OVF files if wanted…We started to work on that…
This is a screenshot of the table of contents to show you that we heavily rely on HTTP and really reuse the features whenever possible. This includes the security features of HTTPS, Versioning, Content-Types, return codes and numerous other topics…
Let‘s start with an very simple example – the creation of a virtual machine...
This examples shows the creation of a very simple VM. Here we rely on the default values of the service provider – but next to that we use some templates (which is optional – and would not be required for this example). So we use the HTTP POST operation to create a new instance under /compute/; We define the content-type and Accept (*/* is the default text rendering in the HTTP body) header; And than define the type/kind the new resource should have – in this case compute. Last but not least the provider will return us a 201 OK...
Now we move on to a sligfhtly more complex setup (but still very simple) in which we want to use and demo more of OCCI‘s features. We‘ll create a VM with a network and storage resource bound to it – and as said while walking through the demo in detail – explain the capabilities of OCCI...
So the first operation a client (or even an cloud broker) wants to do is to check what capabilities the service offering provides. Capabilities are expressed through Categories in OCCI. All actions, links and kinds have corresponding categories which describe them. Now here we see that we do an get on the Query interface but along with the GET request we provide a Category – this is called filtering and tells the service provider to only show that category in this example. The service provider will return the compute category and describe the category it‘s attributes and actions. It also tells us where to find the resource instance of this type in the provider namespace. In this case it‘s compute – but other providers might put it under ‚vms‘...
Now that we know that a compute category exists (good for interop) we can create an instance. Nothing parcitulary interesting happens in this request. We do a normal HTTP poST operation on /compute/ (which we discovered earlier). Along with the request we provide the category – telling the service I want to create a resource of this kind/type. And we provide a link – this tells the provider to directly create a link to another resource when creation the compute resource. In this case it directly links to a network resource...
Now since we know there is a network resource availabel and that we link to it – we can add a mixin to the link between the compute and network – because currently this link is pretty dump. Think of it as just linking the network and compute resource – now we need to tell the service that this links actually is a Ip netowkring link. We do this by applying a mixin to the network link resource (BTW: we have 3 REST resources now – compute, network and a network link)...
Next to the networklink we also want to link to a storage device. Previously we showed that you can create links during the creation of compute resource – but now we show the creation of a standalone link directly. We simple do a PUT on /storage/link/1 – tell the service that it is an storagelink – and add a source and a target. The OCCI compatible service will now link the compute and storage resource through this link. Not that the storagelink – just like the previously mentioned network link – has a meaning – in this case it tells the compute where to mount the storage devices...
Now we can review the compute resource itself again and we‘ll see it has a type information – a bunch of attributes and than the links (for spacing issues the storage link is missing here....) ... Note that it also has a link to an action. An action is not an REST resource itself but just a URL you can use. Also note the rel attribute in the links which tells the client of what type the resource will be when i follow the link...
We can also review the network link itself and we‘ll now see that it has two categories. One defining the type/kind and one is a mixin. The kind categorory (networkinterface) defined the last 3 attribtues in this examples – the mixin adds the top 3 attributes...You also note that the client can tell which category defined the kind and which are mixins or actions...
Now let‘s trigger the start operation of a compute resource. We did get the URI to use from the previously mentioned link field in the compute resource rendering – now we just do a POST and provide the category definition of the action...
So the basic setup is done – Last step will be to demo that i can create a very simple tag and add the network and storage resource to it. What we‘ll do to create a user-defined category (aka mixin) we‘ll do a posT operation on the URI of the query interface and provide a category definition alongside- note that we also provide a location ‚/tag/‘ which tells the provider that i want to find the resource of the collection under the path example.com/tag later...
Now that we have created this mixin (which can also be deleted) we can add the storage and network resource. This request is pretty straight forward – we‘ll do a put operation and provide the URNs of the two resources.
Now adding a resource to a mixin collection also means that the mixin must be applied to the resource itself – so let‘s examine the storage resource – You‘ll notice that an additional Category shows up...Exactly that one which we defined earlier...
We can also do a get on ‚/tag/‘ now and we‘ll see using the Accept-header text/uri-list that we‘ll get back the two resource which we added to the collection...
We#ll and just to be complete here we show the destruction of resource as an example...Just use the HTTP DELETE operation...
Because of the time we could not demo all of OCCI‘s features. Here is a list of some of those – i want to specially mention the following: The differentiation between full and partial updates in OCCI, The future support for multipart, and our filtering mechanisms. Currently they are very limited on attribute and category filter – without the support for boolean operators – this might be a good idea to work on!
Probably most notable here are these two collaborations which we have in place through work registered. The OCCI-wg has countless other links to others groups through it’s member – be it other SDOs or Cloud related groups as well. Notable here is that CDMI and OCCI’ specification reference each other. And now new we have a collaboration with DMTF – and that’s also partly the reason why I give this presentation here for you.
In the context of the last slide. This is a screenshot of a document which will be released soon – which demos how OCCI/OVF and CDMI can work together...
So this is a list of ideas we are currently working on or planning to work on in the near future. Not all is settled – I was wanted to point out that we are looking into the direction of SLAs, Billing, Monitoring. Extending the QI and working on other renderings like JSON are topics which go hand in hand…if you guys are interested in any of this – or have other ideas – we would be more than happy to collaborate and drive this to the next level.
We also have a website which a bunch of information. Including a list of implementations (for clients & services btw), more details on implementations etc. You can visit our website under http://www.occi-wg.org
If you look at our website you will find a numerous of implementations. In different states and even some are missing…We have even some implementation which offer more functionalities than currently defined in the specification - like HTML renderings for nice websites, support for job submissions to clusters, key value stores...just to name some.
Next to a list of implementations you can find a list of tools for OCCI. We have a compliance test tool to test your service – online (hosted in the Cloud with GAE) and offline (with GUI). Next to these we offer an ANTLR grammar to make the creation of parsers for OCCI‘s current rendering simpler!
This is my last slide for today and I just quickly wanted to summarize some points. Again this is probably as good as a presentation can get on OCCI. The OCCI specification itself holds a ton of more detailed information. But don’t forget it’s a specification not an implementers guide (On which we will be working btw) To wrap it up I just wanted to notice the following things again:!! There are some real good parts in OCCI* QI - > discovery (for Cloud brokers)* it’s RESTfulness* Extensibility and flexibility* And last but not least that we’re not strictly bound to IaaS…
