Presentation of the project XReplicator in EMC (26.04.2010). XReplicator is the library and utility provides replication of XAM storage systems. sourceforge.net/projects/xreplicator/

1. Developers: Burmak AlexanderMirov YuriyKovalev Igor Trainer: Alexeev Peter XAM XReplicator project

2. Contents About XAM Technology Why replication? What we’ve done Architecture XReplicator library XUtility Demonstration Resources

3. About XAM Technology eXtensible Access Method Technology providing a unified interface between applications and storage systems Industry standard API for Fixed Content Aware Storage Makes developers independent from hardware Easy migration between storages and applications

4. Why replication? Benefits of data replication Reliability Accessibility Performance Security Shortage of analogues of client applications for XAM Storage Systems replication

5. What we’ve done XReplicator library, which enables XAM storages replication XUtility, which uses XReplicator API and provides console user interface XOnce, a Domain-Specified Language to make easy query to storage

6. Architecture XReplicator XUtility XAM VIM VIM VIM

7. XReplicator library Replication – use RBaseAlgorithm or algorithm from third-party developers XSystems collection manipulation Direction of replication – mode parameter (in, out, inout) Constraint for XSystem synchronization data: XAM Query Set of XUID Configuration (XConfig) Logging

8. Replication – RBaseAlgorithm Uses replication history (state of storage after replication) Serialization/Deserialization to/from file Uses XAM import/export mechanism Solves “dirty” commit time problem Saves timeCommit and timeSynchronize parameters Solves conflicts of similar XSets Supports XReplicator modes and constraints

9. Logger Logs information about replication, exceptions and configuration changes Logs statistics about replication Two types of log format: XML Simple Supports log levels Log entry content – time, method, level, message Uses proxy-classes for incapsulation Log file(s) settings – path, append, size limit , count

10. XReplicator API architecture XConfig RAlgorithm XReplicator factory XHashSet InvocationHandler XReplicator XReplicator Module (Guice) XAM

11. XUtility features Remote control Banning enabled Multiple connections Scripting Reading text file with XUtility commands and executing it (like a command language) Program args enabled (e.g. for third party planners) Encrypt Blowfish encryption Corporate security raise Out of the box planner for replication

12. XUtility features Multithreading Every command runs in a different thread Enabled background threads (e.g remote control, planner) Commands to manage threads (e.g monitor, kill) Extensibility Dynamic plugins loading enabled (plugin = simple impl. of XOperation)

13. XUtility architecture Parameters XOp. Thread XOp. Thread Plugin 1 Plugin N XOperation Collector XUtility Operation Set XReplicator Thread Set XAM

14. XOnce features Implements XAM Query Similar SQL Uses MPS Flexible Saves time

15. XOnce in action XOnceAPI Select from myXSystem where "my.field" < 5 to myCollection XAM API XSet query = myXSystem.createXSet(XSet.MODE_RESTRICTED); query.createProperty(XSet.XAM_JOB_COMMAND, false, XSet.XAM_JOB_QUERY); XStreamqueryStream = query.createXStream( XSet.XAM_JOB_QUERY_COMMAND, false, XAMLibrary.TEXT_PLAIN_MIME_TYPE); queryStream.write ("select quot;.xset.xuidquot; where quot;my.fieldquot; < 5”.getBytes()); queryStream.close(); query.submitJob(); boolean finished = false; while (!finished) { if (query.containsField(XSet.XAM_JOB_ERROR)) if (query.getString(XSet.XAM_JOB_ERRORHEALTH).equals XSet.XAM_JOB_ERRORHEALTH_ERROR)) { query.haltJob(); throw new QueryException(); } if (query.getString(XSet.XAM_JOB_STATUS).equals (XSet.XAM_JOB_STATUS_COMPLETE)) { finished = true; } } //.........

16. XReplicator Demonstration

17. Resources www.snia.org – XAM 1.0.1 developer.emc.com – Many help about XAM groups.google.com/groups/xam-developers-group – Many help too code.google.com/p/google-guice/ – GuiceIoC and DI www.jetbrains.com/mps/ – MPS. Needed to create DSL XOnce maven.apache.org – Maven build manager www.junit.org – Unit tests

18. XReplicaton on Open Source XReplicator.sourceforge.net

19. Thank you for attention! Questions?