O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

FOSDEM19 MySQL Component Infrastructure

Why ? How it works ? How do you use it ?

  • Entre para ver os comentários

FOSDEM19 MySQL Component Infrastructure

  1. 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL 8.0 Component Infrastructure Georgi “Joro” Kodinov Team Lead, MySQL SrvGen Team Why ? What's in it ? What's next ? How to use it ?
  2. 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Georgi “Joro” Kodinov, MySQL @ Oracle  Server General Team Lead  Works on MySQL since 2006  Specializes in:  Security  Client/server protocol  Performance monitoring  Component infrastructure  Loves history, diverse world cultures, gardening  A devoted Formula 1 fan (Go, Vettel !)
  3. 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Agenda 3  Why ?  Architecture  The inventory  How to write my own …  Tips & tricks  What’s next ? Homework !
  4. 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Components: Why ? • Too much technical debt accumulated in plugins • Simpler and more extensible infrastructure • Better code isolation and encapsulation • Explicit dependencies • All components are equal: can call and can be called 4
  5. 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Component Infrastructure Architecture 5
  6. 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Server Component Server Binary Server Functionality Persisted Loader File Scheme Implementation MySQL Server Modularization: The Big Picture The Minimal Chassis Implementation 1 Implementation 2 … Registry Component 1 Component 2 … Dynamic Loader Component 1 Component 2 Component 3 Implementation 1 Implementation 2 Plugins Plugin Services Plugin APIs
  7. 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Components: Terminology 7 Component Code Service Implementations Contains Services Implement Registry Register Service References Dynamic Loader Load Unload Persisted Dynamic Loader Encapsulate INSTALL COMPONENT UNINSTALL COMPONENT UDFs Performance Schema System Variables, etc.
  8. 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic Concepts: A Component • A container for code. – Can be an executable or a shared library – As a compromise it can be a static library too, but that’s making the lines fuzzy, so NO • Has init() and deinit() methods (called by the dyloader) – Macros always add a registry reference • Can consume services from the registry – Should not access code in other components in any other way (linker) ! – Can have the dyloader fill these references in at load time • Can provide service implementations to the registry – Can have the dyloader register these at load time 8
  9. 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic Concepts: A Service • An ABSTRACT NAMED interface definition ! – Although it can’t be in the registry without at least one implementation • Can have multiple implementations: – One implementation is default at any time • It’s STATELESS ! – If you want state (a.k.a. data instances) you need to provide factories for these and use handles • Is an IMMUTABLE definition – I.e. if you need to add a method or change any definition of any parameter you need a new name for the service. 9
  10. 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic Concepts: A Service Contd. • Has a public header file to describe the pointer returned – The header must be self-sufficient (include all of the headers it depends on) • It better (controversial, but explained later): – use “simple” data types. A pointer to a 200+ member structure is a RED flag ! – use inout parameters and reserve return value for state. 10
  11. 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic Concepts: A Service Implementation • A NAMED implementation of an abstract service interface registered in the registry – The name is prefixed with the service name and delimited with a dot. E.g. foo.bar is bar’s implementation of the foo service. – The implementation name is usually the component name ! • Resides in a component • Pointer to it is stored in and returned by the registry – Reference counted ! • Usually a C structure with a bunch of function pointers – Defined in the service header, registry is agnostic to that. Macros provided to do it this way 11
  12. 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic Concepts: A Related Service Implementation • A normal service implementation • Compatible with another service implementation – Can operate on the same objects as the other service implementation • Allows implementing sets of services – And breaking large APIs in small logically independent sub-parts • Usually residing in the same component as the other service implementation. • Example: registry query and registry registration services 12
  13. 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic concepts: The Registry • A set of services to manipulate a global map of services. • Does reference counting on service implementations • Allows one to: – Register service implementations – Query for implementations by name and get a counted reference – Release a counted reference to a service implementation – Enumerate the registered service implementations and their metadata (name/value) – Search for related implementations to a service already acquired • Usually the part that’s bootstrapped first as it depends on no services 13
  14. 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic Concepts: The Dynamic Loader • Requires the registry • A single global ordered list of component generations. • Loads and unloads component generations – Handles registration/unregistration for the service implementations provided by component(s) • Can load multiple components in a single “generation” to handle circular dependencies • No persistent state • Loading is done via calling “scheme” service implementations – Currently only “file://” 14
  15. 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Basic Concepts: The Persisted Dynamic Loader • A separate implementation (usually in a different component) • Requires the Dynamic loader • Ensures persistency for the Dynamic loader: – remembers the sequence of component generations loaded on a persistent storage – re-loads the stored sequence on startup into the dynamic loader • The current implementation inside the server component uses a system table and provides SQL commands. 15
  16. 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Concepts: The Minimal Chassis • A static library • The basic component infrastructure – The registry – The dynamic loader • Enough to bootstrap a functional registry and a dynamic loader • Can load any compliant component, given its service dependencies are met by service implementations registered in the registry • Great for (unit) testing ! • Allows using the component infrastructure outside of the server binary 16
  17. 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Component Services Inventory 17
  18. 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Component Services: The Inventory • Service Registry • Dynamic Loader • Error logging • System variables • Status variables • User defined functions • Performance Schema – New tables and Instrumentation • Security Context • Password validation • Runtime errors • Collated Strings • Etc. – 92 service related headers ! 18
  19. 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How To Write Your Own Components And Services 19
  20. 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How To Write Your Own Component/Service • Use the utility macros • Copy some of the existing components – plenty of examples, specially in the tests • Copy some of the existing services – The services are done a bit differently in the server component • Compile using a server source distribution • Be careful ! – No crash protection, No security checks ! 20 The Executive Summary
  21. 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Relevant Source Tree Directories 21 Root Components Library_mysys Mysql_server C1 C2 Include MySQL Components Services Library_mysys sql Server_component Agenda:  3d party Component Code  Minimal Chassis  Server Component Specific  Implementation macros
  22. 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Tutorial: How To Write A Component #include <mysql/components/component_implementation.h> #include <mysql/components/service_implementation.h> #include <mysql/components/services/udf_registration.h> REQUIRES_SERVICE_PLACEHOLDER(udf_registration); // my code static mysql_service_status_t deinit() { return false; } static mysql_service_status_t init() { return false; } 22 Step 1: Add your component code: components/comp1/comp1.cc
  23. 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Tutorial: How To Write A Component BEGIN_COMPONENT_PROVIDES(comp1) END_COMPONENT_PROVIDES(); BEGIN_COMPONENT_REQUIRES(comp1) REQUIRES_SERVICE(udf_registration), END_COMPONENT_REQUIRES(); BEGIN_COMPONENT_METADATA(comp1) METADATA("mysql.author", "Oracle Corporation"), METADATA("mysql.license", "GPL"), END_COMPONENT_METADATA(); DECLARE_COMPONENT(comp1, “comp1") init, deinit END_DECLARE_COMPONENT(); DECLARE_LIBRARY_COMPONENTS &COMPONENT_REF(comp1) END_DECLARE_LIBRARY_COMPONENTS 23 Step 2: Add the component boilerplate: components/comp1/comp1.cc
  24. 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Tutorial: How To Write A Component MYSQL_ADD_COMPONENT(comp1 comp1.cc TEST MODULE) 24 Step 3: Add a make file: components/comp1/CMakeFile.txt
  25. 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Tutorial: How To Create a Service #include <mysql/components/service.h> /** @ingroup group_components_services_inventory */ BEGIN_SERVICE_DEFINITION(host_application_signal) /** Send a signal. */ DECLARE_BOOL_METHOD(signal, (int signal_no, void *arg)); END_SERVICE_DEFINITION(host_application_signal) 25 Step 1: Add a Service definition in include/mysql/components/services/
  26. 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Tutorial: How To Create a Service #include <mysql/components/service_implementation.h> #include <mysql/components/services/host_application_signal.h> /** An implementation of host application signal service for the mysql server as a host application. */ class mysql_component_host_application_signal_imp { public: static DEFINE_BOOL_METHOD(signal, (int signal_no, void *arg)); }; 26 Step 2: OPTIONAL: Add an implementation header (server: sql/server_component/)
  27. 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Tutorial: How To Create a Service #include <mysql/components/service_implementation.h> #include <sql/server_component/host_application_signal_imp.h> #include <components/mysql_server/server_component.h> /** An implementation of host application signal service for the mysql server as a host application. */ class mysql_component_host_application_signal_imp { public: static DEFINE_BOOL_METHOD(signal, (int signal_no, void *arg)); }; 27 Step 3: Add an implementation file (server: sql/server_component/, your component dir)
  28. 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Tutorial: How To Create a Service #include <sql/server_component/host_application_signal_imp.h> … BEGIN_SERVICE_IMPLEMENTATION(comp1, host_application_signal) mysql_component_host_application_signal_imp::signal END_SERVICE_IMPLEMENTATION(); … BEGIN_COMPONENT_PROVIDES(comp1) PROVIDES_SERVICE(comp1, host_application_signal) END_COMPONENT_PROVIDES(); 28 Step 4: Add it to a component. (components/mysql_server/server_component.cc if server)
  29. 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Component Services Tips And Tricks 29
  30. 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | C++ Way • Obj *inst; • inst = new Obj(12); • int ret; • ret = Inst->m1(1); • delete inst; The Handle Way • HOBJ hobj; • obj_svc->create(&hobj); – obj_svc->init(hobj, 12); • int ret; • obj_svc->m1(hobj, 1, &ret); • obj_svc->dispose(hobj); 30 Working with State: Handles
  31. 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Single Service • HOBJ hobj; • obj_svc->create(&hobj); – obj_svc->init(hobj, 12); • int ret; • obj_svc->m1(hobj, 1, &ret); • obj_svc->dispose(hobj); Related Services • HOBJ hobj; • obj_factory_svc->create(&hobj); – obj_integer_init_svc->init(hobj, 12); • int ret; • obj_svc->m1(hobj, 1, &ret); • obj_factory_svc->dispose(hobj); 31 Working with State: Related Services
  32. 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 32 Related Service Sets: Why ? C1 obj_svc.C1 obj_factory_svc.C1 obj_svc.C1 obj_svc.C1 obj_svc obj_svc.C1 obj_factory_svc.C1 obj_factory_svc.C1 obj_factory_svc obj_factory_svc.C2 obj_svc.C2 obj_svc.C2 obj_factory_svc.C2 obj_factory_svc.C2 C2 obj_svc.C2 obj_factory_svc.C2
  33. 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 33 Overloading C1 obj_svc.C1 obj_factory_svc.C1 my_service<SERVICE_TYPE(obj_svc)> obj_svc(“obj_svc”, registry); C2 obj_svc.C2 obj_factory_svc.C2 my_service<SERVICE_TYPE(obj_svc)> obj_svc2(“obj_svc”, registry); obj_svc
  34. 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Random Guidelines • Try to use the component infrastructure macros • Use basic parameter types. Anything more complex goes into an ANONYMOUS(!) handle and has service(s) to drive it • Taking references is “expensive” (global structure). Reuse references. • Do not make high volume calls services. Price to call a function pointer may be too much ! • Make sure to avoid reference or handle leaks ! • Always use the designated destructor for handles ! • Keep the service reference with the consumer. 34
  35. 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | DEFINE_SERVICE_HANDLE(HOBJ); BEGIN_SERVICE_DEFINITION(obj_svc) DECLARE_BOOL_METHOD(create, (HOBJ *outHandle)); DECLARE_BOOL_METHOD(dispose, (HOBJ outHandle)); DECLARE_BOOL_METHOD(init, (HOBJ handle, int arg)); DECLARE_BOOL_METHOD(m1, (HOBJ handle, int arg, int *outRet)); END_SERVICE_DEFINITION(obj_svc) DEFINE_SERVICE_HANDLE(HOBJ); BEGIN_SERVICE_DEFINITION(obj_factory_svc) DECLARE_BOOL_METHOD(create, (HOBJ *outHandle)); DECLARE_BOOL_METHOD(dispose, (HOBJ outHandle)); END_SERVICE_DEFINITION(obj_factory_svc) BEGIN_SERVICE_DEFINITION(obj_integer_init_svc) DECLARE_BOOL_METHOD(init, (HOBJ handle, int arg)); END_SERVICE_DEFINITION(obj_integer_init_svc) BEGIN_SERVICE_DEFINITION(obj_svc) DECLARE_BOOL_METHOD(m1, (HOBJ handle, int arg, int *outRet)); END_SERVICE_DEFINITION(obj_svc) 35 Working with State: Related Services Definitions
  36. 36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | my_service<SERVICE_TYPE(obj_svc)> obj_svc(“obj_svc”, registry); my_service<SERVICE_TYPE(obj_svc)> obj_svc(“obj_svc”, registry); my_service<SERVICE_TYPE(obj_factory_svc)> obj_factory_svc(“obj_factory_svc”, registry); my_service<SERVICE_TYPE(obj_integer_init_svc)> obj_int_init_svc(“obj_integer_init_svc”, registry); 36 Working with State: Related Services Use
  37. 37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | my_service<SERVICE_TYPE(obj_svc)> obj_svc(“obj_svc.c1”, registry); my_service<SERVICE_TYPE(obj_svc)> obj_svc(“obj_svc.c1”, registry); my_service<SERVICE_TYPE(obj_factory_svc)> obj_factory_svc(“obj_factory_svc.c1”, registry); my_service<SERVICE_TYPE(obj_integer_init_svc)> obj_int_init_svc(“obj_integer_init_svc.c1”, registry); 37 Working with State: 2nd Try
  38. 38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | my_service<SERVICE_TYPE(obj_svc)> obj_svc(“obj_svc”, registry); my_service<SERVICE_TYPE(obj_svc)> obj_svc(“obj_svc”, registry); my_service<SERVICE_TYPE(obj_factory_svc)> obj_factory_svc(“obj_factory_svc”, obj_svc, registry); my_service<SERVICE_TYPE(obj_integer_init_svc)> obj_int_init_svc(“obj_integer_init_svc”, obj_svc, registry); 38 Working with State: Final !
  39. 39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | What’s Next ? 39
  40. 40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | What’s Ahead ? • Migration of major plugin APIs and plugin services into components – Current State • No new plugin APIs • No new plugin service APIs – Goal • Deprecate the plugins ! • Further sub-division of the (now extremely chunky) server component 40
  41. 41. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 41

×