Introduction to D programming language at Weka.IO
•
4 gostaram•3,329 visualizações
Introduction and basic use cases of the D programming language at Weka.IO, a stealth mode storage startup.
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Introduction to D programming language at Weka.IO
Semelhante a Introduction to D programming language at Weka.IO (20)
Último
Último (20)
Introduction to D programming language at Weka.IO
- 1. 1 D in real life By: Liran Zvibel, Weka.IO CTO
- 2. 2D in real life • Defining the future of software defined, scale out storage for the cloud-‐based datacenter • VC backed company (NVP, Gemini), largest round-‐A of 2014 • Large, strong engineering team, many XIV veterans • Founded by Omri Palmon, Maor Ben-‐Dayan, Liran Zvibel About Weka.IO
- 3. • Extremely reliable • High performance data path • Complicated “control plane”/“Management code” • Distributed nature due to HA requirements • Low level interaction with HW devices • Some kernel-‐level code, some assembly • Language has to be efficient to program, and fit for large projects 3 Storage system requirements D in real life
- 4. • C codebase • A lot of auto-‐generated code from XML for RPC, clustering code and external APIs (CLI, GUI) - Requires a complicated build process - Difficult to understand where “magic” code came from • Our own implementation of Classes/polymorphism and templates 4 What did we do at XIV? D in real life
- 5. • Data-‐path in C++, control path in Python (or Java, C#, Scala, etc) - Deciding whether to run in the same process or via RPC, both options have downsides • All in C++ - Outside the scope of this talk, C++ has many downsides writing very large projects, many pitfalls • Google Go - Can’t disable GC, no generic programming, inheritance, no C ABI. Not performant • Mozilla Rust - Currently a moving target 5 Some other possible options D in real life
- 6. 6D in real life “Often, programming teams will resort to a hybrid approach, where they will mix Python and C++, trying to get the productivity of Python and the performance of C++.The frequency of this approach indicates that there is a large unmet need in the programming language department.” “D intends to fill that need. It combines the ability to do low-level manipulation of the machine with the latest technologies in building reliable, maintainable, portable, high-level code. D has moved well ahead of any other language in its abilities to support and integrate multiple paradigms like imperative, OOP, and generic programming.” — Walter Bright, Co-designer of D
- 7. • Created ’99 by Walter Bright, Andrei Alexandrescu, as a C++ replacement • Work on D2 started 2007, considered stable in late 2010. • Modern programming language with native efficiency, but leveraging 20 years of experience making mistakes while designing C++ and Java. • Combines the power of C++, but with the usability of languages like Python, Java and C#. • Has several compilers : dmd, gdc, ldc 7 What is D? D in real life
- 8. • Smaller community • Stability reminds C++ of the early 2000s • Binary compatible to C, no source compatibility - We must “translate” h files to d - Inability to inline c code (that would otherwise be inlined in C++) • Compilation time grows if many templates are instantiated across different compilation units • Missing keyword arguments when calling functions (a la python) 8 D down sides D in real life
- 9. • range is the D equivalent of an iterable • InputRange requires 3 basic methods: - bool empty; - T front; - void popFront(); • foreach(elem; range) • foreach(k, v; dict) • foreach(idx, ref e; array) 9 Ranges and Algorithms D in real life
- 10. Output copy(Input, Output)(ref Input source, ref Output sink) if (isInputRange!Input && isOutputRange!(Output, ElementType!Input)) { foreach(c; source) sink.put(c); return sink; } 10 Simple Algorithm D in real life
- 11. Symbol[] getSymbols(string filename) { return executeShell("nm -S " ~ filename) .output .splitLines .map!” a.split()" .filter!"4 == a.length" .map!((x) => new Symbol(x)) .filter!`a.typeFits(["d", "D", "v", "V"])` .array; } 11 UFCS and Pipeline programming D in real life
- 13. • It supports raw pointers if required • Has native C ABI so linking with existing libraries is easier than C++ • Supports static local variables • Has Templates (Generic programming) - They are much easier and flexible than in C++ - D supports compile time execution — CTFE • Supports Class hierarchies and polymorphism • Statically typed, but with very strong type inference • Supports operator overloading • Inline assembler support 13 D is similar to C++ D in real life
- 14. • It supports Garbage Collection - this is the default, but you can opt-‐out, do pointer arithmetic - Compiler helps you verify GC is indeed not being used - Allows prototyping in a high level way, then optimizing GC out where needed • It has inner-‐classes • It has delegates • Vast support for attributes • Very strong type introspection/reflection allowing “duck typing” 14 Python that compiles? D in real life
- 15. @StatsCategory(“fs_driver_user"): @RateQuery @StatUnit("Ops/Sec") { @("Number of read operations per second") IncrementalStat READS; @("Number of write operations per second") IncrementalStat WRITES; } @StatUnit("Microseconds", 0.1) { @("Average latency of READ operations") @MeasuredQuery!`{READS}` TimerStat READ_LATENCY; @("Average latency of WRITE operations") @MeasuredQuery!`{WRITES}` TimerStat WRITE_LATENCY } 15 Attributes usage D in real life
- 16. • It has modules, packages and imports system • Native Array, Tuple and Associative Array types with bounds checking • Supports nested functions and closures - Also very strong anonymous functions/delegates (lambda with a closure) • Has pure (enforced!) functional programming subset - Higher order functions: reduce, map, filter, compose, curry - supports lazy evaluation of function parameters 16 Python that compiles? D in real life
- 17. auto dumpField(U)(U field) { static if (__traits(hasMember, field, "toJSON")) { return field.toJSON(); } else static if (is(typeof({auto x = JSONValue(field);}))) { return field; } else static if (is(typeof(structToJSON!U))) { return structToJSON(field); } else static if (isArray!U) { return field.map!(dumpField!(typeof(field[0]))).array; } else static if (is(typeof({text(field);}))) { return text(field); } else { return "<unjsonable>"; } } 17 Introspection D in real life
- 18. JSONValue structToJSON(T)(ref T obj) if (is(T == struct) || is(T == class)) { string[string] dummy; auto jv = JSONValue(dummy); foreach(i, U; typeof(obj.tupleof)) { enum NAME = __traits(identifier, obj.tupleof[i]); jv.object[NAME] = dumpField(obj.tupleof[i]); } return jv; } 18 Introspection D in real life
- 19. struct RobinHashTable(K, V, size_t maxLength) { static if (NumEntries < ushort.max-1) { alias CellIdx = ushort; } else { alias CellIdx = uint; } static if (K.sizeof % 8 < 7) { align(8) struct KV { align(1): K k; ubyte cellData; align(8): V v; } } 19 Compile time magic D in real life else { align(8) struct KV { K k; align(8): V v; align(1): ubyte cellData; } }
- 20. • All variables are initialized to default values • Contract programming • Casts are explicit, with special keyword • Exceptions support • mixin support • Concurrent programming support - native thread local storage - native locks and message passing mechanisms 20 D is for real SW engineers D in real life
- 21. • static if • scope(exit/success/failure) statement • Unit testing support • -‐debug and -‐release default support • Versioned compilation (special debug case) • @safe/@trusted/@system code • Documentation comments 21 D is for real SW engineers D in real life
- 22. void testing(T...)(T values) { writeln("Called with ", values.length, " arguments."); // Access each index and each value foreach (i, value; values) { writeln(i, ": ", typeid(T[i]), " ", value); } } void main() { testing(5, "hello", 4.2); } Called with 3 arguments. 0: int 5 1: immutable(char)[] hello 2: double 4.2 22 (Hetero) Variadic functions D in real life
- 24. Questions? Peta Exa Zetta Yotta Xenna Weka (1030)
- 25. • Tracing system - space and cpu efficient - keep track of func enter/exit - Easy way to keep logs • RPC - Very efficient - Easy to use and declare • CLI • Statistics • Events (Telemetry) 25 Infrastructure D in real life