P

1. Highly Available DNS and Request Routing Using Apache Cassandra A Real-World Introduction to Cassandra's Data Structures + Python's pyCassa module David Strauss / Founder + CTO / Pantheon Systems

4. Maintaining data persistence and replication should not be the DNS server's problem.

6. Built-in replication and high availability

7. Multi-master: writers don't need to understand the replication topology.

8. Data model similarity to DNS

9. Eventual consistency isn't a problem

11. High memory requirements

15. Some columns may be replicated before others.

16. Column names are unique in each row or SuperC.

28. Using data as sub-column name results in keeping the latest metadata for any record.

29. Visualizing as a dictionary { ”test.example.com”: { ”A”: { ”192.168.0.1”: {”ttl”: 86400} ”192.168.0.2”: {”ttl”: 86400} } ”MX”: { ”mail.example.com”: {”preference”: 10, ”ttl”: 86400} } } } Key Super Column Name Super Column Name Sub- Column Names Stored in Cassandra as a JSON-encoded sub-column value. Sub- Column Name Sub- Column Values

32. Code on GitHub: https://github.com/pantheon-systems/cassandra-dns/

34. Schema setup def install_schema(drop_first=False, rf=3): keyspace_name = "dns" sm = pycassa.system_manager .SystemManager("127.0.0.1:9160") [snip the drop_first implementation] sm.create_keyspace(keyspace_name, replication_factor=rf) sm.create_column_family(keyspace_name, "names", super=True, key_validation_class= pycassa.system_manager.UTF8_TYPE, comparator_type= pycassa.system_manager.UTF8_TYPE, default_validation_class= pycassa.system_manager.UTF8_TYPE)

35. The CassandraNames class class CassandraNames: def __init__(self): self.pool = pycassa.connect("dns") [rest on upcoming slides]

36. Adding new records def insert(self, fqdn, type, data, ttl=900, preference=None): # Connect to the ColumnFamily cf = pycassa.ColumnFamily(self.pool, "names") # Start the metadata with just a TTL metadata = {"ttl": int(ttl)} # Add in a ”preference” if requested. if preference is not None: metadata["preference"] = int(preference) # Actually perform the insertion. cf.insert(fqdn, {str(type): {data: json.dumps(metadata)}})

37. Reading records def lookup(self, fqdn, type=ANY): cf = pycassa.ColumnFamily(self.pool, "names") try: columns = {} if type == ANY: # Pull all types of records. columns = dict(cf.get(fqdn)) else: # Pull only one type of record. columns = {str(type): dict(cf.get(fqdn, super_column=str(type)))} # Convert the JSON metadata into valid Python data. [snip] return decoded_columns except pycassa.cassandra.ttypes.NotFoundException: # If no records exist for the FQDN or type, # fail gracefully. pass return {}

38. Deleting records def remove(self, fqdn, type=ANY, data=None): cf = pycassa.ColumnFamily(self.pool, "names") if type == ANY: # Delete all records for the FQDN. cf.remove(fqdn) elif data is None: # Delete all records of a certain type from the FQDN. cf.remove(fqdn, super_column=str(type)) else: # Delete all records for a certain type and data. cf.remove(fqdn, super_column=str(type), columns=[data])

39. Making it actually serve DNS class CassandraNamesResolver(common.ResolverBase): implements(interfaces.IResolver) def __init__(self): self.names = cassandranames.CassandraNames() common.ResolverBase.__init__(self) def _lookup(self, name, cls, type, timeout): log.msg(”Type %s records for name: %s" % (type, name)) all_types = self.names.lookup(name, type) results = [] authority = [] additional = [] [continued on next slide] Python's Twisted includes a complete DNS server implementation with a pluggable resolver base (IResolver and common.ResolverBase).

40. Making it actually serve DNS def _lookup(self, name, cls, type, timeout): [function started on previous slide] for type, records in all_types.items(): for data, metadata in records.items(): if type == A: payload = dns.Record_A(data) elif type == MX: payload = dns.Record_MX( metadata["preference"], data) elif type == NS: payload = dns.Record_NS(data) header = dns.RRHeader(name, type=type, payload=payload, ttl=metadata["ttl"], auth=True) results.append(header) return defer.succeed((results, authority, additional))

42. Use the Python shell to manage records

43. Import a BIND zone file on one server

44. Query the imported records on a different server

47. Finish the CNAME implementation (and other record types)

48. Consider a non-blocking library, like txCQL

49. GeoDNS using a Python GeoIP library

52. Code on GitHub: https://github.com/pantheon-systems/cassandra-dns/

53. Pantheon Systems is hiring engineers and developers in the San Francisco Bay Area