O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Kafka Pluggable Authorization for Enterprise Security (Anna Kepler, Viasat) Kafka Summit NYC 2019

281 visualizações

Publicada em

At Viasat, Kafka is a backbone for a multi-tenant streaming platform that transports data for 1000 streams and used by more than 60 teams in a production environment. Role-based access control to the sensitive data is an essential requirement for our customers who must comply with a variety of regulations including GDPR. Kafka ships with a pluggable Authorizer that can control access to resources like cluster, topic or consumer group. However, maintaining ACLs in the large multi-tenant deployment can be support-intensive. At Viasat, we developed a custom Kafka Authorizer and Role Manager application that integrates our Kafka cluster with Viasat’s internal LDAP services. The presentation will cover how we designed and built Kafka LDAP Authorizer, which allows us to control resources within the cluster as well as services built around Kafka. We apply our permissions model to our data forwarders, ETL jobs, and stream processing. We will also share how we achieved a stress free migration to secure infrastructure without interruption to the production data flow. Our secure deployment model accomplishes multiple goals: – Integration into an LDAP central authentication system. – Use of the same authorization service to control permissions to data in Kafka as well as services built around Kafka. – Delegation of permissions control to the security officers on the teams using the service. – Detailed audit and breach notifications based on the metrics produced by the custom authorizer. We plan to open source our custom Kafka Authorizer.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Kafka Pluggable Authorization for Enterprise Security (Anna Kepler, Viasat) Kafka Summit NYC 2019

  1. 1. Kafka Pluggable Authorizer for Enterprise Security Anna Kepler Data Engineer
  2. 2. Data Security at Scale is Hard
  3. 3. Databus Streaming Platform
  4. 4. Shifting Objective Over Time Data Democratization Fast Customer Onboarding Self-Service High Volume Stream Processing 2014 4 teams 60 streams
  5. 5. Shifting Objective Over Time Data Security Data Governance Accountability2019 50+ teams 1,000+ streams
  6. 6. Default Kafka® Authorization authorizer.class.name = kafka.security.auth.SimpleAclAuthorizer allow.everyone.if.no.acl.found=true super.users=User:Bob;User:Alice bin/kafka-acls --add --allow-principal User:Bob --producer --topic test-topic
  7. 7. Databus Kafka Authorization authorizer.class.name = com.viasat.databus.DatabusKafkaAuthorizer role.manager.url = https://roles.visat.io permissions.expiration.sec = 60
  8. 8. Role Manager Service Endpoints /tenancies /resources /subjects /capabilities
  9. 9. Role Manager cli COMMANDS: version Get version information token Get a JWT for authentication tenancy Interact with tenancies resource Interact with resources capability Interact with capabilities subject Interact with subjects help, h Shows a list of commands role capability list –r stream:my-stream
  10. 10. Working with Role Manager { "id": "tenancy:team-awesome", "groups": [ { "stripe": ”team-awesome", "group": ”team-awesome-admins", "capabilities": [ "read", "write", "describe", "modify", "delete" ] } ] }
  11. 11. Working with Role Manager { "id": "tenancy:team-awesome", "groups": [ { "stripe": "team-awesome", "group": "team-awesome-readers", "capabilities": [ ”read", "describe" ] } ] }
  12. 12. Granular Permissions { "fromSubjectId": "Bob”, "toResourceId": "stream:shared-stream", "action": "read" } # With the cli role capability create capability.json
  13. 13. Why do it Integration into Central Authentication System Delegation of controls to team admins REST API used by various components in the platform In-depth monitoring
  14. 14. Thank you Anna Kepler Data Engineer, Viasat https://www.linkedin.com/in/akepler https://github.com/Viasat https://careers.viasat.com/