SlideShare uma empresa Scribd logo
1 de 35
Baixar para ler offline
®
© 2014 MapR Technologies 1
®
© 2014 MapR Technologies
Keys Botzum, Senior Principal Technologist
July 2014
®
© 2014 MapR Technologies 2
Agenda
•  What’s MapR
•  Why Secure Hadoop
•  Securing MapR Hadoop
•  Security beyond the core
®
© 2014 MapR Technologies 3
MapR Data Platform
Management
MapR Data Platform MAPR-DBMAPR-FS
APACHE HADOOP AND OSS ECOSYSTEM
Hue ...SharkImpalaDrill
Hive/
Stinger/
Tez
Sqoop
Storm SentrySparkSolrCascadingMahoutFlume
Oozie HBaseMapReduceYARNPigWhirrZookeeper
MapR Data Platform TABLESFILES MapR Data Platform MAPR-DBMAPR-FS
Patent
Pending
Enterprise-grade Security OperationalPerformance
• High availability
• Data protection
• Disaster recovery
• Standard file access
• Standard database
access
• Pluggable services
• Broad developer
support
• Enterprise security
authorization
• Wire-level
authentication
• Data governance
• Ability to support
predictive analytics,
real-time database
operations, and
support high arrival
rate data
• Ability to logically
divide a cluster to
support different
use cases, job
types, user groups,
and administrators
• Ability to deliver 2X
to 7X performance
• Consistent low
latency
Multi-tenancyInter-operability
MapR Distribution for Hadoop
®
© 2014 MapR Technologies 4
MapR: Best Product, Best Business, Best Customers, Best
Partners, Best Investors
Top Ranked
Exponential
Growth
500+
Customers Cloud Leaders
3X bookings Q1 ‘13 – Q1 ‘14
80% of accounts expand 3X
90% software licenses
<2% lifetime churn
>$1B
in incremental revenue
generated by 1 customer
®
© 2014 MapR Technologies 5
Why Secure Hadoop Now?
•  Historically security wasn’t a high priority
–  Reflection of the type of data and the type of organizations using Hadoop
•  Hadoop is now being used by more traditional firms as well as
organizations with high security requirements
–  Highly regulated
–  Sensitive data sets
–  People with experience with security in existing enterprise technologies (e.g.,
databases) are asking for the same in Hadoop
•  Think for a moment and imagine the value of the data in a Hadoop
cluster used as a data lake
–  Much valuable operational data about your customers, systems, sales, etc.
®
© 2014 MapR Technologies 6
Typical Hadoop Deployment Weaknesses
•  Client operating system is trusted to identify user (weak
authentication)
–  If I can compromise client, I can run jobs or access HDFS as anyone
–  Think about virtual machines with root access
•  Hadoop servers trust anyone that can reach them on the network
–  Could I falsify a data node, job tracker, etc.?
•  Hive Server runs as ‘system’ user
–  All Hive Server submitted jobs run as that ‘system’ user
•  Intruders can see and modify all network traffic
®
© 2014 MapR Technologies 7
Agenda
•  What’s MapR
•  Why Secure Hadoop
•  Securing MapR Hadoop
•  Security beyond the core
®
© 2014 MapR Technologies 8
MapR 3.1: Securing MapR Hadoop
•  Core goals
–  Leverage Apache Hadoop security functionality
–  Authenticate network traffic
•  Users authenticate
•  Servers authenticate to each other
•  Support but do not require Kerberos
–  Encrypt network traffic
–  Authorization
•  Integrate with existing authorization functionality
•  Enhance MapR Tables authorization with fine grained controls
–  Low barrier to entry
•  Low performance overhead
•  Simple and easy to administer
•  Support, but do not require Kerberos
®
© 2014 MapR Technologies 9
MapR Native Security
•  Hadoop security without Kerberos
–  But borrows heavily from Kerberos design
•  Password authentication is natively supported
•  Kerberos integration if desired
®
© 2014 MapR Technologies 10
Architecture
•  Shared secrets like Kerberos
–  Managed at cluster level
–  Two shared keys: cldb key and server key
•  Identity represented using a ticket which is issued by MapR
CLDB servers (Container Location DataBase)
®
© 2014 MapR Technologies 11
Tickets
•  A ticket represents a valid authenticated identity
•  Contains
–  An expiration time, renewal lifetime, and creation time
–  A randomly generated secret key
–  Information about the identity – userid, group ids
•  Signed and encrypted when issued by CLDB
–  CLDB key used for ‘permanent’ server tickets
–  Server key used for ephemeral tickets issued for users
•  A client authenticates to trusted servers using the ticket
®
© 2014 MapR Technologies 12
User Experience
•  User invokes maprlogin
–  maprlogin connects to CLDB (over https)
•  Provide userid & password (or Kerberos ticket) for validation by CLDB
–  Ticket is returned, saved in file in /tmp file and accessible only by owning user
– file name is /tmp/maprticket_<uid>
•  All processes automatically pick up ticket (nothing to do)
–  Java and C/C++ clients implicitly look for valid ticket and use it
–  Clients optionally use existing Kerberos identity to get MapR ticket
•  MapR PAM module
–  Optional MapR provided PAM module creates MapR tickets automatically
during Unix login
®
© 2014 MapR Technologies 13
Maprlogin
•  Primary user visible security tool
•  Actions are
–  password - authenticate to a MapR cluster using a valid password
–  kerberos - authenticate to a MapR cluster using Kerberos
–  print - print information on your existing credentials
–  authtest - test authentication as a generic client
–  end / logout - logout of cluster
–  renew - renew existing ticket
®
© 2014 MapR Technologies 14
Registry Neutral – Use the Operating System Security
•  User information is obtained using PAM and Linux pwent APIs
–  Fully pluggable – any registry works
–  Passwords are verified using PAM
–  User information obtained via Unix getpw* APIs
•  nsswitch controlled
–  Basically, if it works with Linux authentication, it should work with MapR
®
© 2014 MapR Technologies 15
CLI Example
$ hadoop fs -ls /
Bad connection to FS. command aborted. exception: failure to login: Unable
to obtain MapR credentials
$ maprlogin password
[Password for user 'fred' at cluster 'my.cluster.com': ]
MapR credentials of user 'fred' for cluster 'my.cluster.com' are written to
‘/tmp/maprticket_1001'
$ hadoop fs -ls /
Found 3 items
-rwxr-xr-x 3 mapr mapr 0 2013-12-10 13:25 /hbase
drwxr-xr-x - mapr mapr 1 2013-12-10 13:25 /user
drwxr-xr-x - mapr mapr 1 2013-12-10 13:25 /var
®
© 2014 MapR Technologies 16
Maprlogin – Under the Covers
maprlogin	
  
	
  
MapR	
  
CLDB	
  
1.	
  username/passwd	
  
sent	
  on	
  h;ps	
   LDAP/	
  
Kerberos
/NIS	
  
2.	
  uses	
  PAM	
  to	
  
authenDcate	
  
3.	
  	
  Dcket	
  +	
  user	
  
key	
  returned	
  
FileServer/	
  
CLDB	
  
4.	
  	
  Dcket	
  +	
  key	
  saved	
  in	
  file	
  in	
  /tmp	
  
hadoop	
  fs	
  –ls	
  /	
  
5.	
  	
  cmd	
  picks	
  up	
  
Dcket	
  +	
  key	
  from	
  
file	
  
6.	
  	
  client	
  sends	
  RPC	
  
encrypted	
  with	
  
user-­‐key	
  +	
  Dcket	
  	
  
7.	
  	
  server	
  decrypts	
  Dcket	
  to	
  
authenDcate	
  user	
  and	
  
checks	
  permissions	
  on	
  ACL	
  
®
© 2014 MapR Technologies 17
Client First Contact
•  Client sends the ticket and data encrypted using secret key
•  Receiving server
–  Extracts and decrypts ticket to obtain secret key
–  Checks expiration
–  Uses the secret key to decrypt the data
•  This proves that the client possesses the key that corresponds to the ticket
–  Extracts identity information from ticket and uses that for authorization
–  Returns encrypted response to client
•  MapR user identity is independent of host or operating system
identity
®
© 2014 MapR Technologies 18
Server First Contact
•  When a trusted server starts it uses a local server ticket to
authenticate to the CLDB
–  CLDB verifies the ticket’s authenticity using secret key
–  CLDB returns the server key that is used to create and validate user
tickets
–  The server is now a trusted member of the cluster
®
© 2014 MapR Technologies 19
Component Security
•  Security between MapR unique components (CLDB, file server, etc.)
is handled via changes to the MapR RPC layer
•  Apache components support pluggable security mechanisms –
typically SASL
–  We are providing a new mechanism called ‘maprsasl’
–  maprsasl secures communication following the same techniques as the MapR
RPC layer
•  Existing authorization code simply leverages the securely
authenticated identity
–  File access
–  Job submission
–  Queue ACLs
–  And so on …
®
© 2014 MapR Technologies 20
Example: Job Tracker Integration
JT can create user tickets. TT copies ticket to private job directory on local disk.
taskcontroller copies it to user private local disk dir and tasks set
MAPR_TICKET_LOCATION to that place.
JobClient JobTracker TaskTracker
submit
job
(maprsasl)
schedule
job
(maprsasl)
File system
1. JC copies
job conf securely to FS 4. TT launches job using ticket identity
3. TT fetches
ticket
2. JT creates
user ticket
®
© 2014 MapR Technologies 21
Out of the Box Defaults
•  User experience
–  Users authenticate using maprlogin and passwords
–  User ‘mapr’ is admin as always
•  User must authenticate however, OS identity irrelevant
–  Operating system identity (on or off cluster) no longer relevant to MapR
security
•  Obviously root user and ‘mapr’ user can read/write /opt/mapr
•  We’ve also tightened permissions for many directories under /opt/mapr
–  Web UIs require authentication
–  MapR CLIs require authentication
•  hadoop fs/mfs/jar/job/etc
•  maprcli
–  Any user can submit jobs, but can only admin their own jobs
®
© 2014 MapR Technologies 22
Out of the Box Defaults
•  Cluster operations
–  All MapR servers authenticate to each other
•  Most communication paths encrypted
–  All nodes share common maprserverticket
•  Nodes can only join cluster if they have maprserverticket
–  Self-signed wildcard certificates created for HTTPS traffic
•  ssl_keystore contains certificate and private key, ssl_truststore contains certificate
–  We set JVM system property: javax.net.ssl.trustStore
•  Used by Web UIs, MCS, and maprlogin to CLDB
•  Uses hostname command to get DNS domain for cluster and put that into
certificate
®
© 2014 MapR Technologies 23
Cryptography
•  Encrypted using current NIST standards
–  AES-256 in GCM mode for encryption and signing
•  http://en.wikipedia.org/wiki/Galois/Counter_Mode
•  NIST standard - http://csrc.nist.gov/publications/fips/fips140-2/fips1402annexa.pdf
–  Leverage Intel hardware encryption where available, software otherwise
•  Use the open source crypto++ library for our C++ cryptography –
http://cryptopp.com
•  Random number generation
–  Use secure random number generation as documented here
http://www.cryptopp.com/docs/ref/
class_auto_seeded_random_pool.html#_details
®
© 2014 MapR Technologies 24
Let’s Build a Secure Cluster!
Node 1
apt-get install mapr….
configure.sh –C … -Z … -secure –genkeys
–  Generates all needed keys for MapR-RPC as well as for HTTPS
Node N
apt-get install mapr….
scp rootORmapr@node1:/opt/mapr/conf/
{cldb.key,maprserverticket,ssl_keystore,ssl_truststore} /opt/mapr/conf
configure.sh –C … -Z … -secure
Clients
apt-get install mapr…
scp anyuser@nodeN:/opt/mapr/conf/ssl_truststore /opt/mapr/conf
configure.sh … -secure
®
© 2014 MapR Technologies 25
That Was Easy – What Happened?
•  Secret keys generated for encryption of RPC traffic
•  Certificates generated for HTTPS traffic
•  Configuration updated so that core components authenticate
traffic
–  Most traffic also encrypted
–  Bulk data transfer encryption optional on per file/directory basis
•  Web UIs switched to HTTPS
•  Web UIs require password authentication – JT/TT/MCS
•  Hadoop Java code switched to maprsasl
•  Authorization components now rely upon MapR identity
®
© 2014 MapR Technologies 26
Encrypted Shuffle (?)
•  No need to special case encrypting shuffle
•  MapR-FS is store for Map output
–  Shuffle inherits the same encryption, authentication, and authorization
functionality of the rest of MapR-FS
®
© 2014 MapR Technologies 27
Kerberos
•  Not required but can use
•  Kerberos SSO
–  Explicitly using ‘maprlogin kerberos’
–  Implicitly
•  If no MapR ticket available, client automatically detects and uses Kerberos ticket
and uses it to obtain MapR ticket
•  Kerberos SSO requires only
–  Kerberos client on CLDB and client machines
–  Kerberos identity only for CLDB – typically 3-5 CLDBs
•  No need to manage identities for every node
•  SPNEGO/Kerberos for Web UIs requires creating host Kerberos
identities
®
© 2014 MapR Technologies 28
Agenda
•  What’s MapR
•  Why Secure Hadoop
•  Securing MapR Hadoop
•  Security beyond the core
®
© 2014 MapR Technologies 29
Hadoop Map Reduce Clients
•  Many components simply generate Map Reduce jobs. As such
they implicitly leverage the security we’ve defined for Map
Reduce previously. They are:
–  Hive (except Hive Server)
–  Pig
–  Mahout
–  Sqoop
®
© 2014 MapR Technologies 30
Ecosystem Security
•  All ecosystem components run securely as well in a secure
MapR cluster
–  Some by default
–  Some with minor configuration
•  Most Web UIs enhanced to use userid & password
authentication and HTTPS
–  Can configure Kerberos SPNEGO, same as from Apache
®
© 2014 MapR Technologies 31
MapR Ecosystem Security – by Default
•  By default, out of the box when security enabled
–  Hive Server 2 supports password authentication
•  Can configure Kerberos and SSL function, same as from Apache, including
secure impersonation
–  Oozie supports MapR ticket authentication
•  Can configure Kerberos and SSL function, same as from Apache, including
secure impersonation
•  MapR Tables (HBase APIs) use native MapR security, no
configuration needed
•  HBase and Hive MetaServer require Kerberos to be secured
®
© 2014 MapR Technologies 32
MapR Tables Authorization
•  boolean logic constraints on access to M7 tables
–  Uses user & group information
–  Very powerful
•  ( u:bob | g:admins)
•  ( g:managers & ! g:restricted)
•  ( g: managers & g:businessunity) | g:executives
–  Settable at table, column, and column family level for various actions
–  Queries silently hide data you are not authorized to see
®
© 2014 MapR Technologies 33
MapR Hadoop Advantage
•  Vastly simpler
–  Core secured by default in one step
–  No requirement for Kerberos in core and associated complexity
•  Easier integration
–  Leverage existing Linux authentication (PAM and NSSwitch)
•  Faster
–  Leverage Intel AES hardware cryptography
®
© 2014 MapR Technologies 34
Further Reading
•  MapR
–  http://mapr.com
•  MapR Native Security
–  http://www.mapr.com/blog/getting-started-mapr-security-0
–  http://www.mapr.com/press-release/mapr-technologies-integrates-security-
into-hadoop
–  http://www.mapr.com/products/only-with-mapr/mapr-integrates-security-into-
hadoop
•  Adding Security to Apache Hadoop
–  http://hortonworks.com/wp-content/uploads/2011/10/security-
design_withCover-1.pdf
•  The Evolution of Hadoop’s Security Model
–  http://www.infoq.com/articles/HadoopSecurityModel/
®
© 2014 MapR Technologies 35
Q&A
@mapr maprtech
kbotzum@mapr.com
Engage with us!
MapR
maprtech
mapr-technologies

Mais conteúdo relacionado

Mais procurados

IBM Informix dynamic server and websphere MQ integration
IBM Informix dynamic server and websphere MQ  integrationIBM Informix dynamic server and websphere MQ  integration
IBM Informix dynamic server and websphere MQ integrationKeshav Murthy
 
Improving HDFS Availability with Hadoop RPC Quality of Service
Improving HDFS Availability with Hadoop RPC Quality of ServiceImproving HDFS Availability with Hadoop RPC Quality of Service
Improving HDFS Availability with Hadoop RPC Quality of ServiceMing Ma
 
Охота на уязвимости Hadoop
Охота на уязвимости HadoopОхота на уязвимости Hadoop
Охота на уязвимости HadoopPositive Hack Days
 
Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive Hortonworks
 
Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...
Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...
Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...CA API Management
 
Deploying Docker Containers at Scale with Mesos and Marathon
Deploying Docker Containers at Scale with Mesos and MarathonDeploying Docker Containers at Scale with Mesos and Marathon
Deploying Docker Containers at Scale with Mesos and MarathonDiscover Pinterest
 
Session 49 - Semantic metadata management practical
Session 49 - Semantic metadata management practical Session 49 - Semantic metadata management practical
Session 49 - Semantic metadata management practical ISSGC Summer School
 

Mais procurados (13)

Kafka Security
Kafka SecurityKafka Security
Kafka Security
 
cisco 7200
cisco 7200 cisco 7200
cisco 7200
 
IBM Informix dynamic server and websphere MQ integration
IBM Informix dynamic server and websphere MQ  integrationIBM Informix dynamic server and websphere MQ  integration
IBM Informix dynamic server and websphere MQ integration
 
Improving HDFS Availability with Hadoop RPC Quality of Service
Improving HDFS Availability with Hadoop RPC Quality of ServiceImproving HDFS Availability with Hadoop RPC Quality of Service
Improving HDFS Availability with Hadoop RPC Quality of Service
 
Google Compute and MapR
Google Compute and MapRGoogle Compute and MapR
Google Compute and MapR
 
Охота на уязвимости Hadoop
Охота на уязвимости HadoopОхота на уязвимости Hadoop
Охота на уязвимости Hadoop
 
Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive Hortonworks Technical Workshop: Interactive Query with Apache Hive
Hortonworks Technical Workshop: Interactive Query with Apache Hive
 
Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...
Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...
Seasonal Burst Handling Using Hybrid Cloud Infrastructure from Cloud Security...
 
Apache Kafka Security
Apache Kafka Security Apache Kafka Security
Apache Kafka Security
 
Deploying Docker Containers at Scale with Mesos and Marathon
Deploying Docker Containers at Scale with Mesos and MarathonDeploying Docker Containers at Scale with Mesos and Marathon
Deploying Docker Containers at Scale with Mesos and Marathon
 
Session 49 - Semantic metadata management practical
Session 49 - Semantic metadata management practical Session 49 - Semantic metadata management practical
Session 49 - Semantic metadata management practical
 
Hadoop Security Preview
Hadoop Security PreviewHadoop Security Preview
Hadoop Security Preview
 
Session9part2 Servers Detailed
Session9part2  Servers DetailedSession9part2  Servers Detailed
Session9part2 Servers Detailed
 

Semelhante a Securing Hadoop by Sr. Principal Technologist Keys Botzum

Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)MapR Technologies
 
Securing Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys BotzumSecuring Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys BotzumMapR Technologies
 
Hadoop Security Architecture
Hadoop Security ArchitectureHadoop Security Architecture
Hadoop Security ArchitectureOwen O'Malley
 
Webinar: What's new in CDAP 3.5?
Webinar: What's new in CDAP 3.5?Webinar: What's new in CDAP 3.5?
Webinar: What's new in CDAP 3.5?Cask Data
 
Clocker - The Docker Cloud Maker
Clocker - The Docker Cloud MakerClocker - The Docker Cloud Maker
Clocker - The Docker Cloud MakerAndrew Kennedy
 
Hadoop and Data Access Security
Hadoop and Data Access SecurityHadoop and Data Access Security
Hadoop and Data Access SecurityCloudera, Inc.
 
Walking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOS
Walking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOSWalking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOS
Walking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOSCody Thomas
 
大数据数据治理及数据安全
大数据数据治理及数据安全大数据数据治理及数据安全
大数据数据治理及数据安全Jianwei Li
 
Clocker: Managing Container Networking and Placement
Clocker: Managing Container Networking and PlacementClocker: Managing Container Networking and Placement
Clocker: Managing Container Networking and PlacementDocker, Inc.
 
Configuring a Secure, Multitenant Cluster for the Enterprise
Configuring a Secure, Multitenant Cluster for the EnterpriseConfiguring a Secure, Multitenant Cluster for the Enterprise
Configuring a Secure, Multitenant Cluster for the EnterpriseCloudera, Inc.
 
AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...
AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...
AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...IJNSA Journal
 
Risk Management for Data: Secured and Governed
Risk Management for Data: Secured and GovernedRisk Management for Data: Secured and Governed
Risk Management for Data: Secured and GovernedCloudera, Inc.
 
Fiware: Connecting to robots
Fiware: Connecting to robotsFiware: Connecting to robots
Fiware: Connecting to robotsJaime Martin Losa
 
Building Open Source Identity Management with FreeIPA
Building Open Source Identity Management with FreeIPABuilding Open Source Identity Management with FreeIPA
Building Open Source Identity Management with FreeIPALDAPCon
 

Semelhante a Securing Hadoop by Sr. Principal Technologist Keys Botzum (20)

Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)
 
Securing Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys BotzumSecuring Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys Botzum
 
Hadoop Security Architecture
Hadoop Security ArchitectureHadoop Security Architecture
Hadoop Security Architecture
 
Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon
Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheConTechnical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon
Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon
 
Hadoop security
Hadoop securityHadoop security
Hadoop security
 
Webinar: What's new in CDAP 3.5?
Webinar: What's new in CDAP 3.5?Webinar: What's new in CDAP 3.5?
Webinar: What's new in CDAP 3.5?
 
Clocker - The Docker Cloud Maker
Clocker - The Docker Cloud MakerClocker - The Docker Cloud Maker
Clocker - The Docker Cloud Maker
 
Hadoop and Data Access Security
Hadoop and Data Access SecurityHadoop and Data Access Security
Hadoop and Data Access Security
 
Walking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOS
Walking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOSWalking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOS
Walking the Bifrost: An Operator's Guide to Heimdal & Kerberos on macOS
 
大数据数据治理及数据安全
大数据数据治理及数据安全大数据数据治理及数据安全
大数据数据治理及数据安全
 
Clocker: Managing Container Networking and Placement
Clocker: Managing Container Networking and PlacementClocker: Managing Container Networking and Placement
Clocker: Managing Container Networking and Placement
 
Configuring a Secure, Multitenant Cluster for the Enterprise
Configuring a Secure, Multitenant Cluster for the EnterpriseConfiguring a Secure, Multitenant Cluster for the Enterprise
Configuring a Secure, Multitenant Cluster for the Enterprise
 
AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...
AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...
AUTHENTICATION MECHANISM ENHANCEMENT UTILISING SECURE REPOSITORY FOR PASSWORD...
 
Risk Management for Data: Secured and Governed
Risk Management for Data: Secured and GovernedRisk Management for Data: Secured and Governed
Risk Management for Data: Secured and Governed
 
20a installation
20a installation20a installation
20a installation
 
Keys for Success from Streams to Queries
Keys for Success from Streams to QueriesKeys for Success from Streams to Queries
Keys for Success from Streams to Queries
 
Fiware: Connecting to robots
Fiware: Connecting to robotsFiware: Connecting to robots
Fiware: Connecting to robots
 
Building Open Source Identity Management with FreeIPA
Building Open Source Identity Management with FreeIPABuilding Open Source Identity Management with FreeIPA
Building Open Source Identity Management with FreeIPA
 
Securing Spark Applications
Securing Spark ApplicationsSecuring Spark Applications
Securing Spark Applications
 
Null talk
Null talkNull talk
Null talk
 

Mais de MapR Technologies

Converging your data landscape
Converging your data landscapeConverging your data landscape
Converging your data landscapeMapR Technologies
 
ML Workshop 2: Machine Learning Model Comparison & Evaluation
ML Workshop 2: Machine Learning Model Comparison & EvaluationML Workshop 2: Machine Learning Model Comparison & Evaluation
ML Workshop 2: Machine Learning Model Comparison & EvaluationMapR Technologies
 
Self-Service Data Science for Leveraging ML & AI on All of Your Data
Self-Service Data Science for Leveraging ML & AI on All of Your DataSelf-Service Data Science for Leveraging ML & AI on All of Your Data
Self-Service Data Science for Leveraging ML & AI on All of Your DataMapR Technologies
 
Enabling Real-Time Business with Change Data Capture
Enabling Real-Time Business with Change Data CaptureEnabling Real-Time Business with Change Data Capture
Enabling Real-Time Business with Change Data CaptureMapR Technologies
 
Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...
Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...
Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...MapR Technologies
 
ML Workshop 1: A New Architecture for Machine Learning Logistics
ML Workshop 1: A New Architecture for Machine Learning LogisticsML Workshop 1: A New Architecture for Machine Learning Logistics
ML Workshop 1: A New Architecture for Machine Learning LogisticsMapR Technologies
 
Machine Learning Success: The Key to Easier Model Management
Machine Learning Success: The Key to Easier Model ManagementMachine Learning Success: The Key to Easier Model Management
Machine Learning Success: The Key to Easier Model ManagementMapR Technologies
 
Data Warehouse Modernization: Accelerating Time-To-Action
Data Warehouse Modernization: Accelerating Time-To-Action Data Warehouse Modernization: Accelerating Time-To-Action
Data Warehouse Modernization: Accelerating Time-To-Action MapR Technologies
 
Live Tutorial – Streaming Real-Time Events Using Apache APIs
Live Tutorial – Streaming Real-Time Events Using Apache APIsLive Tutorial – Streaming Real-Time Events Using Apache APIs
Live Tutorial – Streaming Real-Time Events Using Apache APIsMapR Technologies
 
Bringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale StorageBringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale StorageMapR Technologies
 
Live Machine Learning Tutorial: Churn Prediction
Live Machine Learning Tutorial: Churn PredictionLive Machine Learning Tutorial: Churn Prediction
Live Machine Learning Tutorial: Churn PredictionMapR Technologies
 
An Introduction to the MapR Converged Data Platform
An Introduction to the MapR Converged Data PlatformAn Introduction to the MapR Converged Data Platform
An Introduction to the MapR Converged Data PlatformMapR Technologies
 
How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...
How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...
How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...MapR Technologies
 
Best Practices for Data Convergence in Healthcare
Best Practices for Data Convergence in HealthcareBest Practices for Data Convergence in Healthcare
Best Practices for Data Convergence in HealthcareMapR Technologies
 
Geo-Distributed Big Data and Analytics
Geo-Distributed Big Data and AnalyticsGeo-Distributed Big Data and Analytics
Geo-Distributed Big Data and AnalyticsMapR Technologies
 
MapR Product Update - Spring 2017
MapR Product Update - Spring 2017MapR Product Update - Spring 2017
MapR Product Update - Spring 2017MapR Technologies
 
3 Benefits of Multi-Temperature Data Management for Data Analytics
3 Benefits of Multi-Temperature Data Management for Data Analytics3 Benefits of Multi-Temperature Data Management for Data Analytics
3 Benefits of Multi-Temperature Data Management for Data AnalyticsMapR Technologies
 
Cisco & MapR bring 3 Superpowers to SAP HANA Deployments
Cisco & MapR bring 3 Superpowers to SAP HANA DeploymentsCisco & MapR bring 3 Superpowers to SAP HANA Deployments
Cisco & MapR bring 3 Superpowers to SAP HANA DeploymentsMapR Technologies
 
MapR and Cisco Make IT Better
MapR and Cisco Make IT BetterMapR and Cisco Make IT Better
MapR and Cisco Make IT BetterMapR Technologies
 
Evolving from RDBMS to NoSQL + SQL
Evolving from RDBMS to NoSQL + SQLEvolving from RDBMS to NoSQL + SQL
Evolving from RDBMS to NoSQL + SQLMapR Technologies
 

Mais de MapR Technologies (20)

Converging your data landscape
Converging your data landscapeConverging your data landscape
Converging your data landscape
 
ML Workshop 2: Machine Learning Model Comparison & Evaluation
ML Workshop 2: Machine Learning Model Comparison & EvaluationML Workshop 2: Machine Learning Model Comparison & Evaluation
ML Workshop 2: Machine Learning Model Comparison & Evaluation
 
Self-Service Data Science for Leveraging ML & AI on All of Your Data
Self-Service Data Science for Leveraging ML & AI on All of Your DataSelf-Service Data Science for Leveraging ML & AI on All of Your Data
Self-Service Data Science for Leveraging ML & AI on All of Your Data
 
Enabling Real-Time Business with Change Data Capture
Enabling Real-Time Business with Change Data CaptureEnabling Real-Time Business with Change Data Capture
Enabling Real-Time Business with Change Data Capture
 
Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...
Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...
Machine Learning for Chickens, Autonomous Driving and a 3-year-old Who Won’t ...
 
ML Workshop 1: A New Architecture for Machine Learning Logistics
ML Workshop 1: A New Architecture for Machine Learning LogisticsML Workshop 1: A New Architecture for Machine Learning Logistics
ML Workshop 1: A New Architecture for Machine Learning Logistics
 
Machine Learning Success: The Key to Easier Model Management
Machine Learning Success: The Key to Easier Model ManagementMachine Learning Success: The Key to Easier Model Management
Machine Learning Success: The Key to Easier Model Management
 
Data Warehouse Modernization: Accelerating Time-To-Action
Data Warehouse Modernization: Accelerating Time-To-Action Data Warehouse Modernization: Accelerating Time-To-Action
Data Warehouse Modernization: Accelerating Time-To-Action
 
Live Tutorial – Streaming Real-Time Events Using Apache APIs
Live Tutorial – Streaming Real-Time Events Using Apache APIsLive Tutorial – Streaming Real-Time Events Using Apache APIs
Live Tutorial – Streaming Real-Time Events Using Apache APIs
 
Bringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale StorageBringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale Storage
 
Live Machine Learning Tutorial: Churn Prediction
Live Machine Learning Tutorial: Churn PredictionLive Machine Learning Tutorial: Churn Prediction
Live Machine Learning Tutorial: Churn Prediction
 
An Introduction to the MapR Converged Data Platform
An Introduction to the MapR Converged Data PlatformAn Introduction to the MapR Converged Data Platform
An Introduction to the MapR Converged Data Platform
 
How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...
How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...
How to Leverage the Cloud for Business Solutions | Strata Data Conference Lon...
 
Best Practices for Data Convergence in Healthcare
Best Practices for Data Convergence in HealthcareBest Practices for Data Convergence in Healthcare
Best Practices for Data Convergence in Healthcare
 
Geo-Distributed Big Data and Analytics
Geo-Distributed Big Data and AnalyticsGeo-Distributed Big Data and Analytics
Geo-Distributed Big Data and Analytics
 
MapR Product Update - Spring 2017
MapR Product Update - Spring 2017MapR Product Update - Spring 2017
MapR Product Update - Spring 2017
 
3 Benefits of Multi-Temperature Data Management for Data Analytics
3 Benefits of Multi-Temperature Data Management for Data Analytics3 Benefits of Multi-Temperature Data Management for Data Analytics
3 Benefits of Multi-Temperature Data Management for Data Analytics
 
Cisco & MapR bring 3 Superpowers to SAP HANA Deployments
Cisco & MapR bring 3 Superpowers to SAP HANA DeploymentsCisco & MapR bring 3 Superpowers to SAP HANA Deployments
Cisco & MapR bring 3 Superpowers to SAP HANA Deployments
 
MapR and Cisco Make IT Better
MapR and Cisco Make IT BetterMapR and Cisco Make IT Better
MapR and Cisco Make IT Better
 
Evolving from RDBMS to NoSQL + SQL
Evolving from RDBMS to NoSQL + SQLEvolving from RDBMS to NoSQL + SQL
Evolving from RDBMS to NoSQL + SQL
 

Último

How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 

Último (20)

How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 

Securing Hadoop by Sr. Principal Technologist Keys Botzum

  • 1. ® © 2014 MapR Technologies 1 ® © 2014 MapR Technologies Keys Botzum, Senior Principal Technologist July 2014
  • 2. ® © 2014 MapR Technologies 2 Agenda •  What’s MapR •  Why Secure Hadoop •  Securing MapR Hadoop •  Security beyond the core
  • 3. ® © 2014 MapR Technologies 3 MapR Data Platform Management MapR Data Platform MAPR-DBMAPR-FS APACHE HADOOP AND OSS ECOSYSTEM Hue ...SharkImpalaDrill Hive/ Stinger/ Tez Sqoop Storm SentrySparkSolrCascadingMahoutFlume Oozie HBaseMapReduceYARNPigWhirrZookeeper MapR Data Platform TABLESFILES MapR Data Platform MAPR-DBMAPR-FS Patent Pending Enterprise-grade Security OperationalPerformance • High availability • Data protection • Disaster recovery • Standard file access • Standard database access • Pluggable services • Broad developer support • Enterprise security authorization • Wire-level authentication • Data governance • Ability to support predictive analytics, real-time database operations, and support high arrival rate data • Ability to logically divide a cluster to support different use cases, job types, user groups, and administrators • Ability to deliver 2X to 7X performance • Consistent low latency Multi-tenancyInter-operability MapR Distribution for Hadoop
  • 4. ® © 2014 MapR Technologies 4 MapR: Best Product, Best Business, Best Customers, Best Partners, Best Investors Top Ranked Exponential Growth 500+ Customers Cloud Leaders 3X bookings Q1 ‘13 – Q1 ‘14 80% of accounts expand 3X 90% software licenses <2% lifetime churn >$1B in incremental revenue generated by 1 customer
  • 5. ® © 2014 MapR Technologies 5 Why Secure Hadoop Now? •  Historically security wasn’t a high priority –  Reflection of the type of data and the type of organizations using Hadoop •  Hadoop is now being used by more traditional firms as well as organizations with high security requirements –  Highly regulated –  Sensitive data sets –  People with experience with security in existing enterprise technologies (e.g., databases) are asking for the same in Hadoop •  Think for a moment and imagine the value of the data in a Hadoop cluster used as a data lake –  Much valuable operational data about your customers, systems, sales, etc.
  • 6. ® © 2014 MapR Technologies 6 Typical Hadoop Deployment Weaknesses •  Client operating system is trusted to identify user (weak authentication) –  If I can compromise client, I can run jobs or access HDFS as anyone –  Think about virtual machines with root access •  Hadoop servers trust anyone that can reach them on the network –  Could I falsify a data node, job tracker, etc.? •  Hive Server runs as ‘system’ user –  All Hive Server submitted jobs run as that ‘system’ user •  Intruders can see and modify all network traffic
  • 7. ® © 2014 MapR Technologies 7 Agenda •  What’s MapR •  Why Secure Hadoop •  Securing MapR Hadoop •  Security beyond the core
  • 8. ® © 2014 MapR Technologies 8 MapR 3.1: Securing MapR Hadoop •  Core goals –  Leverage Apache Hadoop security functionality –  Authenticate network traffic •  Users authenticate •  Servers authenticate to each other •  Support but do not require Kerberos –  Encrypt network traffic –  Authorization •  Integrate with existing authorization functionality •  Enhance MapR Tables authorization with fine grained controls –  Low barrier to entry •  Low performance overhead •  Simple and easy to administer •  Support, but do not require Kerberos
  • 9. ® © 2014 MapR Technologies 9 MapR Native Security •  Hadoop security without Kerberos –  But borrows heavily from Kerberos design •  Password authentication is natively supported •  Kerberos integration if desired
  • 10. ® © 2014 MapR Technologies 10 Architecture •  Shared secrets like Kerberos –  Managed at cluster level –  Two shared keys: cldb key and server key •  Identity represented using a ticket which is issued by MapR CLDB servers (Container Location DataBase)
  • 11. ® © 2014 MapR Technologies 11 Tickets •  A ticket represents a valid authenticated identity •  Contains –  An expiration time, renewal lifetime, and creation time –  A randomly generated secret key –  Information about the identity – userid, group ids •  Signed and encrypted when issued by CLDB –  CLDB key used for ‘permanent’ server tickets –  Server key used for ephemeral tickets issued for users •  A client authenticates to trusted servers using the ticket
  • 12. ® © 2014 MapR Technologies 12 User Experience •  User invokes maprlogin –  maprlogin connects to CLDB (over https) •  Provide userid & password (or Kerberos ticket) for validation by CLDB –  Ticket is returned, saved in file in /tmp file and accessible only by owning user – file name is /tmp/maprticket_<uid> •  All processes automatically pick up ticket (nothing to do) –  Java and C/C++ clients implicitly look for valid ticket and use it –  Clients optionally use existing Kerberos identity to get MapR ticket •  MapR PAM module –  Optional MapR provided PAM module creates MapR tickets automatically during Unix login
  • 13. ® © 2014 MapR Technologies 13 Maprlogin •  Primary user visible security tool •  Actions are –  password - authenticate to a MapR cluster using a valid password –  kerberos - authenticate to a MapR cluster using Kerberos –  print - print information on your existing credentials –  authtest - test authentication as a generic client –  end / logout - logout of cluster –  renew - renew existing ticket
  • 14. ® © 2014 MapR Technologies 14 Registry Neutral – Use the Operating System Security •  User information is obtained using PAM and Linux pwent APIs –  Fully pluggable – any registry works –  Passwords are verified using PAM –  User information obtained via Unix getpw* APIs •  nsswitch controlled –  Basically, if it works with Linux authentication, it should work with MapR
  • 15. ® © 2014 MapR Technologies 15 CLI Example $ hadoop fs -ls / Bad connection to FS. command aborted. exception: failure to login: Unable to obtain MapR credentials $ maprlogin password [Password for user 'fred' at cluster 'my.cluster.com': ] MapR credentials of user 'fred' for cluster 'my.cluster.com' are written to ‘/tmp/maprticket_1001' $ hadoop fs -ls / Found 3 items -rwxr-xr-x 3 mapr mapr 0 2013-12-10 13:25 /hbase drwxr-xr-x - mapr mapr 1 2013-12-10 13:25 /user drwxr-xr-x - mapr mapr 1 2013-12-10 13:25 /var
  • 16. ® © 2014 MapR Technologies 16 Maprlogin – Under the Covers maprlogin     MapR   CLDB   1.  username/passwd   sent  on  h;ps   LDAP/   Kerberos /NIS   2.  uses  PAM  to   authenDcate   3.    Dcket  +  user   key  returned   FileServer/   CLDB   4.    Dcket  +  key  saved  in  file  in  /tmp   hadoop  fs  –ls  /   5.    cmd  picks  up   Dcket  +  key  from   file   6.    client  sends  RPC   encrypted  with   user-­‐key  +  Dcket     7.    server  decrypts  Dcket  to   authenDcate  user  and   checks  permissions  on  ACL  
  • 17. ® © 2014 MapR Technologies 17 Client First Contact •  Client sends the ticket and data encrypted using secret key •  Receiving server –  Extracts and decrypts ticket to obtain secret key –  Checks expiration –  Uses the secret key to decrypt the data •  This proves that the client possesses the key that corresponds to the ticket –  Extracts identity information from ticket and uses that for authorization –  Returns encrypted response to client •  MapR user identity is independent of host or operating system identity
  • 18. ® © 2014 MapR Technologies 18 Server First Contact •  When a trusted server starts it uses a local server ticket to authenticate to the CLDB –  CLDB verifies the ticket’s authenticity using secret key –  CLDB returns the server key that is used to create and validate user tickets –  The server is now a trusted member of the cluster
  • 19. ® © 2014 MapR Technologies 19 Component Security •  Security between MapR unique components (CLDB, file server, etc.) is handled via changes to the MapR RPC layer •  Apache components support pluggable security mechanisms – typically SASL –  We are providing a new mechanism called ‘maprsasl’ –  maprsasl secures communication following the same techniques as the MapR RPC layer •  Existing authorization code simply leverages the securely authenticated identity –  File access –  Job submission –  Queue ACLs –  And so on …
  • 20. ® © 2014 MapR Technologies 20 Example: Job Tracker Integration JT can create user tickets. TT copies ticket to private job directory on local disk. taskcontroller copies it to user private local disk dir and tasks set MAPR_TICKET_LOCATION to that place. JobClient JobTracker TaskTracker submit job (maprsasl) schedule job (maprsasl) File system 1. JC copies job conf securely to FS 4. TT launches job using ticket identity 3. TT fetches ticket 2. JT creates user ticket
  • 21. ® © 2014 MapR Technologies 21 Out of the Box Defaults •  User experience –  Users authenticate using maprlogin and passwords –  User ‘mapr’ is admin as always •  User must authenticate however, OS identity irrelevant –  Operating system identity (on or off cluster) no longer relevant to MapR security •  Obviously root user and ‘mapr’ user can read/write /opt/mapr •  We’ve also tightened permissions for many directories under /opt/mapr –  Web UIs require authentication –  MapR CLIs require authentication •  hadoop fs/mfs/jar/job/etc •  maprcli –  Any user can submit jobs, but can only admin their own jobs
  • 22. ® © 2014 MapR Technologies 22 Out of the Box Defaults •  Cluster operations –  All MapR servers authenticate to each other •  Most communication paths encrypted –  All nodes share common maprserverticket •  Nodes can only join cluster if they have maprserverticket –  Self-signed wildcard certificates created for HTTPS traffic •  ssl_keystore contains certificate and private key, ssl_truststore contains certificate –  We set JVM system property: javax.net.ssl.trustStore •  Used by Web UIs, MCS, and maprlogin to CLDB •  Uses hostname command to get DNS domain for cluster and put that into certificate
  • 23. ® © 2014 MapR Technologies 23 Cryptography •  Encrypted using current NIST standards –  AES-256 in GCM mode for encryption and signing •  http://en.wikipedia.org/wiki/Galois/Counter_Mode •  NIST standard - http://csrc.nist.gov/publications/fips/fips140-2/fips1402annexa.pdf –  Leverage Intel hardware encryption where available, software otherwise •  Use the open source crypto++ library for our C++ cryptography – http://cryptopp.com •  Random number generation –  Use secure random number generation as documented here http://www.cryptopp.com/docs/ref/ class_auto_seeded_random_pool.html#_details
  • 24. ® © 2014 MapR Technologies 24 Let’s Build a Secure Cluster! Node 1 apt-get install mapr…. configure.sh –C … -Z … -secure –genkeys –  Generates all needed keys for MapR-RPC as well as for HTTPS Node N apt-get install mapr…. scp rootORmapr@node1:/opt/mapr/conf/ {cldb.key,maprserverticket,ssl_keystore,ssl_truststore} /opt/mapr/conf configure.sh –C … -Z … -secure Clients apt-get install mapr… scp anyuser@nodeN:/opt/mapr/conf/ssl_truststore /opt/mapr/conf configure.sh … -secure
  • 25. ® © 2014 MapR Technologies 25 That Was Easy – What Happened? •  Secret keys generated for encryption of RPC traffic •  Certificates generated for HTTPS traffic •  Configuration updated so that core components authenticate traffic –  Most traffic also encrypted –  Bulk data transfer encryption optional on per file/directory basis •  Web UIs switched to HTTPS •  Web UIs require password authentication – JT/TT/MCS •  Hadoop Java code switched to maprsasl •  Authorization components now rely upon MapR identity
  • 26. ® © 2014 MapR Technologies 26 Encrypted Shuffle (?) •  No need to special case encrypting shuffle •  MapR-FS is store for Map output –  Shuffle inherits the same encryption, authentication, and authorization functionality of the rest of MapR-FS
  • 27. ® © 2014 MapR Technologies 27 Kerberos •  Not required but can use •  Kerberos SSO –  Explicitly using ‘maprlogin kerberos’ –  Implicitly •  If no MapR ticket available, client automatically detects and uses Kerberos ticket and uses it to obtain MapR ticket •  Kerberos SSO requires only –  Kerberos client on CLDB and client machines –  Kerberos identity only for CLDB – typically 3-5 CLDBs •  No need to manage identities for every node •  SPNEGO/Kerberos for Web UIs requires creating host Kerberos identities
  • 28. ® © 2014 MapR Technologies 28 Agenda •  What’s MapR •  Why Secure Hadoop •  Securing MapR Hadoop •  Security beyond the core
  • 29. ® © 2014 MapR Technologies 29 Hadoop Map Reduce Clients •  Many components simply generate Map Reduce jobs. As such they implicitly leverage the security we’ve defined for Map Reduce previously. They are: –  Hive (except Hive Server) –  Pig –  Mahout –  Sqoop
  • 30. ® © 2014 MapR Technologies 30 Ecosystem Security •  All ecosystem components run securely as well in a secure MapR cluster –  Some by default –  Some with minor configuration •  Most Web UIs enhanced to use userid & password authentication and HTTPS –  Can configure Kerberos SPNEGO, same as from Apache
  • 31. ® © 2014 MapR Technologies 31 MapR Ecosystem Security – by Default •  By default, out of the box when security enabled –  Hive Server 2 supports password authentication •  Can configure Kerberos and SSL function, same as from Apache, including secure impersonation –  Oozie supports MapR ticket authentication •  Can configure Kerberos and SSL function, same as from Apache, including secure impersonation •  MapR Tables (HBase APIs) use native MapR security, no configuration needed •  HBase and Hive MetaServer require Kerberos to be secured
  • 32. ® © 2014 MapR Technologies 32 MapR Tables Authorization •  boolean logic constraints on access to M7 tables –  Uses user & group information –  Very powerful •  ( u:bob | g:admins) •  ( g:managers & ! g:restricted) •  ( g: managers & g:businessunity) | g:executives –  Settable at table, column, and column family level for various actions –  Queries silently hide data you are not authorized to see
  • 33. ® © 2014 MapR Technologies 33 MapR Hadoop Advantage •  Vastly simpler –  Core secured by default in one step –  No requirement for Kerberos in core and associated complexity •  Easier integration –  Leverage existing Linux authentication (PAM and NSSwitch) •  Faster –  Leverage Intel AES hardware cryptography
  • 34. ® © 2014 MapR Technologies 34 Further Reading •  MapR –  http://mapr.com •  MapR Native Security –  http://www.mapr.com/blog/getting-started-mapr-security-0 –  http://www.mapr.com/press-release/mapr-technologies-integrates-security- into-hadoop –  http://www.mapr.com/products/only-with-mapr/mapr-integrates-security-into- hadoop •  Adding Security to Apache Hadoop –  http://hortonworks.com/wp-content/uploads/2011/10/security- design_withCover-1.pdf •  The Evolution of Hadoop’s Security Model –  http://www.infoq.com/articles/HadoopSecurityModel/
  • 35. ® © 2014 MapR Technologies 35 Q&A @mapr maprtech kbotzum@mapr.com Engage with us! MapR maprtech mapr-technologies