IBM Spectrum Scale Authentication For Object - Deep Dive

IBM Spectrum Scale™
Authentication (for Object
Access)
Smita Raut
Spectrum Scale Cloud and Object
Sandeep Patil
STSM, Spectrum Scale
Deepak Ghuge
Spectrum Scale Cloud and Object

Agenda and Flow
• Object Authentication in IBM Spectrum Scale™
• Administration – Prerequisites and Overview
• Administration – Install Toolkit Method
• Administration – Using CLI
• Administration of Unified File and Object
• Validating Object Authentication
• Creating projects, roles and setting ACLs
• Problem Determination Guide
2

Agenda and Flow
3

Introduction to OpenStack Keystone
• Identity service used by OpenStack for authentication and high-level
authorization
• Supports token based authentication and user-service authorization
• Implements OpenStack’s identity API
• OpenStack Keystone packages bundled and shipped with Spectrum
Scale
• When configured , Keystone runs on all the spectrum scale protocol
nodes ensuring HA .
• Requests coming to keystone can be load balanced using DNS round
robin or HAProxy with Spectrum Scale
• Spectrum Scale supports Keystone V2.0 and V3 4

Spectrum Scale Object Authentication Flow
• Swift clients make a request
to keystone to get the auth
token
• Auth token is valid for a
configured duration of time,
typically 24 hrs
• Swift clients pass on this
token to Swift service to
perform Object IO
• Swift validates this token with
keystone
5

Supported Types for Object Authentication
• LDAP/AD(Active Directory)
• Users from LDAP(RFC2307) or AD can be used for authentication
• Support for single domain for Active Directory
• TLS supported for communication with LDAP/AD
• Kerberos, Trust in AD, LDAP referral is not supported
• Local Authentication
• Users are stored in postgresql database
• Userdefined Authentication
• If External keystone needs to be used(Advance Functionality)
• Support v2.0 and v3 keystone api
6

Agenda and Flow
7

Object Authentication Prerequisites
• The system administrator needs to ensure that the authentication
server is set up properly and the connection between the IBM
Spectrum Scale™ system and authentication server is established
properly.
• Depending on the requirement, the IBM Spectrum Scale™ system
administrator needs to set up the following servers:
• Microsoft Active Directory (AD) for file and object access
• Lightweight Directory Access Protocol server for file and object access
• If external keystone is to be used, then keystone server must be configured
• Ensure the server details such as IP address or host name, admin user
name, password, base dn, and user dn are known.
8

Administration commands for Authentication
IBM Spectrum Scale™ can be configured with the following authentication
servers for object access:
• Active Directory (AD)
• Light Weight Directory Access Protocol (LDAP)
• Local Authentication Server (Postgres)
• User Defined Authentication (External Keystone)
Two methods available for Managing/Administering
• Spectrum Scale Installation toolkit
• Using CLI
o During Object deploy (mmobj swift base)
o After Object deploy (mmuserauth service create)
Administration
9

Agenda and Flow
10

Configuring Object Authentication Using Install
Toolkit
• During First time Object Install/Enable
• Four authentication options:
- Local Authentication
- Active Directory
- LDAP
- User Defined (External Keystone)
• By default object is configured with local auth.
• Object configuration with SSL-enabled external keystone is not
supported using the install toolkit
• Cant be used for changing authentication
Administration
11

spectrumscale auth object
• To setup object authentication, run the
installer command-
spectrumscale auth object [-h]
[--https] [--pki]
{local,external,ldap,ad}
• This will automatically open a template file
for you to fill with the required auth
settings. TLS and SSL related settings can
also be done here. Save the file and close
it.
• If this install toolkit auth command has
been run, authentication will automatically
be enabled by the installer.
• This command must be run before running
“spectrumscale deploy”. After
deploy, object gets configured with these
authentication settings.
• This command can only be used during
initial deployment. It cannot be used on a
cluster with object deployed to configure
or change object auth.
Administration
12
Sample AD auth configuration file
[object]
remote_keystone = False
[object_auth]
enable_object_auth = True
backend_server = ad
# mandatory settings for object
authentication:
# Specifies the host name or IP address of
the authentication server.
servers =
# Specifies the base DN of the authentication
server.
base_dn =
# Specifies the DN for user search base.
user_dn =
# Specifies the user which will be assigned
the administrator role
# in Keystone.
admin_user =
# Specifies the AD user which will be used as
the swift service user.
# This user's details will be updated in
proxy-server.conf.
swift_user =
# Specifies the password of the swift_user.
# Leave as [prompt] to be prompted for the
password in
# a secure manner.
swift_password = [prompt]
Sample external keystone
auth configuration file
[object_auth]
# This installer will not
configure your external
keystone server
enable_object_auth = False
backend_server = external
[object]
remote_keystone = True
# Set to True to create swift
service, user and endpoint in
remote keystone
configure_remote_keystone =
False
# Supply the full URL for
your external keystone server
keystone_url =
http://extserver.com

Agenda and Flow
13

mmobj swift base
• Used for initial configuration of object protocol when Spectrum Scale install toolkit is not used for
object deployment.
• Supports configuring local authentication or user defined authentication. It is mandatory to select
either of the authentication option.
• AD or LDAP authentication configuration is not supported through this command.
• Sample command:
mmobj swift base -g /gpfs1 -o swift --cluster-hostname
c6f1c1p1v1 --local-keystone --admin-password Passw0rd --admin-
user keystone
Note:
- admin-password parameter can be skipped on the command line if desired for security
reasons. It will be prompted in that case.
- If AD or LDAP authentication must be used, earlier auth configuration done via mmobj must
be removed and new auth with AD/LDAP to be configured using mmuserauth.
Administration
14

mmuserauth service Suite
• This command suite manages the authentication configuration of file and object
access protocols.
• The configuration allows protocol access methods to authenticate users who
need to access data that is stored on the system over these protocols.
• The different commands in the mmuserauth service suite are:
• mmuserauth service create - Configures authentication for file and object access
protocols.
• mmuserauth service list - Displays the details of the authentication method that is
configured for both file and object access protocols.
• mmuserauth service check - Verifies the authentication method configuration details
for file and object access protocols. Validates the connectivity to the configured
authentication servers. It also supports corrections to the configuration details on the
erroneously configured protocol nodes.
• mmuserauth service remove - Removes the authentication method configuration of
file and object access protocols and ID maps if any.
Note : use option --data-access-method object in all mmuserauth service
<operation> command for object authentication
Administration
15

Configuring Object with Local Authentication
Administration
mmuserauth service create –data-access-method object –type local –ks-
dns-name cesobjnode –ks-admin-user admin –ks-admin-pwd Password –ks-
swift-user swift –ks-swift-pwd Password
Verify the configuration by running:
mmuserauth service list --data-access-
method object
OBJECT access configuration : LOCAL
PARAMETERS VALUES
————————————————-
ENABLE_KS_SSL false
ENABLE_KS_CASIGNING false
KS_ADMIN_USER admin
The openrc file should look like:
export OS_AUTH_URL=”https://127.0.0.1:35357/v3
export OS_IDENTITY_API_VERSION=3
export OS_AUTH_VERSION=3
export OS_USERNAME=”admin”
export OS_PASSWORD=”Password”
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_PROJECT_DOMAIN_NAME=Default
16
Note:
- ks-admin-user is the keystone administrative user. If using local auth, this user is automatically created in postgres
database and appropriate role assigned.
- ks-swift-user is the user to be used by swift services to communicate with keystone. If using local auth, this user is
automatically created in postgres database and appropriate role assigned.

17
Protocol Node 1 Protocol Node 2 Protocol Node n
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Object Authentication with Local Authentication Postgres Keystone DB
Username-Password
Domain
Role
Project
User-Project-Role
Token

18
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Object Authentication with Local Authentication Postgres Keystone DB
Username-Password
Domain
Role
Project
User-Project-Role
Token
1
2
3
4
5
6
7

19
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Object Authentication with Local Authentication
--enable-ks-ssl
Postgres Keystone DB
Username-Password
Domain
Role
Project
User-Project-Role
Token
Secure Communication

Configuring Object with LDAP Authentication
Administration
mmuserauth service create –type ldap –data-access-method object –user-name
“cn=manager,dc=essldapdomain” –password “Passw0rd” –base-dn
dc=isst,dc=aus,dc=stglabs,dc=ibm,dc=com –ks-dns-name 192.168.6.99 –ks-admin-user
user1 –servers 192.168.101.55 –user-dn “ou=People,dc=essldapdomain” –ks-swift-user
swift –ks-swift-pwd Passw0rd
mmuserauth service list –data-access-method object
OBJECT access configuration : LDAP
PARAMETERS VALUES
————————————————-
ENABLE_ANONYMOUS_BIND false
ENABLE_SERVER_TLS false
ENABLE_KS_SSL false
USER_NAME cn=manager,dc=essldapdomain
SERVERS 192.168.101.55
BASE_DN dc=isst,dc=aus,dc=stglabs,dc=ibm,dc=com
USER_DN ou=people,dc=essldapdomain
USER_OBJECTCLASS posixAccount
USER_NAME_ATTRIB cn
USER_ID_ATTRIB uid
USER_MAIL_ATTRIB mail
USER_FILTER none
KS_ADMIN_USER user1
20
Note: Both the –ks-admin-user and the –
ks-swift-user specified in the command
must already exist in LDAP.

Configuring Object with AD Authentication
Administration
mmuserauth service create –type ad –data-access-method object –user-name
“cn=Administrator,cn=Users,dc=adcons,dc=spectrum” –password “Passw0rd3” –base-dn
“dc=adcons,dc=spectrum” –ks-dns-name 192.168.6.99 –ks-admin-user Administrator –ks-swift-user
swift –ks-swift-pwd Passw0rd2 –servers 192.168.76.50 –user-id-attrib cn –user-name-attrib
sAMAccountName –user-objectclass organizationalPerson –user-dn “cn=Users,dc=adcons,dc=spectrum”
OBJECT access configuration : AD
PARAMETERS VALUES
————————————————-
ENABLE_KS_SSL false
USER_NAME cn=Administrator,cn=Users,dc=adcons,dc=spectrum
SERVERS 192.168.76.50
BASE_DN dc=adcons,dc=spectrum
USER_DN cn=users,dc=adcons,dc=spectrum
USER_OBJECTCLASS organizationalPerson
USER_NAME_ATTRIB sAMAccountName
USER_ID_ATTRIB cn
USER_FILTER none
KS_ADMIN_USER Administrator 21
Note: Both the –ks-admin-user and
the –ks-swift-user specified in the
command must already exist in AD

22
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Active Directory /
LDAP
Object Authentication with Active Directory or LDAPPostgres Keystone DB
Username-Password
Domain
Role
Project
User-Project-Role
Token

23
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Active Directory /
LDAP
Object Authentication with Active Directory or LDAPPostgres Keystone DB
Username-Password
Domain
Role
Project
User-Project-Role
Token
1
2
3
4
5
6
7
8

Configuring Object Authentication with TLS
Administration
–enable-server-tls needs to be specified in the mmuserauth
command in order to configure server TLS.
E.g. the command to configure AD-TLS would look like:
mmuserauth service create –type ad –data-
access-method object –user-name
“cn=Administrator,cn=Users,dc=adcons,dc=spectrum
” –password “Passw0rd3” –base-dn
“dc=adcons,dc=spectrum” –ks-dns-name
192.168.6.99 –ks-admin-user Administrator –ks-
swift-user swift –ks-swift-pwd Passw0rd2 –
servers 192.168.76.50 –user-id-attrib cn –user-
name-attrib sAMAccountName –user-objectclass
organizationalPerson –user-dn
“cn=Users,dc=adcons,dc=spectrum” –enable-server-
tls
OBJECT access configuration : AD
PARAMETERS VALUES
————————————————-
ENABLE_SERVER_TLS true
ENABLE_KS_SSL false
USER_NAME cn=Administrator,cn=Users,dc=a
dcons,dc=spectrum
SERVERS 192.168.76.50
BASE_DN dc=adcons,dc=spectrum
USER_DN cn=users,dc=adcons,dc=spectrum
USER_NAME_ATTRIB sAMAccountName
USER_ID_ATTRIB cn
USER_FILTER none
KS_ADMIN_USER Administrator
In order to configure Object with AD-TLS or LDAP-TLS, copy the TLS certificate to local ces node
from where CLI will be run. The TLS certificate should be named object_ldap_cacert.pem and
copied to /var/mmfs/tmp
24

25
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Active Directory /
LDAP
Object Authentication with Active Directory or LDAP
--enable-server-tls
Username-Password
Domain
Role
Project
User-Project-Role
Token

Configuring Object Authentication with
Keystone https (SSL)
Administration
mmuserauth service create –data-access-method object –type local
–ks-dns-name cesobjnode –enable-ks-ssl –ks-admin-user admin –ks-
admin-pwd Password –ks-swift-user swift –ks-swift-pwd Password
mmuserauth service list --data-
access-method object
OBJECT access configuration : LOCAL
PARAMETERS VALUES
————————————————-
ENABLE_KS_SSL true
KS_ADMIN_USER admin
export OS_CACERT=”/etc/keystone/ssl/certs/ssl_cacert.pem”
export OS_AUTH_URL=”https://cesobjnode:35357/v3;
export OS_PASSWORD=”Password”
26
Pre-requisite: Get set of ssl certificate(Private key, Certificate and Cacert) and copy these files to /var/mmfs/tmp

27
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Active Directory /
LDAP
--enable-ks-ssl
Username-Password
Domain
Role
Project
User-Project-Role
Token

28
CesSharedRoot
Postgresql
Keystone DB
Swift
Keystone
Swift
Keystone
Swift
Keystone
Postgres
Active Directory /
LDAP
--enable-server-tls & --enable-ks-ssl
Username-Password
Domain
Role
Project
User-Project-Role
Token

Configuring User Defined Object Authentication with
External Keystone Server
Administration
mmuserauth service create --data-access-method object --type
userdefined --ks-ext-endpoint http://192.168.126.156:35357/v3 --ks-
swift-user swift --ks-swift-pwd password
mmuserauth service list --data-access-
method object
OBJECT access configuration : USERDEFINED
PARAMETERS VALUES
------------------------------------------
-------
# Mon May 2 13:58:12 IST 2016
export OS_AUTH_URL=http://192.168.126.156:35357/v3
export OS_PASSWORD=
29

30
CesSharedRoot
Swift Swift
Keystone
Swift
Object Authentication with External Keystone

31
CesSharedRoot
Swift Swift
Keystone
Swift
Object Authentication with External Keystone
1
3
4
5
2

32
CesSharedRoot
Swift Swift
Keystone
Swift
Object Authentication with External Keystone : --enable-ks-ssl

Verifying the authentication services
configured in the system
• mmuserauth service check --data-access-method object [-
N|--nodes] {node-list|cesNodes} [--server-reachability]
[-r|--rectify ]
• The mmuserauth service check command helps to
check whether the authentication configuration is consistent
across the cluster and the required services are enabled and
running.
• This command validates and corrects the authentication
configuration files and starts any associated services if
needed.
Administration
33

Deleting authentication and ID mapping
configuration
• Deleting the authentication and ID mapping configuration results
in loss of access to data.
• Object IDMapping = Relationship { user-project-role }
(mmuserauth service remove –data-access-method object—
idmapdelete - delete this relationship)
• Issue the mmuserauth service remove command to
remove the authentication configuration as shown in the
following example:
# mmuserauth service remove –data-acess-method object
mmcesuserauthrmservice: Command successfully completed.
• # mmuserauth service remove –data-acess-method object --
idmapdelete
Administration
34

Modifying the authentication method
IMPORTANT:
• Modification = remove + create
• Modifying authentication method should only be done during pre-production phase where
customer trying to see which mechanism really suite his requirement.
• If data already exists or is created with the existing authentication, it is not recommended to
change the authentication. It might result in access to data loss or unauthorized access.
• There is support for changing authentication parameter but limited to only set parameter(refer
documentation). Eg : Change of ldap/ad server ip, password change,ldap filter change.
• Object Authentication parameter should only be changed via mmobj config change command.
Note – mmobj config change is object only command, it does not applies to file authentication
• Parameter updated via mmobj config change command does not reflect in mmuser service list
command.
Administration
35

Agenda and Flow
36

Configuring Object Authentication for
Unified File and Object
Administration
Local_mode - Separate identity between object and file
(Default mode)
• Object authentication setup is independent of file
authentication setup
Unified_mode - Shared identity between object and file
• Supported only with Active Directory (AD) with UNIX-
mapped domains and LDAP authentication
configurations
• Authentication for both file and object access must be
configured and the authentication schemes must be
the same and configured with the same server
37
Ref: Video of presentation done on this topic at OpenStack summit April 2016 in Austin-
https://www.youtube.com/watch?v=6ovLb6aktbM&t=93s

Unified File and Object – unified_mode of ID Mapping
Administration
38
• Users from object and file are expected to be common and coming from the same directory service (only
AD+RFC 2307 or LDAP)
• Object created from the object interface is owned by the user doing the object PUT operation
• If the object already exists, existing ownership of the corresponding file is retained if retain_owner is set to yes
in object-server-sof.conf
• Object access follows the object ACL semantics and file access follows the file ACL semantics
• If the object is created or updated over existing file then existing file ACL, xattrs, and winattrs are retained if
retain_acl, retain_xattr, and retain_winattr are set to yes in object-server-sof.conf
• Security or system extended attributes and other IBM Spectrum Scale extended attributes such as immutability,
pcache, etc. are not retained
• Swift metadata (user.swift.metadata) is also not retained and it is replaced according to object semantics
• Change id_mgmt in the object-server-sof.conf file using the mmobj config change command as follows
mmobj config change --ccrfile object-server-sof.conf --section DEFAULT --
property id_mgmt --value unified_mode
• If object authentication is configured with AD, set ad_domain in the object-server-sof.conf file
mmobj config change --ccrfile object-server-sof.conf --section DEFAULT --
property ad_domain --value POLLUX
• List the currently configured id_mgmt mode using the mmobj config list command as follows
mmobj config list --ccrfile object-server-sof.conf --section DEFAULT --
property id_mgmt

Agenda and Flow
39

Validating Object Authentication Using
Openstack client
• Swift and
openstack
clients are
installed on
CES nodes by
default
• They use
environment
variables from
openrc if not
specified in
commandline
• Keystone
AD/LDAP
interface is
read-only and
new users
cannot be
created
through
keystone
40
List current keystone endpoints:
List current projects: List current users
Show current defined roles:

Agenda and Flow
41

Creating Users, Projects, Roles and Setting ACLS
42
Create a new role: Create a new project:
Assign new role to the user:
Setting ACLs on container:
All these operations can also
be performed through GUI
Create a new user (only for local auth):

Agenda and Flow
43

Problem Determination Guide
This section describes the following:
• Monitoring IBM Spectrum Scale™
• Collecting details of issues using available methods
• Usecases/Common Problems
• Debugging
44

Monitoring IBM Spectrum Scale™
Monitoring GUI:
• Monitoring -> Events Page in the GUI allows you to review the set of events that are
reported in the IBM Spectrum Scale™ system.
• You can filter the Events as Current Issues, Unread Issues and All Events.
• You can also determine if the event is Informational, Warning or an Error.
• You can mark the even as Read and also resolve some issues by Running a Fixed
Procedure. “Run Fix Procedure” Action helps to do so.
• The system can also use SMTP traps and email to notify you of an event.
• Settings -> Event Notifications Page allows to do so.
• Notifications are usually sent immediately after an event is raised.
• Reports of all events can also be sent. Reports are sent once a day.
• You can configure Email Notification for receiving emails for Authentication Events.
• To create email recipients, select Email Recipients from the Event Notifications page,
and then click Create Recipient
• Refer the Knowledge center to know more on how to setup up SMTP Manager.
45

Monitoring using CLI
• The mmhealth command is used to monitor the health status of the system and the
services running on the nodes.
• The sub-components of CES service such as NFS, SMB, Object, and authentication have
their own health monitors.
• The mmhealth command gets the health details from these monitoring services.
• Monitoring health of CES Node:
• Node role: This node role is active on the CES nodes that are listed by
mmlscluster --ces.
• Once a node obtains this role, all corresponding CES sub-services are activated on that node.
• The CES service does not have its own monitoring service or events. The status of the CES is an
aggregation of the status of its sub-services.
• The following few sub-services are monitored: (Refer the knowledge center for more sub-services)
a. AUTH – Tasks: Monitors LDAP, AD and or NIS-based authentication services.
b. AUTH_OBJ – Tasks: Monitoring the OpenStack identity service functionalities.
c. OBJECT – Tasks: Monitors the IBM Spectrum Scale™ for object functionality. Especially, the status of relevant
system services and accessibility to ports are checked.
46

The following are the possible status of nodes and services:
• UNKNOWN - Status of the node or the service hosted on the node is not known.
• HEALTHY - The node or the service hosted on the node is working as expected. There are no
active error events.
• CHECKING - The monitoring of a service or a component hosted on the node is starting at the
moment. This state is a transient state and is updated when the startup is completed.
• TIPS - There might be an issue with the configuration and tuning of the components. This status is
only assigned to a Tip event.
• DEGRADED - The node or the service hosted on the node is not working as expected. That is, a
problem occurred with the component but it did not result in a complete failure.
• FAILED - The node or the service hosted on the node failed due to errors or cannot be reached
anymore.
• DEPEND - The node or the services hosted on the node have failed due to the failure of some
components. For example, an NFS or SMB service shows this status if authentication has failed.
47

Collecting details of the issue
Collecting details of the issue involves collecting data using gpfs.snap for:
1. Authentication
2. Object Protocol
Authentication Related – For such issues, gpfs.snap command would
collect all Authentication configuration and error logs. Also the different log
files for the authentication components can be checked.
Object Protocol Related – For such issues, gpfs.snap command would
collect keystone and http server related configuration and logs.
48

LDAP Attributes related issuesScenario : Object authentication with LDAP. The default values of mmuserauth option are not matching with actual values on LDAP
[root@c1n4 ~]# mmuserauth service create --data-access-method object --type ldap --servers 192.168.122.27 --
user-name administrator@sonas.com --password Passw0rd --user-dn dc=sonas,dc=com --base-dn dc=sonas,dc=com --ks-
admin-user administrator --ks-swift-user longnameuser --ks-swift-pwd Passw0rd --ks-dns-name c1ces
[E] Didn't find entry for user administrator with ldap search mmuserauth service create: Command failed. Examine
previous error messages to determine cause.
The command failed because it is not able to find user administrator using the option specified and default options. One need to specify the option explicitly on
command line if default values are not matching with the LDAP server environment.
49
Default values of mmuserauth when –type=ldap
and –data-access-method=object
--user-objectclass= posixAccount
--user-name-attrib= cn
--user-id-attrib=uid
[root@c1n4 ~]# ldapsearch -x -h 192.168.122.27 -b dc=sonas,dc=com -D administrator@sonas.com -w Passw0rd
cn=administrator
# extended LDIF
# Administrator, Users, SONAS.COM
dn: CN=Administrator,CN=Users,DC=SONAS,DC=COM
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: Administrator
distinguishedName: CN=Administrator,CN=Users,DC=SONAS,DC=COM
name: Administrator
objectGUID:: gWYgEijUAkG6rDsjQ6fC7A==
sAMAccountName: Administrator
sAMAccountType: 805306368
uid: Administrator
mail: administrator@sonas.com
uidNumber: 20021
gidNumber: 21000
unixHomeDirectory: /home/Administrator
loginShell: /bin/sh

50
Executing mmuserauth again with specifying the default option with correct values on command line itself
[root@c1n4 ~]# /usr/lpp/mmfs/bin/mmuserauth service create --data-access-method object --type ldap --servers 192.168.122.27
--user-name administrator@sonas.com --password Passw0rd --user-dn dc=sonas,dc=com --base-dn dc=sonas,dc=com --ks-admin-user
administrator --ks-swift-user longnameuser --ks-swift-pwd Passw0rd --ks-dns-name c1ces --user-objectclass
organizationalPerson --user-id-attrib CN --user-name-attrib uid
mmcesobjcrbase: Validating execution environment.
mmcesobjcrbase: Performing SELinux configuration.
mmcesobjcrbase: Configuring Keystone server in /ibm/gpfs0/ces/object/keystone.
mmcesobjcrbase: Initiating action (start) on postgres in the cluster.
mmcesobjcrbase: Validating Keystone environment.
mmcesobjcrbase: Validating Swift values in Keystone.
mmcesobjcrbase: Configuration complete.
Object configuration with LDAP as the identity backend has completed successfully.
Object authentication configuration completed successfully.
[root@c1n4 ~]# /usr/lpp/mmfs/bin/mmuserauth service list --data-access-method object
PARAMETERS VALUES
-------------------------------------------------
ENABLE_KS_SSL false
USER_NAME administrator@sonas.com
SERVERS 192.168.122.27
BASE_DN dc=sonas,dc=com
USER_DN dc=sonas,dc=com
USER_NAME_ATTRIB uid
USER_ID_ATTRIB CN
USER_FILTER none
KS_ADMIN_USER administrator
[root@c1n4 ~]# /usr/lpp/mmfs/bin/mmuserauth service check
--data-access-method object -N cesNodes
Userauth object check on node: c1n3
Checking keystone.conf: OK
Checking wsgi-keystone.conf: OK
Checking /etc/keystone/ssl/certs/signing_cert.pem: OK
Checking /etc/keystone/ssl/private/signing_key.pem: OK
Checking /etc/keystone/ssl/certs/signing_cacert.pem: OK
Service 'httpd' status: OK
[root@c1n4 ~]#
LDAP Attributes related issues…Continue

External Keystone – Consideration and Issues
• api supported by external keystone V2.0 or v3?
• ssl/non-ssl - In case of ssl CN is ssl certificate should match with hostname is keystoneURL
• <swiftuser> must exist in external keystone, and it should have ‘admin’ role in ‘service’ project in ’Default’
domain
• Validating external keystone before configuration
For v3:
export OS_AUTH_URL=“<keystoneURL>”
export OS_USERNAME=“<swift user>”
export OS_PASSWORD=“<swift Password>”
export OS_PROJECT_NAME=service
openstack --insecure role list --user <swiftUser> --project service -f value -c Name
In case of ssl - openstack --os-cacert <cacert path> role list --user <swiftUser> --project service -f value -c Name
Command should return the ‘admin’ role
Once pre-validation is successful use following command to configure authentication service
# mmuserauth service create --data-access-method object --type userdefined --ks-ext-endpoint
http://192.168.126.156:35357/v3 --ks-swift-user swift --ks-swift-pwd password
For v2.0:
/usr/bin/keystone [--os-cacert <cacert path>] --os-username <swiftUser> --os-password <swiftPassword> --os-tenant-name
service --os-auth-url <keystoneURL> user-role-list --user <swiftUser>
Command should return the ‘admin’ role
Once pre-validation is successful use following command to configure authentication service
#mmuserauth service create --data-access-method object --type userdefined --ks-ext-endpoint
http://192.168.126.156:35357/v2.0 --ks-swift-user swift --ks-swift-pwd password
Note : mmuserauth try to find the api version by querying the external keystone if not specified in keystoneURL. It is recommended to have api version in keystoneURL
51

SSL Certificate related issue
52
Scenario : Object authentication is configured with SSL using certificate having CN that does not match hostname(Endpoint)
[root@c1n4 ~]# openstack user list
Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from
URL.
SSL exception connecting to https://192.168.126.180:35357/v3/auth/tokens: hostname ’192.168.126.180' doesn't match u'c1ces’
Issue : CN in the ssl certificate(c1ces) is not matching with hostname(192.168.126.180)
Check the CN used in certificate by using following command
[root@c1n4 ~]# openssl x509 -in /var/mmfs/tmp/ssl_cert.pem -noout -purpose -text
Certificate purposes:
SSL client : Yes
SSL client CA : No
. . . . .
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1 (0x1)
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, ST=Unset, L=Unset, O=Unset, CN=c1ces
Validity
Not Before: May 11 10:03:12 2017 GMT
Not After : May 9 10:03:12 2027 GMT
Subject: C=US, ST=Unset, O=Unset, CN=c1ces
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
[root@c1n4 tmp]# /usr/lpp/mmfs/bin/mmuserauth service create --type local --data-access-method object --ks-admin-user deepak --
ks-admin-pwd password --enable-ks-ssl --ks-dns-name c1ces

LDAP/AD - TLS certificate Related issue
53
Scenario : Object authentication with TLS enabled LDAP. The CN used in TLS certificate on LDAP server and the IP/Hostname being used in mmuserauth are different.
[root@c1n4 tmp]# /usr/lpp/mmfs/bin/mmuserauth service create --data-access-method object --type ldap --servers
192.168.122.27 --user-name administrator@sonas.com --password Passw0rd --user-dn dc=sonas,dc=com --base-dn
dc=sonas,dc=com --ks-admin-user administrator --ks-swift-user longnameuser --ks-swift-pwd Passw0rd --ks-dns-name
c1ces --user-objectclass organizationalPerson --user-id-attrib CN --user-name-attrib uid --enable-server-tls
[E] Failed to execute command ldapsearch
mmuserauth service create: Command failed. Examine previous error messages to determine cause.
# Check using ldapsearch command whether LDAP communication succeeds with provided certificate
[root@c1n4 tmp]# export LDAPTLS_CACERT=/var/mmfs/tmp/object_ldap_cacert.pem; ldapsearch -x -h 192.168.122.27 -b
dc=sonas,dc=com -D administrator@sonas.com -w Passw0rd cn=administrator -ZZZ
ldap_start_tls: Connect error (-11)
additional info: TLS error -8179:Peer's Certificate issuer is not recognized.
# Above error shows the certificate is invalid
# Get the correct certificate for LDAP/AD admin and try same command.
export LDAPTLS_CACERT=/var/mmfs/tmp/object_ldap_cacert.pem; ldapsearch -x -h 192.168.122.27 -b dc=sonas,dc=com -
D administrator@sonas.com -w Passw0rd cn=administrator -ZZZ
ldap_start_tls: Connect error (-11)
additional info: TLS: hostname does not match CN in peer certificate
#Above error shows Certificate is correct but the CN of certificate is not matching with the LDAP hostname
# Get the correct name from LDAP/AD using which TLS certificate is issued and try ldapsearch again
[root@c1n4 tmp]# export LDAPTLS_CACERT=/var/mmfs/tmp/object_ldap_cacert.pem; ldapsearch -x -h w2k8-phy-
sonas.sonas.com -b dc=sonas,dc=com -D administrator@sonas.com -w Passw0rd cn=administrator -ZZZ
# extended LDIF
# Administrator, Users, SONAS.COM
dn: CN=Administrator,CN=Users,DC=SONAS,DC=COM
objectClass: top
objectClass: person
……

54
Executing mmuserauth with valid tls certificate and CN of LDAP server
[root@c1n4 tmp]# /usr/lpp/mmfs/bin/mmuserauth service create --data-access-method object --type ldap --servers w2k8-phy-sonas.sonas.com --user-name
administrator@sonas.com --password Passw0rd --user-dn dc=sonas,dc=com --base-dn dc=sonas,dc=com --ks-admin-user administrator --ks-swift-user longnameuser --ks-
swift-pwd Passw0rd --ks-dns-name c1ces --user-objectclass organizationalPerson --user-id-attrib CN --user-name-attrib uid --enable-server-tls
mmcesobjcrbase: Validating execution environment.
mmcesobjcrbase: Performing SELinux configuration.
mmcesobjcrbase: Configuring Keystone server in /ibm/gpfs0/ces/object/keystone.
mmcesobjcrbase: Initiating action (start) on postgres in the cluster.
mmcesobjcrbase: Validating Keystone environment.
mmcesobjcrbase: Validating Swift values in Keystone.
mmcesobjcrbase: Configuration complete.
Object configuration with LDAP as the identity backend has completed successfully.
Object authentication configuration completed successfully.
[root@c1n4 tmp]# /usr/lpp/mmfs/bin/mmuserauth service list --data-access-method object
PARAMETERS VALUES
-------------------------------------------------
ENABLE_SERVER_TLS true
ENABLE_KS_SSL false
USER_NAME administrator@sonas.com
SERVERS w2k8-phy-sonas.sonas.com
BASE_DN dc=sonas,dc=com
USER_DN dc=sonas,dc=com
USER_NAME_ATTRIB uid
USER_ID_ATTRIB CN
USER_FILTER none
KS_ADMIN_USER administrator
[root@c1n4 tmp]# /usr/lpp/mmfs/bin/mmuserauth service check --data-access-
method object -N cesNodes
Checking /etc/keystone/ssl/certs/object_ldap_cacert.pem: OK
LDAP/AD - TLS certificate Related issue…Continue

55
Scenario : Object authentication is configured LDAP and LDAP server is not reachable from one or more protocol nodes.
An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-237a98d6-9973-4251-9ae7-
f118eb214804)
[root@c1n3 ~]# /usr/lpp/mmfs/bin/mmuserauth service check --data-access-method object --server-reachability
LDAP servers status
LDAP server w2k8-phy-sonas.sonas.com : ERROR
[root@c1n3 ~]# /usr/lpp/mmfs/bin/mmuserauth service check --data-access-method object --server-reachability
LDAP servers status
LDAP server w2k8-phy-sonas.sonas.com : OK
+------------------------------+---------------+
| ID | Name |
+------------------------------+---------------+
| Administrator | Administrator |
LDAP/AD server is not reachable

Commonly occurring issues
AD/LDAP
• LDAP/AD is not reachable – Network issue/Server Down/Firewall issue
• LDAP/AD bind password changed/User deleted/Lockef/Permission changed
• LDAP/AD TLS certificate expired
• Swift user password changed/expired
• Swift user account locked/disabled
• Role of swift user removed from service account
External Keystone(Userdefined)
• External keystone is not rechable - Network issue/Server Down/Firewall issue
• Swift user deleted/Password Changed/Account locked
• Role changes on External Keystone
Local
• Postgresql is not running
• Password of swift user changed / User deleted
• Role change
56

Debugging
Check output of following commands
$mmuserauth service list
$mmuserauth service check --data-access-method object --nodes cesNodes
$mmuserauth service check --data-access-method object --nodes cesNodes --server-reachability
$mmces service list –v –a
$mmces events active
Enable Debugging :
1. CLI debugging - $mmces log level 3
2. Enabling debugging of keystone - /usr/lpp/mmfs/bin/mmobj config change --ccrfile keystone.conf --section DEFAULT
--property debug --value true
Note : Disable the debugging once problem is resolved. Debugging create lots of logs.
****** Do not modify any configuration file manually ******
Log Files to check if issue with object authentication
1. /var/adm/ras/mmfs.log*
2. /var/log/keystone/*
3. /var/log/message
4. /var/log/secure
5. /var/adm/ras/mmsysmonitor.log
57

IBM Spectrum Scale Authentication For Object - Deep Dive

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a IBM Spectrum Scale Authentication For Object - Deep Dive

Semelhante a IBM Spectrum Scale Authentication For Object - Deep Dive (20)

Último

Último (20)

IBM Spectrum Scale Authentication For Object - Deep Dive

Notas do Editor