Step by Step to Install oracle grid 11.2.0.3 On solaris 11.1

1. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 1
Step By Step to Install Oracle Grid Infra-Structure 11.2.0.3 on Solaris 11.1
Introduction
Oracle Cluster ware is portable cluster software that allows clustering of independent servers so that they cooperate as a single system. Oracle
Cluster ware was first released with Oracle Database 10g Release 1 as the required cluster technology for Oracle Real Application Clusters
(RAC). Oracle Cluster ware is an independent cluster infrastructure, which is fully integrated with Oracle RAC, capable of protecting any kind of
application in a failover cluster.
Oracle Grid Infrastructure introduces a new server pool concept allowing the partitioning of the grid into groups of servers. “Role-separated
Management” can be used by organizations, in which cluster, storage, and database management functions are strictly separated. Cluster-aware
commands and an Enterprise Manager based cluster and resource management simplify grid management regardless of size. Further
enhancements in Oracle ASM, like the new ASM cluster file system or the new dynamic
Volume manager, complete Oracle’s new Grid Infrastructure solution.

2. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 2
About the Author
Osama Mustafa – Oracle ACE, a database specialist, Certified Oracle Professional (10g, 11g), Certified Ethical hacker (Penetration testing),
and Sun System Administrator, author of book Oracle Penetration Testing. Publishes many articles, including Oracle database articles in his blog.
Twitter: @OsamaOracle.
G+: Osama Mustafa.
SlidShare: Osama Mustafa.
LinkedIn: http://www.linkedin.com/in/osamamustafa.
Blog: http://osamamustafa.blogpsot.com.
The Final Architecture:
The Below Diagram Describe what is my setup, Note there’s no IP , Hostname will be mentioned in this document and if you notice any It will
not be the real one, This Part of the document only Describe the Grid and Database installation for Fusion middlware part it will be Published
Separately. Also if you follow this document included with other document you can build full Environment Configured and ready to use with
Single-Sign-On.
Rac Node Number: 2
Database Service Name: TEST
Database Location (Home): /u01/app/oracle/product/11.2.0/dbhome_1
Grid Location (Home): /u01/app/11.2.0/grid
Operating System (OS): Oracle SOLARIS SPARC 11.1
Note: All Software was downloaded from oracle OTN Here and e-delivery here.

3. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 3

4. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 4
Now after you already know what the setup will look like, Every Information about Locations, and Finally Operating system, during installation
for I faced lot of bugs Since Solaris 11.1 New But it was amazing experience and learn something new.
Let’s Start:
Step #1:
You need to know how the ect/hosts will look like after adding IP’s:
#########NODES#########
180.111.20.21 Test-db1
180.111.20.22 Test-db2
########################
#########NODE-One-IP###########
180.111.20.28 Test-db1-vip
10.0.0.1 Test-db1-priv
################################
#########NODE-Two-ip############
180.111.20.29 Test-db2-vip
10.0.0.2 Test-db2-priv
################################
######SCAN-IP##################
180.111.20.30 Test-db-scan
###############################
Step #2:
Check OS Version Using the below command:
/usr/platform/’uname –I’/sbin/prtdiag
Step #3 (Optional):
Because I was working remotely not directly from Data Center, I configure Vncserver to enable access to server GUI and Run the Installer from
there.
a. Install Required Package using
pkg info SUNWxvnc
b. Add the below line to /etc/services or use step “d” as command line
vnc-server 5900/tcp # Xvnc
c. Configure /etc/X11/gdm/custom/conf
[xdmcp]
Enable=true
[security]
DisallowTCP=false
AllowRoot=true
AllowRemoteRoot=true
d. Instead of step “b” you can do the below , I just want to mention both :
svccfg -s x11-server setprop options/tcp_listen=true
svccfg -s xvnc-inetd setprop inetd/wait=true
svccfg -s xvnc-inetd
setprop inetd_start/exec=astring:"/usr/bin/Xvnc -geometry 1280x720 -inetd -query
localhost -once securitytypes=none"
e. Finally Enable Services
svcadm restart gdm xvnc-inetd
svcadm disable gdm xvnc-inetd;
svcadm enable gdm xvnc-inetd

5. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 5
f. Just to make sure my work was Right I restart the Server and check vncsever again.
Step #4 (Optional):
1- I was thinking should I copy oracle software 4 times and it’s almost 35 GB so 35*4 you are talking about huge miss of time so
what I did here copy the files Once on one server and Configure NFS to share it between all Nodes, more easy and Save your
time.
a. First you have to enable NFS on All Server using the below command
svcadm enable nfs/server
svcadm restart nfs/server
b. If you copied files on server one then this command should be done on server one, if you copied the files on server 2
this command should be done on server 2. Depend where you copied software.
share -F nfs -o rw /base
c. On All Server you can run The Mount command to Share all the files and start setup, More Easy Save time.
mount -F nfs Base-Server-IP:software-location-on-remote-server mount-point-on-all-servers
Step #5: Prerequisite
As any Linux/Unix There is Prerequisite for Operating system follow the below
1. Users And Groups
Oracle Solaris 11 provide you with Command Called ZFS (amazing command to manage File system), I Create Oracle Home Using
this command, also create mount point /u01
This is For Oracle User
zfs create -o mountpoint=/u01 rpool/u01
Create Groups
groupadd -g 1001 oinstall
groupadd -g 1002 dba
groupadd -g 1003 oper
Create Oracle User
zfs create -o mountpoint=/export/home/oracle rpool/export/home/oracle
useradd -g oinstall -G dba oracle
passwd oracle
Create necessary Directory for Our Installation
chown -R oracle:oinstall /export/home/oracle
mkdir -p /u01/app/oracle
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/
chown -R oracle:oinstall /u01

6. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 6
This is For Grid User
Create Group for Grid User
groupadd -g 1020 asmadmin
groupadd -g 1022 asmoper
groupadd -g 1021 asmdba
Create Grid User
zfs create -o mountpoint=/export/home/grid rpool/export/home/grid
useradd -g oinstall -G dba grid
usermod -g oinstall -G dba,asmdba grid
passwd grid
chown -R grid:oinstall /export/home/grid
Create necessary Directory for Our Installation
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/grid
chown grid:oinstall /u01/app/11.2.0/grid
chown grid:oinstall /u01/app/grid
#!important!# chown -R grid:oinstall /u01
Step #6:
Configure .Profile which is located in /export/home/oracle and /export/home/grid
export ORACLE_BASE=/u01/app/oracle/
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
export GRID_HOME=/u01/app/11.2.0/grid/
export ORACLE_SID=TEST1
export
PATH=$PATH:/usr/sbin:/usr/X11/bin:/usr/dt/bin:/usr/openwin/bin:/usr/sfw/bin:/usr/sfw/sbin: /usr/ccs/bin: /usr/local/bin:/usr/local/sbin:
$ORACLE_HOME /bin:$GRID_HOME/bin:.
Note: copy .profile to all Database Servers using scp command
cd /export/home/oracle
scp .profile oracle@Server-ip:/export/home/oracle/
Step #7:
Oracle Grid and Networking Notes:
1. The broadcast must work across any configured VLANs as used by the public or private interfaces.
2. Across the broadcast domain as defined for the private interconnect
3. On the IP address subnet ranges 224.0.0.0/24 and 230.0.1.0/24
Regarding to oracle to you need to check udp time using the below command
ndd /dev/udp udp_xmit_hiwat
ndd /dev/udp udp_recv_hiwat
To avoid reboot you can set it on memory and reboot time using the below
On Memory
ndd -set /dev/udp udp_xmit_hiwat 65536
ndd -set /dev/udp udp_recv_hiwat 65536

7. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 7
On Reboot
ndd -set /dev/udp udp_xmit_hiwat 65536
ndd -set /dev/udp udp_recv_hiwat 65536
Step #8:
In this step you need to make sure of disks on both nodes, in my case I am using EMC storage.
 List Disk Using
/usr/sbin/format
 fdisk the raw disk using fdisk command.
 Change owner to grid using
Chown grid:asmadmin /dev/rdsk/Disk-name
Chmod 660 /dev/rdsk/disk-name
 You can list your available using
ls –ltr /dev/rdsk/emc*
 Change owner for these disk
Chown 660 emc*
Chown grid:asmadmin emc*
Note: In Unix Each Disk have slide from 0-6 like the below :
crw-rw---- 1 grid asmdba 302, 0 Apr 23 19:02 emcp@0:a,raw
crw-rw---- 1 grid asmdba 302, 8 Apr 23 20:38 emcp@1:a,raw
crw-rw---- 1 grid asmdba 302, 16 Apr 22 14:05 emcp@2:a,raw
crw-rw---- 1 grid asmdba 302, 24 Apr 23 21:00 emcp@3:a,raw
crw-rw---- 1 grid asmdba 302, 32 Apr 23 21:00 emcp@4:a,raw
crw-rw---- 1 grid asmdba 302, 40 Apr 23 21:00 emcp@5:a,raw
crw-rw---- 1 grid asmdba 302, 48 Apr 22 14:05 emcp@6:a,raw
Step #9:
Disable ntp using
svcadm disable ntp
Step #10:
By Default Oracle Solaris SPARC prevent root access, for example we created oracle and grid user but we cannot access to root using su
command to enable root access for oracle and grid do below
Edit file /etc/user_attr
Add the below lines
oracle::::defaultpriv=basic,net_privaddr;roles=root
grid::::defaultpriv=basic,net_privaddr;roles=root
Step #11:
Now we need to configure memory Parameter for Oracle and Grid User and make permanent
To check memory Current Value for Both user
prctl -n project.max-shm-memory -i process $$
Modify memory values using
projmod -a -K "project.max-shm-memory=(privileged,32G,deny)" -U oracle default
projmod -a -K "project.max-shm-memory=(privileged,32G,deny)" -U grid default
projmod -s -K "project.max-shm-memory=(privileged,32G,deny)" default

8. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 8
To make sure new memory value effect applied on oracle and grid user open new terminal and run prctl command again.
Step #12:
During installation Oracle will check swap memory so you need to increase swap memory depend on your setup for sure I will use ZFS
command.
Check swap value
bash-3.00# swap -lh
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 256,1 16 4194288 4194288
Remove Swap using root user
bash-3.00# swap -d /dev/zvol/dsk/rpool/swap
Configure New Swap
bash-3.00# zfs set volsize=20G rpool/swap
bash-3.00# swap -a /dev/zvol/dsk/rpool/swap
Check New Value
bash-3.00# swap -lh
Step #13:
One more step Configure SSH Between Nodes You can do this Step During installation or Using the the Below command, Oracle Provide you
with new way, you can now configure SSH using sshsetup it’s already exists within media
For Oracle User
# ./sshUserSetup.sh -hosts " node1 node2 node3 node4" -user oracle -advanced –noPromptPassphrase
For Grid User
# ./sshUserSetup.sh -hosts " node1 node2 node3 node4" -user grid -advanced –noPromptPassphrase
Congratulations you can start your setup now!!

9. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 9
Step #14:
Start Install Grid Infrastructure, in my case I choose to install Software only then Configure ASM in this way if error appeared I will know where
to start troubleshooting. Follow the screens ( this steps should be Done As Grid User )

10. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 10

11. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 11

12. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 12

13. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 13

14. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 14
Step #15:
After success installation now I need to Configure ASM ( This Step should be Done as Grid User) , From One Node Only.
export ORACLE_HOME=/u01/app/11.2.0/grid/
cd $ORACLE_HOME/bin
Run ./asmca
The Below Screen Should be Open
Three ASM Disks should be created:
1- DATA : DataFiles and Parameter files ( should be big )
2- FRA : FLASH RECOVERY AREA ( should be big )
3- CRS : this Disk for OCR and VOTING ( 4-5G will be enough )
After installation done you have to see MOUNTED on Both Node.
REDUNDANCY:
NORMAL REDUNDANCY - Two-way mirroring, requiring two failure groups.
HIGH REDUNDANCY - Three-way mirroring, requiring three failure groups.
EXTERNAL REDUNDANCY - No mirroring for disks that are already protected using hardware mirroring or RAID.
If you have hardware RAID it should be used in preference to ASM redundancy, so this will be the standard option for most installations.
Step #16:
Finally You Are done with Grid Infrastructure and now you should configure Database on RAC (this step should be done as Oracle User),
Usually I am installing Software only then Called dbca to configure Instance Install Software.

15. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 15

16. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 16

17. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 17

18. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 18

19. Step By Step Install Oracle Grid 11.2.0.3 on Solaris 11.1
Osama Mustafa Page 19
Done!!
You can reboot node to test your Setup.
Reference:
1- Oracle Documentation Here.
2- Oracle White Paper Here