eTEAM
Oracle 10g RAC Installation Guide
Document Information
Document Owner: Prepared by: Document Version No.: Preparation Date: eTeam DBA Karthik V.01.00.00 August 11th, 2006
Reviewers
Name Position
Version History
Version No. V.01.00.01 Version Date 08/11/06 Revised by Description Initial Document Creation Filename
ADP Confidential
Page 1 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Document Information................................................................................................... 1 Reviewers ..................................................................................................................... 1 Version History ............................................................................................................. 1 RAC Architecture Overview .......................................................................................... 4 System Requirements................................................................................................... 5
Storage Foundation........................................................................................................................................... 5 EMC Shared Storage and I/O Fencing ........................................................................................................... 5 Private and Virtual IP’s ................................................................................................................................... 5
ifconfig -a on d1sol5................................................................................................................................... 6 ifconfig -a on d1sol6................................................................................................................................... 6
UNIX Preinstallation Steps .............................................................................................................................. 7
Create Oracle User and Group .................................................................................................................. 7 Configure Secure Shell -SSH..................................................................................................................... 7 Configure rsh ............................................................................................................................................. 8 Set Environment variables ......................................................................................................................... 8 Configure Kernel Resources ...................................................................................................................... 9
Before the Installation .................................................................................................................................... 11
Pre-Install checks for 10gR2 RDBMS (10.2.x) - SUN Solaris Platforms ................................................... 11 Copying SFRAC Libraries to /opt/ORCLcluster/lib ................................................................................... 12 Creating $CRS_HOME on each node...................................................................................................... 12 Create volumes for CRS and VOTE disk on shared disk ......................................................................... 12 Modify the CVM service group (main.cf) to add CRS resources............................................................... 13 Final Checkup before installing CRS and Oracle Binaries....................................................................... 14
Installing CRS for Oracle 10g........................................................................................................................ 14 Installing the Oracle 10g Binaries ................................................................................................................. 35 After Oracle Binary Installation.................................................................................................................... 52
ADP Confidential Page 2 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Copying and Verifying IPC and VCSMM libraries..................................................................................... 52 Verifying ODM library............................................................................................................................... 52
Adding Oracle 10g Patches 10.2.0.2 .............................................................................................................. 53
Adding patch 10.2.0.2 to CRS.................................................................................................................. 53 Unzip the file ............................................................................................................................................ 53 Set the Environment Variables................................................................................................................. 53 Stop all Oracle processes ........................................................................................................................ 53 Adding patch 10.2.0.2 to Oracle Binaries ................................................................................................. 64 Post Installation ....................................................................................................................................... 76 Mandatory Patch for 10.2.0.2................................................................................................................... 78
main.cf with CRS resources ......................................................................................................................... 118
ADP Confidential
Page 3 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
RAC Architecture Overview
Oracle Real Application Clusters (RAC) allows Oracle Database to run any packaged or custom application, unchanged across a set of clustered servers. This provides the highest levels of availability and the most flexible scalability. If a clustered server fails, Oracle continues running on the remaining servers. In addition, when you need more processing power, simply add another server without taking users offline. To keep costs low, even the highest-end systems can be built out of standardized, commodity parts. Oracle Real Application Clusters provides a foundation for Oracle’s Enterprise Grid Computing Architecture. Oracle RAC technology enables a low-cost hardware platform to deliver the highest quality of service that rivals and exceeds the levels of availability achieved by the most expensive, mainframe SMP computers. By dramatically reducing administration costs and providing new levels of administration flexibility, Oracle is enabling the enterprise Grid environment. Real Application Clusters run on top of a hardware cluster. A cluster is a group of independent servers (nodes) that cooperate as a single system. The primary cluster components are processor nodes, a cluster interconnect, and a shared storage subsystem. The nodes share access to the storage subsystem and resources that manage data, but they do not physically share main memory in their respective nodes. Oracle cluster database combines the memory in the individual nodes to provide a single view of the distributed cache memory for the entire database system. A node can be made up of multiple processors. Each node has its own dedicated system memory as well as its own operating system and database instance.
ADP Confidential
Page 4 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
System Requirements
Please consult with the infrastructure team on the requirements for Sun servers, EMC Storage, Cluster Interconnects, Public networks, Switch options, Memory, Swap and CPU. Moreover, check the RAC/Veritas on Sun certification matrix for RAC updates on currently supported hardware/software in Metalink. Before the installation of Oracle 10g RAC on Solaris 10, make sure following components has been installed and configured appropriately. Storage Foundation Installed Veritas Storage foundation for Oracle Real Application Cluster (SFRAC 4.1 packages with MP1) and Clustering configured on all Nodes that are part of the Cluster. EMC Shared Storage and I/O Fencing Verify that the shared disk arrays support SCSI-3 persistent reservations and I/O fencing. Three types of shared storage required 1. Data disks to store shared data, created based on the file system layout, 2. CRS disks to store OCR and VOTE volumes and Directories 3. Coordinator disks for I/O fencing, at least three-coordinator disk possibly smallest LUN’s on different disk arrays to resolve potential split-brain conditions and prevent data corruption. Private and Virtual IP’s Apart from host IP, make sure a public virtual IP exist on public NIC and a private IP address on the Private NIC. Refer to page 44 and 47 on Release Notes: VERITAS Storage Foundation 4.1 for Oracle RAC Maintenance Pack 1 Document id 280145 regarding creation of Private and Public IP address. Hostname d1sol5 d1sol6 Host IP 192.168.201.203 192.168.201.204 Virtual IP d1sol5-virt-192.168.201.193 d1sol6-virt-192.168.201.194 Private IP d1sol5-priv-10.10.10.1 d1sol6-priv10.10.10.2
ADP Confidential
Page 5 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
ifconfig -a on d1sol5
Oracle 10g RAC Installation Guide
ip for Veritas Cluster
Public Virtual ip on Public NIC
Private ip on Private NIC
ifconfig -a on d1sol6
Public Virtual ip on Public NIC
Private ip on Private NIC
ADP Confidential
Page 6 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
UNIX Preinstallation Steps
Create Oracle User and Group
Oracle 10g RAC Installation Guide
Logon as root and Create a local group (dba and oinstall) and local user (oracle) # groupadd -g 201 dba # groupadd -g 202 oinstall # useradd -c "Oracle Software Owner" -g dba -G oinstall -u 201 -m -d /export/home/oracle -s /bin/ksh oracle Create a password for user oracle
# passwd oracle
To verify the oracle account,
# id oracle
Verifying that the User nobody Exists
# id nobody Configure Secure Shell -SSH
From each node, logon as oracle
$ mkdir ~/.ssh $ chmod 755 ~/.ssh $ /usr/bin/ssh-keygen -t rsa
Cut and paste the following line separately
$ /usr/bin/ssh-keygen -t dsa
From the first node (d1sol5) ONLY, logged in as oracle (copy the local account's keys so that ssh to the local node will work):
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
Now copy the keys to the other node (d1sol6) so that we can ssh to the remote node without being prompted for a password.
$ ssh oracle@d1sol6 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
(Run each of them separately. SSH will prompt for the oracle password each time and if the commands are pasted at the same time, the other commands will be lost when the first one flushes the input buffer prior to prompting for the password.)
$ ssh oracle@d1sol6 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ADP Confidential Page 7 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
$ $ $ $ $ $
Oracle 10g RAC Installation Guide
chmod 644 ~/.ssh/authorized_keys cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys ssh oracle@d1sol5 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys ssh oracle@d1sol5 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys chmod 644 ~/.ssh/authorized_keys
As oracle on the node where the Oracle 10g Release 2 software will be installed (d1sol5)
$ exec /usr/bin/ssh-agent $SHELL $ /usr/bin/ssh-add
To Test
$ ssh d1sol6 date
Should connect to the remote host without password prompt, If not refer to Metalink and Oracle Document for further information on setting up the ssh.
Configure rsh
Make sure that the user Oracle can rsh between the nodes without having to use or confirm a password. Veritas Vxfentsthdw command requires “+” in /.rhosts file. If not, in the user oracle account directory /export/home/oracle, create a .rhosts file that includes a plus character(“+”) in the first line. On Node1 (d1sol5) create a sample test.txt file and copy the file to node2 (d1sol6). Logon to node2 (d1sol6) and check if the file exists. This tests both rsh and rcp services.
$ $ $ $ $ $ touch test.txt rcp test.txt d1sol6:/tmp rsh d1sol6 ls -al /tmp/test.txt rm /tmp/test.txt rm test.txt
Set Environment variables
Edit .profile file in /export/home/oracle directory and add following lines, save it
ORACLE_BASE=/opt/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/10g/db_1; export ORACLE_HOME CRS_HOME=$ORACLE_BASE/product/10g/crs; export CRS_HOME LD_LIBRARY_PATH_64=$ORACLE_HOME/lib:$CRS_HOME/lib; export LD_LIBRARY_PATH_64 LD_LIBRARY_PATH=$ORACLE_HOME/lib32; export LD_LIBRARY_PATH CLASSPATH=$CLASSPATH:$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/j lib:$ORACLE_HOME/network/jlib; export CLASSPATH ORACLE_TERM=xterm; export ORACLE_TERM PATH=$ORACLE_HOME/bin:/usr/ccs/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/lo cal/sbin:/sbin:/bin::/usr/lib/vxvm/bin:/opt/VRTSvxfs/sbin:/opt/VRTSvcs/bin:/
Page 8 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
ADP Confidential
eTEAM
Oracle 10g RAC Installation Guide
opt/VRTS/bin:/opt/VRTSvcs/rac/bin:/opt/VRTSob/bin:/etc:/opt/EMCpower/bin/spa rcv9:/etc/emc/bin:/usr/dt/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/ucb:/et c:/usr/openwin/bin:.; export PATH EDITOR=vi; export EDITOR set -o vi stty istrip stty erase ^H umask 022 export umask ulimit -n 8192 >/dev/null 2>&1 ulimit -s 32768 >/dev/null 2>&1 PS1=`hostname`':$PWD[$ORACLE_SID]$ ' ULIMIT=999999999999999 export ULIMIT HISTSIZE=1000 export HISTSIZE HISTFILE=/var/userlog/`who am i|cut -f1 -d' '`.log export HISTFILE alias h=history ORAENV_ASK=NO MAIL=/usr/mail/${LOGNAME:?} Configure Kernel Resources
Prior to Solaris 10, the System V IPC resources, consisting primarily of shared memory, message queues, and semaphores, were set in the /etc/system file and reconfiguration needs a reboot. In Solaris 10 it is no longer requires changes in the /etc/system file. Instead, it uses the resource control facility. For more information, refer to “8.2 Appendix-2 Setting System V IPC kernel parameters” in Metalink Doc ID: 317257.1 Parameter set shmsys:shminfo_shmmax set shmsys:shminfo_shmmin set shmsys:shminfo_shmmni set semsys:seminfo_semmni set semsys:seminfo_semmns set semsys:seminfo_semmnu set semsys:seminfo_semopm set semsys:seminfo_semvmx set shmsys:shminfo_shmseg set semsys:seminfo_semmap set semsys:seminfo_semmsl set semsys:seminfo_semume
ADP Confidential
Value 18446744073709551615 1 256 0x1002 0x7631 0x7631 4096 32767 256 0x7631 4096 512
Solaris 10 Resource Control project.max-shm-memory N/A project.max-shm-ids project.max-sem-ids N/A Obsolete project.max-sem-ops Obsolete N/A Obsolete project.max-sem-nsems Obsolete
Default Value 1/4 of physical memory 128 128 512
512
Page 9 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Before moving further, please refer to Bug No 5237047 regarding INCORRECT SYSTEM REQUIREMENTS FOR SOLARIS 10 in Metalink On each node, logon as root
# # # # # # projadd projmod projmod projmod projmod projmod -U -s -s -s -s -s oracle -G dba -c "OracleProject" -p 201 user.oracle -K "project.max-shm-memory=(priv,10g,deny)" user.oracle -K "project.max-shm-ids=(priv,256,deny)" user.oracle -K "project.max-sem-ids=(priv,4096,deny)" user.oracle -K "project.max-sem-ops=(priv,4096,deny)" user.oracle -K "project.max-sem-nsems=(priv,4096,deny)" user.oracle
To set UDP parameters to 65536 bytes, write a startup script in /etc/init.d
# cd /etc/init.d # touch udp_rac # chmod 775 udp_rac # vi udp_rac ---------------------------Script Start------------------------------------#!/sbin/sh case "$1" in 'start') ndd -set /dev/udp udp_xmit_hiwat 65536 ndd -set /dev/udp udp_recv_hiwat 65536 ;; . 'state') ndd /dev/udp udp_xmit_hiwat ndd /dev/udp udp_recv_hiwat ;; . *) echo "Usage: $0 { start | state }" exit 1 ;; esac ---------------------------Script End--------------------------------------# cd /etc/rc3.d # ln -s /etc/init.d/udp_rac /etc/rc3.d/S86udp_rac
to view/verify the settings logon as oracle and run
$ prctl $$
ADP Confidential
Page 10 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
As the srvctl utility not unable to get the right SHM related settings using prctl/projmod, Add following entries to /etc/system file and reboot the servers. For more information, Please refer to Metalink Doc id 367442.1
forceload: sys/semsys forceload: sys/shmsys set shmsys:shminfo_shmmax=18446744073709551615 set shmsys:shminfo_shmmin=1 set shmsys:shminfo_shmmni=256 set semsys:seminfo_semmni=0x1002 set semsys:seminfo_semmns=0x7631 set semsys:seminfo_semmnu=0x7631 set semsys:seminfo_semopm=4096 set semsys:seminfo_semvmx=32767 set shmsys:shminfo_shmseg=256 set semsys:seminfo_semmap=0x7631 set semsys:seminfo_semmsl=4096 set semsys:seminfo_semume=512 set set set set set set tune_t_fsflushr=30 autoup=900 rlim_fd_max=65536 rlim_fd_cur=32768 sq_max_size=0 tcp:tcp_conn_hash_size=8192
Reboot the servers
ulimit settings, following command will display the current ulimit settings: $ ulimit -a
If needed add following entries to the .profile for oracle to reset stack and nofiles values
ulimit -n 4096 >/dev/null 2>&1 ulimit -s 32768 >/dev/null 2>&1
Before the Installation
Pre-Install checks for 10gR2 RDBMS (10.2.x) - SUN Solaris Platforms Refer to Doc ID: 334567.1
Logon as Oracle Set the oracle environment variables, ORACLE_HOME Or validate 10gR2_rdbms_solaris_hcve_091705.txt
$ cd /ora/scr/preinstall $ tar xvf sun_10gR2_hcve_rdbms_091705.tar $ perl validate.pl 10gR2_rdbms_solaris_hcve_091705.txt
Please review the log files and rectify the components that failed the test.
ADP Confidential Page 11 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Copying SFRAC Libraries to /opt/ORCLcluster/lib
Logon as root and Copy SFRAC libraries to enable RAC recognize the cluster environment
# cp /opt/VRTSvcs/rac/lib/libskgxn2_64.so /opt/ORCLcluster/lib/libskgxn2.so Creating $CRS_HOME on each node
CRS and Oracle Binaries will be installed locally on each server. Make sure $CRS_HOME is under $ORACLE_BASE. Make directories for ORACLE_BASE is /opt/oracle CRS_HOME is /opt/oracle/product/10g/crs ORACLE_HOME is /opt/oracle/product/10g/db_1
Create volumes for CRS and VOTE disk on shared disk
Create OCR and VOTE-disk volumes and directories on the cluster file system. Assume that emcpower13s2 is the allocated LUN, execute following commands as root on one of the node (d1sol5)
# # # # # # # # vxdg -s init crs_oradg emcpower13s2 vxassist -g crs_oradg make crs_vol 200M vxvol -g crs_oradg startall mkfs -F vxfs /dev/vx/rdsk/crs_oradg/crs_vol mkdir mount chown chmod /ora_crs -F vxfs -o cluster /dev/vx/dsk/crs_oradg/crs_vol /ora_crs oracle:dba /ora_crs 755 /ora_crs
On both nodes (d1sol5 and d1sol6)
On one node (d1sol5), logon as oracle
$ cd /ora_crs $ mkdir CRS_OCR $ mkdir VOTE-disk
ADP Confidential
Page 12 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Modify the CVM service group (main.cf) to add CRS resources.
Refer to the Modified main.cf in the appendix Log into one of the nodes in the cluster (d1sol5) as root
# # # # # haconf -dump makero hastop -all -force cd /etc/VRTSvcs/conf/config cp main.cf main.orig vi main.cf
Include the following entries, At the beginning of the file
include “PrivNIC.cf”
And in CVM service group
CFSMount crs_ora_mnt ( MountPoint = "/ora_crs" BlockDevice = "/dev/vx/dsk/crs_oradg/crs_vol" ) CVMVolDg crs_voldg ( CVMDiskGroup = crs_oradg CVMVolume = { crs_vol } CVMActivation = sw ) PrivNIC ora_priv ( Device = { ipge2 = 0, ipge3 = 1 Address @d1sol5 = "10.10.10.1" Address @d1sol6 = "10.10.10.2" NetMask = "255.255.255.0" ) crs_ora_mnt requires crs_voldg crs_ora_mnt requires vxfsckd crs_voldg requires cvm_clus }
And in dependencies
Save and Close main.cf Verify the syntax Copy main.cf to the second node (d1sol6) now start VCS in all nodes And check the VCS status
#hastatus –summ
ADP Confidential Page 13 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
# hacf –verify /etc/VRTSvcs/conf/config #hastart
eTEAM
Oracle 10g RAC Installation Guide
All added resources should be online on both the nodes
Final Checkup before installing CRS and Oracle Binaries
Logon as oracle 1. Check ssh config for local node and remote node 2. Use prctl $$ to check the semaphores values 3. Check the Oracle user profile settings 4. Get host ip, public virtual ip and it’s interface, private ip and it’s interface name. 5. Bring down the ip on public virtual interface (as root) # ifconfig ipge0:2 down 6. Create volumes and directories for CRS and VOTE disk 7. CRS resources added to the veritas cluster main.cf and restart SFRAC 8. Copied SFRAC libskgxn2_64.so 9. Check the DISPLAY environmental variable, run xclock and verify 10.
Installing CRS for Oracle 10g Make sure all required software is under /ora/scr directory. Logon to node1 (d1sol5) as Oracle
$ cd /ora/scr/crs $ ./runInstaller
Use ./runInstaller –ignoreSysPrereqs to ignore the prerequisites
ADP Confidential
Page 14 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 15 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 16 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Change the Name to CRS_HOME and path to /opt/oracle/product/10g/crs
ADP Confidential
Page 17 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 18 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Click Edit and modify Virtual hostnames (d1sol5_vip to d1sol5_virt and d1sol6_vip t d1sol6_virt) to match the names in /etc/hosts
ADP Confidential
Page 19 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ipge0 is the public nic, so click Edit and change ipge0 interface type to Public
ADP Confidential
Page 20 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 21 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 22 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 23 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 24 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Don’t click the OK button now. Run the scripts one by one on each node, configure Virtual ip’s using vipca and then comeback and click OK
ADP Confidential
Page 25 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Logon as root on node1 (d1sol5) and run
# /opt/oracle/oraInventory/orainstRoot.sh and # /opt/oracle/product/10g/crs/root.sh
ADP Confidential
Page 26 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Logon as root on 2nd node (d1sol6) and run
# /opt/oracle/oraInventory/orainstRoot.sh and # /opt/oracle/product/10g/crs/root.sh
ADP Confidential
Page 27 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Make sure CSS is active on both the nodes At the end of the script, we might receive following error messages saying “ipge0 is not public”
Running vipca(silent) for configuring nodeapps The given interface(s), "ipge0" is not public. Public interfaces should be used to configure virtual IPs.
This is a known issues with vipca, Refer to page 60 of Release Notes: VERITAS Storage Foundation 4.1 for Oracle RAC Maintenance Pack 1 Document id 280145. please invoke vipca manually
root@d1sol6 # cd /oracle/product/10g/crs/bin root@d1sol6 # ./vipca
ADP Confidential
Page 28 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 29 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 30 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Enter Virtual IP alias name, d1sol5-virt.campd.local, press tab remaining column values will be filled automatically
ADP Confidential
Page 31 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 32 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Click OK after running the scripts and vipca
ADP Confidential
Page 33 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 34 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Installing the Oracle 10g Binaries Log on as oracle
$ cd /ora/scr/db $ ./runInstaller
ADP Confidential
Page 35 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 36 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Select Custom
ADP Confidential
Page 37 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Change Name to ORACLE_HOME and Path to /opt/oracle/product/10g/db_1
ADP Confidential
Page 38 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 39 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 40 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 41 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Ignore the OUI finding in the area of “noexec_user_stack for the Solaris 10 OS (Refer Metalink 377679.1)
Checking for noexec_user_stack=1; found no entry.Failed <<<< Check complete. The overall result of this check is: Failed <<<<
ADP Confidential
Page 42 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 43 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 44 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 45 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 46 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Don’t click OK now
ADP Confidential
Page 47 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Logon as root on node1 (d1sol5) and run
# /opt/oracle/product/10g/db_1/root.sh
ADP Confidential
Page 48 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Logon as root on node1 (d1sol6) and run
#/opt/oracle/product/10g/db_1/root.sh
ADP Confidential
Page 49 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 50 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 51 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
After Oracle Binary Installation
Copying and Verifying IPC and VCSMM libraries
Logon as oracle and copy
$ cp /opt/VRTSvcs/rac/lib/libskgxp102_64.so $ORACLE_HOME/lib/libskgxp10.so Verifying ODM library
Login as Oracle user
$ cd /opt/oracle/product/10g/db_1/lib $ mv /opt/oracle/product/10g/db_1/lib/libodm10.so /opt/oracle/product/10g/db_1/lib/libodm10.so.old $ ln -s /usr/lib/sparcv9/libodm.so libodm10.so $ ls -al /opt/oracle/product/10g/db_1/lib/libodm10.so
(makesure owner is oracle) On d1sol5
On d1sol6
ADP Confidential
Page 52 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Adding Oracle 10g Patches 10.2.0.2 (Metalink Patch # 4547817)
Adding patch 10.2.0.2 to CRS
Should always upgrade Oracle Clusterware before you upgrade Oracle RAC Binaries If we are installing this patch set on an existing Oracle Real Applications Cluster (RAC) 10g release 10.2.0.1 installation, we must run Oracle Universal Installer from the same node(d1sol5) from which the Oracle Database software was installed.
Unzip the file $ unzip p4547817_10202_SOLARIS64.zip Set the Environment Variables
Set ORACLE_HOME, CRS_HOME in the .profile of Oracle user.
Stop all Oracle processes
Login as oracle user On d1sol5 $ srvctl stop nodeapps -n d1sol5
On d1sol6
$ srvctl stop nodeapps -n d1sol6
Logon as root Shut down the Oracle Clusterware process by entering the following command on all nodes as the root user:
ADP Confidential
Page 53 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
On d1sol5
Oracle 10g RAC Installation Guide
# /etc/init.d/init.crs stop
On d1sol6
# /etc/init.d/init.crs stop
Make sure there is no Oracle processes running
$ ps -ef|grep oracle|grep -v grep
Log in as the Oracle user on node1 (d1sol5) Start Oracle Universal Installer
$ cd /ora/scr/patch/10.2.0.2/Disk1 $ ./runInstaller
On the Welcome screen, click Next.
ADP Confidential
Page 54 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 55 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Change the Name to CRS_HOME and Path to /opt/oracle/product/10g/crs
ADP Confidential
Page 56 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 57 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 58 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Don’t click on Exit
ADP Confidential
Page 59 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Don’t click on Exit
ADP Confidential
Page 60 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Login as root on d1sol5 and run
# /opt/oracle/product/10g/crs/bin/crsctl stop crs # /opt/oracle/product/10g/crs/install/root102.sh
To Check is crs is stopped
# ps -ef|grep d.bin
Refer to Metalink Doc id 358258.1 regarding crsctl stop crs, Failure 15 At Css Context Initialization When Shutting Down Crs
ADP Confidential
Page 61 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Login as root on node 2 (d1sol6) and run
# /opt/oracle/product/10g/crs/bin/crsctl stop crs # /opt/oracle/product/10g/crs/install/root102.sh
ADP Confidential
Page 62 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Click Exit now, after running the scripts as root user
ADP Confidential
Page 63 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Adding patch 10.2.0.2 to Oracle Binaries
Login as Oracle User Stop all Oracle Processes On d1sol5 $ srvctl stop nodeapps -n d1sol5
On d1sol6 $ srvctl stop nodeapps -n d1sol6
Login as root User Shut down the Oracle Clusterware process by entering the following command on all nodes as the root user: On d1sol5
# /etc/init.d/init.crs stop
On d1sol6
# /etc/init.d/init.crs stop
Make sure there is no Oracle processes running
# ps -ef|grep oracle|grep -v grep
ADP Confidential Page 64 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Log in as the oracle user on node1(d1sol5) Start Oracle Universal Installer
$ cd /ora/scr/patch/10.2.0.2/Disk1 $ ./runInstaller
ADP Confidential
Page 65 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 66 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Enter Name as ORACLE_HOME and Path as /opt/oracle/product/10g/db_1
ADP Confidential
Page 67 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 68 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 69 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 70 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Don’t click OK now
ADP Confidential
Page 71 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Login as root on d1sol5 and run
# /opt/oracle/product/10g/db_1/root.sh
ADP Confidential
Page 72 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Login as root on d1sol5 and run
# /opt/oracle/product/10g/db_1/root.sh
ADP Confidential
Page 73 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 74 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 75 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Post Installation
Oracle 10g RAC Installation Guide
During patchset installation, all new files and directories are created with restricted access, by default. Users or third party applications with a different group identifier from that of the database, which try to access client-side utilities or libraries in the database home, will see permission errors when trying to access these files or directories. Perform the following steps to change the permissions: Change your directory to:
$ cd $ORACLE_HOME/install
Run changePerm.sh and specify the patched server Oracle home location, before accessing client-side utilities or libraries in the database home. For RAC home, we need to run this script on all the nodes. Login as Oracle On d1sol5
$ cd /opt/oracle/product/10g/db_1/install $ ./changePerm.sh
Look for errors in /tmp/changePerm_err.log and resolve it.
# chown oracle:dba externaljob.ora # chmod 644 externaljob.ora
ADP Confidential Page 76 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
On d1sol6
Oracle 10g RAC Installation Guide
$ cd /opt/oracle/product/10g/db_1/install $ ./changePerm.sh
ADP Confidential
Page 77 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Mandatory Patch for 10.2.0.2
Oracle 10g RAC Installation Guide
After installing 10.2.0.2 apply the mandatory patch for (bug 5117016) for incorrectly located in $ORACLE_HOME/RDBMS/LIB/ For more information, Refer Metalink Doc id 5117016.8 This patch is applicable only on the server side. Ensure all the processes started out of the ORACLE_HOME are being shutdown prior to applying this patch using srvctl , on every node
$ srvctl stop nodeapps -n d1sol5 $ srvctl stop nodeapps -n d1sol6
To apply the patch, unzip the PSE container file:
$ unzip p5117016_10202_SOLARIS64.zip
Set your current directory to the directory where the patch is located
$ cd /ora/scr/patch/5117016
Ensure that the directory containing the opatch script appears in the $PATH, then run the following command on every node
$ $ORACLE_HOME/OPatch/opatch apply -local
ADP Confidential
Page 78 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
On d1sol5
Oracle 10g RAC Installation Guide
ADP Confidential
Page 79 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
On d1sol6
Oracle 10g RAC Installation Guide
ADP Confidential
Page 80 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Check the inventory
Oracle 10g RAC Installation Guide
$ $ORACLE_HOME/OPatch/opatch lsinventory
ADP Confidential
Page 81 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Oracle® Database Critical Patch Update Release 10.2.0.2 Patch 5225799 Ensure that the Oracle Database installation is the same release for which we are applying this patch. In other words, only apply the Release 10.2.0.2 CPUJul2006 patch to an Oracle Database Release 10.2.0.2. Review Section 4, "Known Issues" before proceeding with this installation. 1. Shut down the instance on one node (d1sol5) 2. Shut down all nodeapps services on the node (d1sol5) in step 1:
$ srvctl stop nodeapps -n d1sol5
Apply patch on the node (d1sol5) in step 1 3. Set current directory to the directory where the patch is located and then run OPatch by entering the following command:
$ cd /ora/scr/patch/cpu/5225799 $ORACLE_HOME/OPatch/opatch apply OPatch.SKIP_VERIFY=true
(We need to specify OPatch.SKIP_VERIFY=true. For more information please refer to 353150.1 - OPatch Failing Validation Phase When Archiving Really Did Not Fail) 4. If there are errors, refer to Section 4, "Known Issues". Start all nodeapps services on the node in step 1:
$ srvctl start nodeapps -n d1sol5
Shut down the instance on the next node. Repeat steps 1-6 on all the nodes in the cluster. Check the opatch inventory Logon as root and start crs on both nodes
# /etc/init.d/init.crs start
ADP Confidential
Page 82 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Check the inventory
$ $ORACLE_HOME/OPatch/opatch lsinventory
ADP Confidential
Page 83 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Creating the Oracle Database
CRS should be running
To check crs daemon running or not $CRS_HOME/bin/crs_stat
Create the mount points and directories for the Oracle Database
Login as root and
# # # # # # # # # # # # # mkdir /hapdb cd /hapdb mkdir archive flashdb data1 index1 redo1 redo2 redocopy system temp mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_archive_vol /hapdb/archive mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_flashdb_vol /hapdb/flashdb mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_data1_vol /hapdb/data1 mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_index1_vol /hapdb/index1 mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_redo1_vol /hapdb/redo1 mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_redo2_vol /hapdb/redo2 mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_redocopy_vol /hapdb/redocopy mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_system_vol /hapdb/system mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_temp_vol /hapdb/temp chown -R oracle:dba archive flashdb data1 index1 redo1 redo2 redocopy system temp
Verify all file system exist and owner is Oracle Default template is already created for this database. So Copy hapdb.dbt to /opt/oracle/product/10g/db_1/assistants/dbca/templates. If not select Custom or General Purpose database template
Before creating the database using dbca, configure LISTENER using netca. $ netca
ADP Confidential
Page 84 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 85 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 86 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 87 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 88 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 89 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Creating 10g database using dbca
Logon as Oracle
$ dbca
ADP Confidential
Page 90 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 91 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 92 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 93 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
If hapdb template is not available select Custom Database option
ADP Confidential
Page 94 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 95 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 96 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 97 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 98 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 99 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 100 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 101 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 102 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 103 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 104 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 105 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Click on All Initialization Parameters and Change the location for all dump (a/b/c/udump) init. parameters
ADP Confidential
Page 106 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 107 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 108 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 109 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 110 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 111 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 112 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 113 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ADP Confidential
Page 114 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
After creating the database, check spfile is in /hapdb/system/admin/pfile/ directory if not
$ cd $ORACLE_HOME/dbs
Change the location of spfile in inithapdb1.ora to SPFILE='/hapdb/system/admin/pfile/spfilehapdb.ora' And move the file from /hapdb/data1/hapdb to /hapdb/system/admin/pfile/ (No issue when used a different template, not hapdb_new ) To check the status
root@d1sol6 # /opt/oracle/product/10g/crs/bin/crs_stat -t Name Type Target State Host -----------------------------------------------------------ora....L5.lsnr application ONLINE ONLINE d1sol5 ora.d1sol5.gsd application ONLINE ONLINE d1sol5 ora.d1sol5.ons application ONLINE ONLINE d1sol5 ora.d1sol5.vip application ONLINE ONLINE d1sol5 ora....L6.lsnr application ONLINE ONLINE d1sol6 ora.d1sol6.gsd application ONLINE ONLINE d1sol6 ora.d1sol6.ons application ONLINE ONLINE d1sol6 ora.d1sol6.vip application ONLINE ONLINE d1sol6
ADP Confidential Page 115 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
ora.hapdb.db application ora....b1.inst application ora....b2.inst application
ONLINE ONLINE ONLINE
ONLINE ONLINE ONLINE
d1sol5 d1sol5 d1sol6
To check the service status
$ srvctl status service –d hapdb
Configure the CVM and Oracle Service Groups for Oracle 10g Log into one of the nodes in the cluster (d1sol5) as root
# # # # # haconf -dump makero hastop -all -force cd /etc/VRTSvcs/conf/config cp main.cf main.before10gdb vi main.cf
Refer to the Modified and final main.cf in the appendix Save and Close main.cf Verify the syntax Copy main.cf to the second node (d1sol6) now start VCS in all nodes And check the VCS status All added resources should be online on both the nodes
#hastatus –summ #hastart # hacf –verify /etc/VRTSvcs/conf/config
If voting is disk unavailable, server will be rebooted every few minutes, to avoid the reboot disable the crs
# /etc/init.d/init.crs disable
Correct the issue and reenable it and reboot the server If Veritas Agent is used for Detail Monitoring (Secondary monitoring) then
create user vcsmon identified by vcsmon default tablespace TOOLS temporary tablespace TEMP quota 500K on TOOLS
ADP Confidential Page 116 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
# /etc/init.d/init.crs enable # reboot
eTEAM
Oracle 10g RAC Installation Guide
/ grant create session to vcsmon / create table vcsmon. vcstable_d1sol5 ( tstamp date ) / insert into vcsmon. vcstable_d1sol5 ( tstamp ) values ( SYSDATE ) / create table vcsmon. vcstable_d1sol6 ( tstamp date ) / insert into vcsmon. vcstable_d1sol6 ( tstamp ) values ( SYSDATE ) /
To scp and unzip the files
root@D1SOL5 # scp 10gr2_cluster_sol.cpio.gz root@d1sol6:/opt/oracle/software/crs/ 10gr2_cluster_sol.cp 100% |********************************************************************************** *********| 368 MB 00:51 root@D1SOL5 # gunzip 10gr2_cluster_sol.cpio.gz root@D1SOL5 # cpio -idmv <10gr2_cluster_sol.cpio
Stopping CRS Before Stopping VCS in Oracle 10g Clusters In SFRAC clusters with Oracle 10g, stop the CRS daemons before using the hastop -all command to stop VCS. The required sequence of commands is: 1. Stop Oracle on each node: 2. Stop the CRS daemons on each node: 3. Stop VCS. From one node, type:
# hastop -all # hagrp -offline oracle_group -sys -system_name # /etc/init.d/init.crs stop
After 10.2.0.2 patchset is installed, RDBMS crashes with internal errors on remote nodes Doc id 365851.1 1. Ensure all instances are shut down 2. Manually copy $ORACLE_HOME/rdbms/lib/libknlopt.a from the local node to all remote nodes . 3. Relink Oracle on all remote nodes eg: cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk ioracle
ADP Confidential
Page 117 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
main.cf with CRS resources
include include include include include "types.cf" "CFSTypes.cf" "CVMTypes.cf" "OracleTypes.cf" "PrivNIC.cf"
cluster d1sol-c ( UserNames = { admin = IhiAhcHeiDiiGqiChf } ClusterAddress = "192.168.201.192" Administrators = { admin } CredRenewFrequency = 0 UseFence = SCSI3 HacliUserLevel = COMMANDROOT CounterInterval = 5 ) system d1sol5 ( )
ADP Confidential Page 118 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
system d1sol6 ( ) group ClusterService ( SystemList = { d1sol5 = 0, d1sol6 = 1 } AutoStartList = { d1sol5, d1sol6 } OnlineRetryLimit = 3 OnlineRetryInterval = 120 ) IP webip ( Device = ipge0 Address = "192.168.201.192" NetMask = "255.255.255.0" ) NIC csgnic ( Device = ipge0 ) VRTSWebApp VCSweb ( Critical = 0 AppName = vcs InstallDir = "/opt/VRTSweb/VERITAS" TimeForOnline = 5 RestartLimit = 3 ) VCSweb requires webip webip requires csgnic // resource dependency tree // // group ClusterService // { // VRTSWebApp VCSweb // { // IP webip // { // NIC csgnic // } // } // } group cvm ( SystemList = { d1sol5 = 0, d1sol6 = 1 } AutoFailOver = 0
ADP Confidential Page 119 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Parallel = 1 AutoStartList = { d1sol5, d1sol6 } ) CFSMount crs_ora_mnt ( MountPoint = "/ora_crs" BlockDevice = "/dev/vx/dsk/crs_oradg/crs_vol" ) CVMVolDg crs_voldg ( CVMDiskGroup = crs_oradg CVMVolume = { crs_vol } CVMActivation = sw ) PrivNIC ora_priv ( Device = { ipge2 = 0, ipge3 = 1 Address @d1sol5 = "10.10.10.1" Address @d1sol6 = "10.10.10.2" NetMask = "255.255.255.0" ) CFSQlogckd qlogckd ( Critical = 0 ) CFSfsckd vxfsckd ( ) CVMCluster cvm_clus ( CVMClustName = d1sol-c CVMNodeId = { d1sol5 = 0, d1sol6 = 1 } CVMTransport = gab CVMTimeout = 200 ) CVMVxconfigd cvm_vxconfigd ( Critical = 0 CVMVxconfigdArgs = { syslog } ) crs_ora_mnt requires crs_voldg crs_ora_mnt requires vxfsckd crs_voldg requires cvm_clus cvm_clus requires cvm_vxconfigd qlogckd requires cvm_clus vxfsckd requires qlogckd // resource dependency tree
ADP Confidential Page 120 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
}
eTEAM
// // // // // // // // // // // // // // // // // // // // // // // // //
Oracle 10g RAC Installation Guide
group cvm { CFSMount crs_ora_mnt { CVMVolDg crs_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } } PrivNIC ora_priv }
ADP Confidential
Page 121 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
main.cf with CVM and Oracle Service Groups and without the Oracle Agent
include include include include include "types.cf" "CFSTypes.cf" "CVMTypes.cf" "OracleTypes.cf" "PrivNIC.cf"
cluster d1sol-c ( UserNames = { admin = IhiAhcHeiDiiGqiChf, vcsop = GjkIjgJgjRkd } ClusterAddress = "192.168.201.192" Administrators = { admin } Operators = { vcsop } CredRenewFrequency = 0 UseFence = SCSI3 HacliUserLevel = COMMANDROOT CounterInterval = 5 ) system d1sol5 ( ) system d1sol6 ( ) group ClusterService ( SystemList = { d1sol5 = 0, d1sol6 = 1 } AutoStartList = { d1sol5, d1sol6 } OnlineRetryLimit = 3 OnlineRetryInterval = 120 ) IP webip ( Device = ipge0 Address = "192.168.201.192" NetMask = "255.255.255.0" ) NIC csgnic ( Device = ipge0 ) NotifierMngr ntfr ( SmtpServer = "160.0.10.50" SmtpRecipients = { "kappuk@lexus.elabor.com" = Warning } ) VRTSWebApp VCSweb (
ADP Confidential Page 122 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
Critical = 0 AppName = vcs InstallDir = "/opt/VRTSweb/VERITAS" TimeForOnline = 5 RestartLimit = 3 ) VCSweb requires webip webip requires csgnic // resource dependency tree // // group ClusterService // { // VRTSWebApp VCSweb // { // IP webip // { // NIC csgnic // } // } // NotifierMngr ntfr // } group cvm ( SystemList = { d1sol5 = 0, d1sol6 = 1 } AutoFailOver = 0 Parallel = 1 AutoStartList = { d1sol5, d1sol6 } ) Application cssd ( Critical = 0 StartProgram = "/opt/VRTSvcs/rac/bin/cssd-online" StopProgram = "/opt/VRTSvcs/rac/bin/cssd-offline stopcrs" CleanProgram = "/opt/VRTSvcs/rac/bin/cssd-clean" MonitorProgram = "/opt/VRTSvcs/rac/bin/cssd-monitor" OnlineRetryLimit = 20 ) CFSMount crs_ora_mnt ( MountPoint = "/ora_crs" BlockDevice = "/dev/vx/dsk/crs_oradg/crs_vol" ) CFSMount hapdb_archive_mnt ( Critical = 0 MountPoint = "/hapdb/archive"
ADP Confidential Page 123 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_archive_vol" ) CFSMount hapdb_data1_mnt ( Critical = 0 MountPoint = "/hapdb/data1" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_data1_vol" ) CFSMount hapdb_flashdb_mnt ( Critical = 0 MountPoint = "/hapdb/flashdb" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_flashdb_vol" ) CFSMount hapdb_index1_mnt ( Critical = 0 MountPoint = "/hapdb/index1" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_index1_vol" ) CFSMount hapdb_redo1_mnt ( Critical = 0 MountPoint = "/hapdb/redo1" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redo1_vol" ) CFSMount hapdb_redo2_mnt ( Critical = 0 MountPoint = "/hapdb/redo2" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redo2_vol" ) CFSMount hapdb_redocopy_mnt ( Critical = 0 MountPoint = "/hapdb/redocopy" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redocopy_vol" ) CFSMount hapdb_system_mnt ( Critical = 0 MountPoint = "/hapdb/system" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_system_vol" ) CFSMount hapdb_temp_mnt ( Critical = 0 MountPoint = "/hapdb/temp" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_temp_vol" )
ADP Confidential Page 124 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
CFSQlogckd qlogckd ( Critical = 0 ) CFSfsckd vxfsckd ( ) CVMCluster cvm_clus ( CVMClustName = d1sol-c CVMNodeId = { d1sol5 = 0, d1sol6 = 1 } CVMTransport = gab CVMTimeout = 200 ) CVMVolDg crs_voldg ( CVMDiskGroup = crs_oradg CVMVolume = { crs_vol } CVMActivation = sw ) CVMVolDg hapdb_archive_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_archive_vol } CVMActivation = sw ) CVMVolDg hapdb_data1_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_data1_vol } CVMActivation = sw ) CVMVolDg hapdb_flashdb_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_flashdb_vol } CVMActivation = sw ) CVMVolDg hapdb_index1_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_index1_vol } CVMActivation = sw ) CVMVolDg hapdb_redo1_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_redo1_vol } CVMActivation = sw )
ADP Confidential Page 125 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
CVMVolDg hapdb_redo2_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_redo2_vol } CVMActivation = sw ) CVMVolDg hapdb_redocopy_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_redocopy_vol } CVMActivation = sw ) CVMVolDg hapdb_system_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_system_vol } CVMActivation = sw ) CVMVolDg hapdb_temp_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_temp_vol } CVMActivation = sw ) CVMVxconfigd cvm_vxconfigd ( Critical = 0 CVMVxconfigdArgs = { syslog } ) PrivNIC ora_priv ( Device = { ipge2 = 0, ipge3 = 1 } Address @d1sol5 = "10.10.10.1" Address @d1sol6 = "10.10.10.2" NetMask = "255.255.255.0" ) crs_ora_mnt requires crs_voldg crs_ora_mnt requires vxfsckd crs_voldg requires cvm_clus cssd requires crs_ora_mnt cssd requires hapdb_archive_mnt cssd requires hapdb_data1_mnt cssd requires hapdb_flashdb_mnt cssd requires hapdb_index1_mnt cssd requires hapdb_redo1_mnt cssd requires hapdb_redo2_mnt cssd requires hapdb_redocopy_mnt cssd requires hapdb_system_mnt cssd requires hapdb_temp_mnt
ADP Confidential Page 126 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
cssd requires ora_priv cvm_clus requires cvm_vxconfigd hapdb_archive_mnt requires hapdb_archive_voldg hapdb_archive_mnt requires vxfsckd hapdb_archive_voldg requires cvm_clus hapdb_data1_mnt requires hapdb_data1_voldg hapdb_data1_mnt requires vxfsckd hapdb_data1_voldg requires cvm_clus hapdb_flashdb_mnt requires hapdb_flashdb_voldg hapdb_flashdb_mnt requires vxfsckd hapdb_flashdb_voldg requires cvm_clus hapdb_index1_mnt requires hapdb_index1_voldg hapdb_index1_mnt requires vxfsckd hapdb_index1_voldg requires cvm_clus hapdb_redo1_mnt requires hapdb_redo1_voldg hapdb_redo1_mnt requires vxfsckd hapdb_redo1_voldg requires cvm_clus hapdb_redo2_mnt requires hapdb_redo2_voldg hapdb_redo2_mnt requires vxfsckd hapdb_redo2_voldg requires cvm_clus hapdb_redocopy_mnt requires hapdb_redocopy_voldg hapdb_redocopy_mnt requires vxfsckd hapdb_redocopy_voldg requires cvm_clus hapdb_system_mnt requires hapdb_system_voldg hapdb_system_mnt requires vxfsckd hapdb_system_voldg requires cvm_clus hapdb_temp_mnt requires hapdb_temp_voldg hapdb_temp_mnt requires vxfsckd hapdb_temp_voldg requires cvm_clus qlogckd requires cvm_clus vxfsckd requires qlogckd // resource dependency tree // // group cvm // { // Application cssd // { // CFSMount crs_ora_mnt // { // CFSfsckd vxfsckd // { // CFSQlogckd qlogckd // { // CVMCluster cvm_clus // { // CVMVxconfigd cvm_vxconfigd // } // }
ADP Confidential Page 127 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
// // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // //
ADP Confidential
Oracle 10g RAC Installation Guide
} CVMVolDg crs_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_archive_mnt { CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_archive_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_data1_mnt { CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_data1_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_index1_mnt
Page 128 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
// // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // //
ADP Confidential
Oracle 10g RAC Installation Guide
{ CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_index1_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_redo1_mnt { CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_redo1_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_redo2_mnt { CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } }
Page 129 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
// // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // //
ADP Confidential
Oracle 10g RAC Installation Guide
} CVMVolDg hapdb_redo2_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_redocopy_mnt { CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_redocopy_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_system_mnt { CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_system_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_temp_mnt
Page 130 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
// // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // // //
Oracle 10g RAC Installation Guide
}
{ CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_temp_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CFSMount hapdb_flashdb_mnt { CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } CVMVolDg hapdb_flashdb_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } PrivNIC ora_priv }
ADP Confidential
Page 131 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
main.cf with CVM and Oracle Service Groups and with the Oracle Agent
(make sure table required for detail monitoring is already created) include "types.cf" include "CFSTypes.cf" include "CVMTypes.cf" include "OracleTypes.cf" include "PrivNIC.cf" cluster d1sol-c ( UserNames = { admin = IhiAhcHeiDiiGqiChf } ClusterAddress = "192.168.201.192" Administrators = { admin } CredRenewFrequency = 0 UseFence = SCSI3 HacliUserLevel = COMMANDROOT CounterInterval = 5 ) system d1sol5 ( ) system d1sol6 ( ) group ClusterService ( SystemList = { d1sol5 = 0, d1sol6 = 1 } AutoStartList = { d1sol5, d1sol6 } OnlineRetryLimit = 3 OnlineRetryInterval = 120 ) IP webip ( Device = ipge0 Address = "192.168.201.192" NetMask = "255.255.255.0" ) NIC csgnic ( Device = ipge0 )
ADP Confidential Page 132 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
NotifierMngr ntfr ( SmtpServer = "160.0.10.50" SmtpRecipients = { "kappuk@lexus.elabor.com" = Warning } ) VRTSWebApp VCSweb ( Critical = 0 AppName = vcs InstallDir = "/opt/VRTSweb/VERITAS" TimeForOnline = 5 RestartLimit = 3 ) VCSweb requires webip webip requires csgnic // resource dependency tree // // group ClusterService // { // VRTSWebApp VCSweb // { // IP webip // { // NIC csgnic // } // } // NotifierMngr ntfr // } group cvm ( SystemList = { d1sol5 = 0, d1sol6 = 1 } AutoFailOver = 0 Parallel = 1 AutoStartList = { d1sol5, d1sol6 } ) Application cssd ( Critical = 0 StartProgram = "/opt/VRTSvcs/rac/bin/cssd-online" StopProgram = "/opt/VRTSvcs/rac/bin/cssd-offline stopcrs" CleanProgram = "/opt/VRTSvcs/rac/bin/cssd-clean" MonitorProgram = "/opt/VRTSvcs/rac/bin/cssd-monitor" OnlineRetryLimit = 20 ) CFSMount crs_ora_mnt ( MountPoint = "/ora_crs"
ADP Confidential Page 133 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
BlockDevice = "/dev/vx/dsk/crs_oradg/crs_vol" ) CFSQlogckd qlogckd ( Critical = 0 ) CFSfsckd vxfsckd ( ) CVMCluster cvm_clus ( CVMClustName = d1sol-c CVMNodeId = { d1sol5 = 0, d1sol6 = 1 } CVMTransport = gab CVMTimeout = 200 ) CVMVolDg crs_voldg ( CVMDiskGroup = crs_oradg CVMVolume = { crs_vol } CVMActivation = sw ) CVMVxconfigd cvm_vxconfigd ( Critical = 0 CVMVxconfigdArgs = { syslog } ) PrivNIC ora_priv ( Device = { ipge2 = 0, ipge3 = 1 } Address @d1sol5 = "10.10.10.1" Address @d1sol6 = "10.10.10.2" NetMask = "255.255.255.0" ) crs_ora_mnt requires crs_voldg crs_ora_mnt requires vxfsckd crs_voldg requires cvm_clus cssd requires crs_ora_mnt cssd requires ora_priv cvm_clus requires cvm_vxconfigd qlogckd requires cvm_clus vxfsckd requires qlogckd // resource dependency tree // // group cvm // { // Application cssd
ADP Confidential Page 134 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
// // // // // // // // // // // // // // // // // // // // // // // //
Oracle 10g RAC Installation Guide
}
{ CFSMount crs_ora_mnt { CVMVolDg crs_voldg { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } CFSfsckd vxfsckd { CFSQlogckd qlogckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } } } PrivNIC ora_priv }
group hapdb_rac ( SystemList = { d1sol5 = 0, d1sol6 = 1 } AutoFailOver = 0 Parallel = 1 AutoStartList = { d1sol5, d1sol6 } ) CFSMount hapdb_archive_mnt ( Critical = 0 MountPoint = "/hapdb/archive" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_archive_vol" ) CFSMount hapdb_data1_mnt ( Critical = 0 MountPoint = "/hapdb/data1" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_data1_vol" ) CFSMount hapdb_flashdb_mnt ( Critical = 0 MountPoint = "/hapdb/flashdb" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_flashdb_vol" )
ADP Confidential Page 135 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
CFSMount hapdb_index1_mnt ( Critical = 0 MountPoint = "/hapdb/index1" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_index1_vol" ) CFSMount hapdb_redo1_mnt ( Critical = 0 MountPoint = "/hapdb/redo1" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redo1_vol" ) CFSMount hapdb_redo2_mnt ( Critical = 0 MountPoint = "/hapdb/redo2" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redo2_vol" ) CFSMount hapdb_redocopy_mnt ( Critical = 0 MountPoint = "/hapdb/redocopy" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redocopy_vol" ) CFSMount hapdb_system_mnt ( Critical = 0 MountPoint = "/hapdb/system" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_system_vol" ) CFSMount hapdb_temp_mnt ( Critical = 0 MountPoint = "/hapdb/temp" BlockDevice = "/dev/vx/dsk/ora_dg1/ora_temp_vol" ) CVMVolDg hapdb_archive_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_archive_vol } CVMActivation = sw ) CVMVolDg hapdb_data1_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_data1_vol } CVMActivation = sw ) CVMVolDg hapdb_flashdb_voldg (
ADP Confidential Page 136 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
CVMDiskGroup = ora_dg1 CVMVolume = { ora_flashdb_vol } CVMActivation = sw ) CVMVolDg hapdb_index1_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_index1_vol } CVMActivation = sw ) CVMVolDg hapdb_redo1_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_redo1_vol } CVMActivation = sw ) CVMVolDg hapdb_redo2_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_redo2_vol } CVMActivation = sw ) CVMVolDg hapdb_redocopy_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_redocopy_vol } CVMActivation = sw ) CVMVolDg hapdb_system_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_system_vol } CVMActivation = sw ) CVMVolDg hapdb_temp_voldg ( CVMDiskGroup = ora_dg1 CVMVolume = { ora_temp_vol } CVMActivation = sw ) Oracle Ora-hapdb ( Sid @d1sol5 = hapdb1 Sid @d1sol6 = hapdb2 Owner = oracle Home = "/oracle/product/10g/db_1" Pfile @d1sol5 = "/opt/oracle/product/10g/db_1/dbs/inithapdb1.ora" Pfile @d1sol6 = "/opt/oracle/product/10g/db_1/dbs/inithapdb2.ora" EnvFile = "/opt/VRTSvcs/bin/Oracle/envfile" MonScript = "/opt/VRTSvcs/bin/Oracle/SqlTest.pl"
ADP Confidential Page 137 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
Oracle 10g RAC Installation Guide
DetailMonitor = 1 User = vcsmon Pword = vcsmon Table @d1sol5 = vcstable_d1sol5 Table @d1sol6 = vcstable_d1sol6 ) requires group cvm online local firm Ora-hapdb requires hapdb_archive_mnt Ora-hapdb requires hapdb_data1_mnt Ora-hapdb requires hapdb_index1_mnt Ora-hapdb requires hapdb_redo1_mnt Ora-hapdb requires hapdb_redo2_mnt Ora-hapdb requires hapdb_redocopy_mnt Ora-hapdb requires hapdb_system_mnt Ora-hapdb requires hapdb_temp_mnt Ora-hapdb requires hapdb_flashdb_mnt hapdb_archive_mnt requires hapdb_archive_voldg hapdb_data1_mnt requires hapdb_data1_voldg hapdb_index1_mnt requires hapdb_index1_voldg hapdb_redo1_mnt requires hapdb_redo1_voldg hapdb_redo2_mnt requires hapdb_redo2_voldg hapdb_redocopy_mnt requires hapdb_redocopy_voldg hapdb_system_mnt requires hapdb_system_voldg hapdb_temp_mnt requires hapdb_temp_voldg hapdb_flashdb_mnt requires hapdb_flashdb_voldg // resource dependency tree // // group hapdb_rac // { // Oracle Ora-hapdb // { // CFSMount hapdb_archive_mnt // { // CVMVolDg hapdb_archive_voldg // CVMVolDg hapdb_data1_voldg // CVMVolDg hapdb_index1_voldg // CVMVolDg hapdb_redo1_voldg // CVMVolDg hapdb_redo2_voldg // CVMVolDg hapdb_redocopy_voldg // CVMVolDg hapdb_system_voldg // CVMVolDg hapdb_temp_voldg // } // CFSMount hapdb_data1_mnt // CFSMount hapdb_index1_mnt // CFSMount hapdb_redo1_mnt // CFSMount hapdb_redo2_mnt // CFSMount hapdb_redocopy_mnt
ADP Confidential Page 138 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.
eTEAM
// // // // // //
Oracle 10g RAC Installation Guide
CFSMount hapdb_system_mnt CFSMount hapdb_temp_mnt } CFSMount hapdb_flashdb_mnt CVMVolDg hapdb_flashdb_voldg }
ADP Confidential
Page 139 of 139 Last modified by kappukuttan ©2006 Automatic Data Processing, Inc. All Rights Reserved.