CRS-4639 CRS-4124 Oracle High Availability Services startup failed in 11gR2 RAC

Today I have faced an issue with cluster OHASD service has not started automatically when server started. Then I tried to start cluster manually I received "CRS-4124: Oracle High Availability Services startup failed" error.

       
[oracle@rac1 ~]$ cd /u01/app/11.2.0/grid/bin
[oracle@rac1 bin]$ ./crsctl check crs
CRS-4639: Could not contact Oracle High Availability Services
[oracle@rac1 bin]$ 
[oracle@rac1 bin]$ su - root
Password: 
[root@rac1 ~]# cd /u01/app/11.2.0/grid/bin
[root@rac1 bin]# ./crsctl start crs
CRS-4124: Oracle High Availability Services startup failed
CRS-4000: Command Start failed, or completed with errors
[root@rac1 bin]#

Then I have verified cluster Oracle High Availability auto start-up is configured or not?

       
[root@rac1 bin]# ./crsctl config has
CRS-4622: Oracle High Availability Services autostart is enabled.
[root@rac1 bin]# 
 
 

Oracle High Availability Services auto-start is enabled. Then verified OLR accessibility on local node. Viewed ohasd.log and instance alert log files. Found below error messages and found as OHASD server was not running.

       
2014-08-29 18:15:18.620
[client(25949)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2014-08-29 18:15:18.622
[client(25949)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/rac1/client/emcrsp.log.

 

Parallely I have checked with infrastructure team about this server. They identified some local script has hang. Now I have started OHASD service in background and it's able to start the CRS.

       
[root@rac1 ~]# nohup /etc/init.d/init.ohasd run &
[1] 20959
[root@rac1 ~]#

[root@rac1 bin]# ./crsctl start crs
CRS-4640: Oracle High Availability Services is already active
CRS-4000: Command Start failed, or completed with errors.
[root@rac1 bin]# 
[root@rac1 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@rac1 bin]#

Custer has started.

Go with below of some useful 11gR2 RAC high alert issues :-)

Listener INTERMEDIATE status with "Not All Endpoints Registered" in 11gR2 RAC
ORA-04031 unable to allocate 3896 bytes of shared memory in ASM Instance

Thats It. Hope this will help you... :)

Best Regards,
Chowdari.