Oracle RAC Cluster Maintenance | RAC Startup Sequence | Startup Troubleshooting Guide | Interview Q&A

Clusterware Maintenance: (Bouncing) & Database Maintenance: (Bouncing) 

======================================================================

HAS -> CRS -> ASM -> DB 

DB: oracle (DEVDB1/DEVDB2)

=== 

Node1:

srvctl status instance -i DEVDB1 -d DEVDB

srvctl stop instance -i DEVDB1 -d DEVDB

srvctl start instance -i DEVDB1 -d DEVDB

sqlplus / as sysdba

shut immediate

startup

Node2:

srvctl status instance -i DEVDB2 -d DEVDB

srvctl stop instance -i DEVDB2 -d DEVDB

srvctl start instance -i DEVDB2 -d DEVDB

sqlplus / as sysdba

shut immediate

startup

Node1 & Node2: Entire DB downtime 

srvctl status database -d DEVDB

srvctl stop database -d DEVDB

srvctl start database -d DEVDB

from both the nodes

sqlplus / as sysdba

shut immediate

startup

ASM: oracle (+ASM1/+ASM2)

====

node1:

srvctl status asm 

srvctl stop asm

srvctl start asm

node2:

srvctl status asm

srvctl stop asm

srvctl start asm

CRS: root (+ASM1/+ASM2)

=====

node1:

crsctl check crs 

crsctl stop crs 

crsctl start crs 

node2:

crsctl check crs 

crsctl stop crs 

crsctl start crs 

HAS: root (+ASM1/+ASM2)

====

node1:

crsctl check has 

crsctl stop has 

crsctl start has 

node2:

crsctl check has 

crsctl stop has 

crsctl start has 

Cluster Start-Up sequence:

==========================

crsctl stop crs/has

crsctl start crs/has

crsctl stat res -t -init 

-> Few cluster resource are not starting 

-> cssd is not coming up 

-> gpnpd is not comping up 

What is the reason? 

- No answer until we check those respective logs 

Question where are these logs?

-> $ORACLE_BASE/diag -> /u01/app/oracle/diag

- cluster resource logs -> /u01/app/oracle/diag/crs (/u01/app/oracle/diag/crs/oraclelab1/crs/trace)

- asm logs -> /u01/app/oracle/diag/asm (/u01/app/oracle/diag/asm/+asm/+ASM1/trace) 

- database logs -> /u01/app/oracle/diag/rdbms (/u01/app/oracle/diag/rdbms/devdb/DEVDB1/trace)

- listener log -> /u01/app/oracle/diag/tnslsnr/oraclelab1

cluster startup troubleshooting 

+++++++++++++++++++++++++++++++++

crsctl stop crs/has

crsctl start crs/has

crsctl stat res -t -init 

- cluster resource logs -> /u01/app/oracle/diag/crs (/u01/app/oracle/diag/crs/oraclelab1/crs/trace)

Cluster StartUp sequence:

==========================

OHAS -> LEVEL1 -> LEVEL2 -> LEVEL3 -> LEVEL4

OHASD -> oracle high availability service 

HAS is not starting? (Oracle High availability service demon) 

- OLR is missing or corrupted  (Oracle local registry) 

cssd / cssdmonitor - cluster synchronization service demon 

CSSD is not starting?

- voting disks are missing or corrupted 

CRSD - cluster ready service 

- read OCR file (Oracle Central registry)

CRSD is not starting?

- OCR is missing or corrupted  (Oracle Central registry) 

What is OLR, OCR, Votedisk?  

where they are located? 

Regards,

Mallikarjun / Vismo Technologies

WhatsApp: +91 9880616848 / +91 9036478079

Cell: +91 9880616848 / +91 9036478079

Email: mallikarjun.ramadurg@gmail.com / vismotechnologies@gmail.com