How to STOP and START processes in Oracle RAC and Log Directory Structure
You need to follow the below steps to Stop and Start processes in Oracle RACShutdown RAC Database
You need to Shutdown Database instances on each node. You can either use Oracle Enterprise Manager or SVRCTL to shutdown the instances. If you are using EM Grid control then set a blackout in Grid control for processes that you intend to shutdown. So that records for these processes indicate that the shutdown was planned.
Use below command to stop Enterprise Manager/Grid Control
$ORACLE_HOME/bin/emctl stop dbconsole
Use below command to shutdown all oracle RAC instances on all nodes.
- $ ORACLE_HOME/bin/srvctl stop database -d db_name
- $ ORACLE_HOME/bin/srvctl stop database -d db_name –i instance_name
Once the database is stopped, proceed with ASM Instance shutdown.
Use below command to shutdown ASM instances on all nodes
- $ORACLE_HOME/bin/bin/srvctl stop asm -n node
Use below command to shutdown node apps on all RAC nodes
- $ORACLE_HOME/bin/bin/srvctl stop nodeapps -n node
You need to Shutdown oracle clusterware or CRS as root and run below command on each node in the cluster.
- #crsctl stop crs
From 11g R2, you can do this in two stops
1. Stop Clustwerware stack on local node
- #crsctl stop cluster
- # Crsctl stop cluster –all
-all Start clusterware on all nodes
-n Start clusterware on particular nodes
2. Stop Oracle High availability service demon on each node in the cluster.
- # crsctl stop has
Once all process stopped run the below command to check the status of CRSD,CSSD,EVMD process.
- # crsctl check crs
- $ crsctl stop crs –all –f
Follow the reverse sequence to start all processes in oracle RAC
- # crsctl start crs
- $ORACLE_HOME/bin/bin/srvctl start nodeapps -n node
- $ORACLE_HOME/bin/bin/srvctl start asm -n node
- $ORACLE_HOME/bin/srvctl start database -d db_name
Oracle Clusterware Log Directory Structure
CRS_HOME/log/hostname/crsd/ - The log files for the CRS daemon CRS_HOME/log/hostname/cssd/ - The log files for the CSS daemon
CRS_HOME/log/hostname/evmd/ - The log files for the EVM daemon
CRS_HOME/log/hostname/client/ - The log files for the Oracle Cluster Registry (OCR)
CRS_HOME/log/hostname/racg/ - The log files for the Oracle RAC high availability component
CRS_HOME/log/hostname/racg/ - The log files for the Oracle RAC high availability component
CRS_HOME/log/hostanme/alert.log – The alert.log for Clusterware issues.
Please note that the CRS_HOME is the directory in which the Oracle Clusterware software was installed and hostname is the name of the node
Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com
0 comments
Labels: ASM, OCR - VOTE, Oracle 11g New Feat, RAC Admin
Tuesday, February 16, 2010
ORA-600 [kfcNullConvert20] error on ASM instance
I have experienced ORA-600 error on ASM instance (10.2.0.2), which caused ASM instance restart on that particular node. Due to which all databases instances on that node are restarted, because they lost connectivity with ASM.
This issue is a known Bug 4682861 in Oracle and the effected version are 10.1.0.4 ,10.1.0.5 ,10.2.0.1 ,10.2.0.2. This BUG is fixed in 10.2.0.3 Server patch set and 11.1.0.6 Base release.
Check for Possible Error messages in ASM/Database alert.log and trace files.
If you find below SYMPTOMS then you are hitting a BUG 4682861.
ASM alert. log errors:
ORA-00600: internal error code, arguments: [kfcNullConvert20], [], [], [], [], [], [], []
DBW0: terminating instance due to error 471
Trace file (ASM DBWR trace file contains Stack Trace: kfcNullConvert and errors):-
error 600 detected in background process
ORA-00600: internal error code, arguments: [kfcNullConvert20], [], [], [], [], [], [], []
ksuitm: waiting for [5] seconds before killing DIAG
Database alert log errors:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
ASMB: terminating instance due to error 15064
Cause:
Due to Bug 4682861, ASM instance restarted.
Solution:
Option 1:- Upgrade to 10.2.0.3 or higher release
Option 2:-Download the patch from MetaLink:
i) Click on Patches & Updates Link.
ii) Enter patch number: 4682861 and Select your O/S
iii) Click Go.
iv) Download the patch for the Oracle release that you experienced this issue.
Please note that ANY ORA-600 error indicates Oracle has detected an internal inconsistency or a problem which it doesn’t know how best to address. These are *NOT* necessarily bugs and can occur for reasons such as running out of some resource, Operating System IO problems.. etc.
Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com
0 comments
Labels: ASM, Patch/Bug Fix, RAC-Errors
Monday, October 5, 2009
Create, Drop and Alter ASM disk groups
Crete Disk Group:
Create Disk groups using the CREATE DISKGROUP statement and specify the level of redundancy.
Disk group redundancy types:-
NORMAL REDUNDANCY - Two-way mirroring, requiring two failure groups.
HIGH REDUNDANCY - Three-way mirroring, requiring three failure groups.
EXTERNAL REDUNDANCY - No mirroring for disks that are already protected using hardware RAID or mirroring.
- SQL> CREATE DISKGROUP data NORMAL REDUNDANCY
FAILGROUP failure_group_2 DISK '/dev/sdb1' NAME datab1,'/dev/sdb2' NAME datab2;
Drop Disk Group:
Drop disk group using DROP DISKGROUP statement.
- SQL> DROP DISKGROUP data INCLUDING CONTENTS;
Alter Disk Group:
Add or remove disks from disk groups Using ALTER DISKGROUP statement. You can also use wildcard "*" to reference disks.
Add a disk.
- SQL> ALTER DISKGROUP data ADD DISK '/dev/datac1', '/dev/datac2';
Add all reference disks
- SQL> ALTER DISKGROUP data ADD DISK '/dev/datad*;
Drop/remove a disk.
- SQL> ALTER DISKGROUP data DROP DISK datab2;
The UNDROP command used to undo only pending drop of disks. After you drop the disks you cannot revert.
- SQL> ALTER DISKGROUP data UNDROP DISKS;
Diskgroup Rebalance:
Disk groups can be rebalanced manually Using REBALANCE clause and you can modify the POWER clause default value.
- SQL> ALTER DISKGROUP disk_group_1 REBALANCE POWER 5;
MOUNT and DISMOUNT DiskGroups:
Normally Disk groups are mounted at ASM instance startup and dismounted at shutdown. Using MOUNT and DISMOUNT options you can make one or more Disk Groups available or unavailable.
- SQL> ALTER DISKGROUP data MOUNT;
- SQL> ALTER DISKGROUP data DISMOUNT;
- SQL> ALTER DISKGROUP ALL MOUNT;
- SQL> ALTER DISKGROUP ALL DISMOUNT;
DiskGroup Check:
Use CHECK ALL to verify the internal consistency of disk group metadata and repair in case of any error.
- SQL> ALTER DISKGROUP data CHECK ALL;
DiskGroup resize:
Resize the one or all disks in the Diskgroup.
Resize all disks in a failure group.
- SQL> ALTER DISKGROUP data RESIZE DISKS IN FAILGROUP failure_group_1 SIZE 1024G;
Resize a specific disk.
- SQL> ALTER DISKGROUP data RESIZE DISK dataa1 SIZE 1024G;
Resize all disks in a disk group.
- SQL> ALTER DISKGROUP data RESIZE ALL SIZE 1024G;
Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com/
1 comments
Labels: ASM
Migrate Database to ASM Using RMAN
We are Using RMAN to relocate non-ASM files to ASM files. The ASM files cannot be accessed through normal OS interfaces.
Step1: Query V$CONTROLFILE and V$LOGFILE to get the file names.
- SQL> select * from V$CONTROLFILE;
- SQL> select * from V$LOGFILE;
Step 2: Shutdown the database.
- SQL> SHUTDOWN IMMEDIATE;
Step3: Modify the target database parameter file:
(i) Remove the CONTROL_FILES parameter from the spfile, so the control files will be created automatically in ASM Disk group mentioned in DB_CREATE_FILE_DEST destination
Using a pfile then set CONTROL_FILES parameter to the appropriate ASM files or aliases.
(ii) Set the DB_CREATE_FILE_DEST and DB_CREATE_ONLINE_LOG_DEST_n parameters to the relevant ASM disk groups.
Step 4: Startup nomount mode and Restore control file
- RMAN> STARTUP NOMOUNT;
Restore the control file into the new location.
- RMAN> RESTORE CONTROLFILE FROM 'old_controlfile_name';
Step 5: Startup mount mode and backup the Database.
- RMAN> ALTER DATABASE MOUNT;
Backup database into ASM disk group.
- RMAN> BACKUP AS COPY DATABASE FORMAT '+diskgroup1';
Step 6: Switch database and create or rename Redo log members
Switch all data files to the new ASM Diskgroup location.
- RMAN> SWITCH DATABASE TO COPY;
- RMAN> SQL “ALTER DATABASE RENAME ‘old_redolog_member’ to ‘+diskgroup2’;
Create new redo logs in ASM Disk group and delete the old redo log files.
Step 7: Open Database and create temporary tablespace.
Open database using resetlogs
- SQL> ALTER DATABASE OPEN RESETLOGS;
Create temporary tablespace in ASM disk group.
- SQL> CREATE TABLESPACE temp1 ADD TEMPFILE ‘+diskgroup1’;
Step 8: Drop old database files.
- 1. SQL> DROP TABLESPACE ‘old_temporary_tablespace’ including contents and datafiles;
Regards
Satishbabu Gunukula
http://www.oracleracexpert.com/
4 comments
Labels: ASM, Migrate, RMAN Recovery
Tuesday, September 29, 2009
Createdisk, Deletedisk and Querydisk in ASM
The /etc/init.d/oracleasm script is used to create, delete and query ASM disks and make disks available.
Create and Delete ASM disk:
Run below command to Create ASM disks.
- # /etc/init.d/oracleasm createdisk DATA1 /dev/sdc
Run below command to Delete ASM disks.
- # /etc/init.d/oracleasm deletedisk DATA1
Query/List/Scan ASM disk:
Run the below querydisk command to see if the Disk/Device is used by ASM:
- # /etc/init.d/oracleasm querydisk /dev/sdc
- # /etc/init.d/oracleasm querydisk DATA1
Run below command to list Existing disks
- # /etc/init.d/oracleasm listdisks
Run the below command to scan the disks created on another node.
- # /etc/init.d/oracleasm scandisks
Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com/
0 comments
Labels: ASM
Monday, August 31, 2009
Delete Archivelog files without using RMAN
Here I am explaining two methods to delete archive logs and other database files with out using RMAN from ASM Disk Group in Oracle 10g/11g.
Method 1: asmcmd - ASM command-line utility
- ASMCMD> rm file_name
- For ex:-ASMCMD> rm ‘+dgroup2/testdb/archivelogs/thread_1_seq_363.510.1’
- Or
- ASMCMD> rm ‘+dgroup2/testdb/datafile/USERS.250.5334166963’
- If you use a wildcard, rm deletes all matches except non-empty directories (unless the -r flag is used). The rm command can delete the file or alias only if the file is not currently in use by a client database.
Method 2: SQLPLUS utility
- SQL> ALTER DISKGROUP DROP file
The “asmcmd” and “sqlplus” commands will not update the database views (V$ARCHIVED_LOG, V$FLASH_RECOVERY_AREA_USAGE), controlfile, Recovery Catalog that the files have been removed.
To update the Database views, control file or RMAN Catalog about deleted files you need to run the below command from RMAN.
- RMAN> CROSSCHECK ARCHIVELOG ALL;
- RMAN> DELETE EXPIRED ARCHIVELOG ALL;
Click here to learn How to delete archive logs from ASM.
Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com/
0 comments
Labels: ASM
Monday, August 17, 2009
Install and Configure ASMLib in 10g (Automatic Storage Management)
The Oracle ASM feature was introduced in Oracle 10g Release 1.There are two methods to configure ASM on Linux.
- 1. Configure ASM with ASMLib I/O: This method creates all Oracle database files on raw block devices, which are managed by ASM using ASMLib calls. ASMLib works with block devices and raw devices are not required with this method.
- 2. Configure ASM with Standard Linux I/O: This method creates Oracle database files on raw character devices, which are managed by ASM using standard Linux I/O system calls. It requires creating RAW devices for all disk partitions used by the ASM.
Here we will “Configure ASM with ASMLib I/O” method.
Step 1: Download and Install ASMLib
Download Oracle “ASMLib” software from below link and follow the link for your platform.
http://www.oracle.com/technology/tech/linux/asmlib/index.html
You must install all three packages for the kernel you are running. Use “uname –r “command to determine the version of your kernel.
oracleasm-support-version.cpu_type.rpm
oracleasm-kernel-version.cpu_type.rpm
oracleasmlib-version.cpu_type.rpm
See the below example to install the packages and run the command as root.
- # rpm -ivh oracleasm-support-2.0.3-1.x86_64.rpm \
> oracleasmlib-2.0.2-1.x86_64.rpm
Step 2: Configure and Enable Oracle ASM
Run the below command to configure the Oracle ASM and it will ask for the user and group that default to owing the ASM drivers access point.
- # /etc/init.d/oracleasm configure
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Fix permissions of Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration [ OK ]
Creating /dev/oracleasm mount point [ OK ]
Loading module "oracleasm" [ OK ]
Mounting ASMlib driver filesystem [ OK ]
Scanning system for ASM disks [ OK ]
This command will load the ASM driver and mount the ASM driver filesystem. By selecting “y” during the configuration, the system will always load the module and mount the file system on system boot.
Run the below command to enable automatic start
- #/etc/init.d/oracleasm enable
Regards,
Satishbabu Gunukula
http://www.oracleracexpert.com/
0 comments
Labels: ASM, Installation
