Workshop on Oracle 11gR2 DBA: RAC and ASM

02 Mar

23, 24, 25, 26 February 2013
Hotel Lemon Tree, Electronic City, Bangalore, India.

Workshop on Oracle 11gR2 DBA – RAC and ASM

Recently I had attended the workshop on Oracle RAC and ASM at Electronic City, Bangalore, India. The training was given by RAC guru Murali Vallath. I am sharing my learning and experience from this workshop.

I sincerely thank Murali for inspiring and encouraging me to learn the latest feature in Oracle – RAC and ASM.

The training batch comprised participants from different geographical location and spanning different industries. I feel lucky to be part of this batch.  Lastly, I thank Sai Ram for organizing this workshop.

TrainingBatchStanding (L to R): Syed, Abul, Chandra, Amit, Prabhakar, Vijay (me), Vasu,  Rukmini
Sitting (L to R): Bikash, Gautam, Murali, Kushal, Milind, Vivek, Sathish
Photo clicked by Sai Ram, Venue: Lemon Tree Hotel, Electronic City, Bangalore, India

What is Clustering?

Clustering is the grouping of individual computers in such a way that they can act as a single computer system. These combined resources can then be presented to applications as a single system.  One benefit of a cluster is that the application does not need to be cluster “aware”, and requires no special coding to take advantage of most cluster related services.

Oracle clustering is a shared everything architecture that involves sharing of storage by the systems of the cluster.

Oracle RAC:

Oracle RAC stands for Real Application Cluster. It was introduced by Oracle with 9i release. Latest version is 11gR2.

Oracle RAC is a cluster database with a shared cache architecture that overcomes the limitations of traditional shared-nothing and shared-disk approaches to provide highly scalable and available database solutions for all your business applications. Oracle RAC is a key component of Oracle’s private cloud architecture.

Single Instance Vs RAC Instance

As can be seen from the below diagram Oracle RAC database has more than one instance, whereas the database is single and shared across all the instances. Thus focus of RAC is on share everything concept. This ensures availability of Oracle database and also the scalability. Each RAC instance is installed on different machine (called nodes) thus scaling the computing power and maximizing availability.



Below is the architecture diagram which shows the implementation of Oracle RAC instance in real life scenario. What it additionally shows is the interconnect (hb) i.e high speed communication between the RAC instances to ensure ACID (Atomicity, Consistency, Independent and Durability).



ASM (Automatic Storage Management) Instance:


An Oracle ASM instance is a special Oracle instance that manages Oracle ASM disks.

It’s an instance for Storage Management. Both the ASM and the database instances require shared access to the disks in an ASM disk group.

ASM instances manage the metadata of the disk group and provide file layout information to the database instances. Database instances direct I/O to ASM disks without going through an ASM instance.

An ASM instance is built on the same technology as a database instance. For example, an ASM instance has a system global area (SGA) and background processes that are similar to those of a database instance. However, an ASM instance cannot mount a database and performs fewer tasks than a database instance.

Oracle Commands for RAC & ASM

I have tried to highlight the commands which are used for different instances. As the system has grown with different components, so have the commands to manage and monitor them. Below table shows the list of commands, their usage, where used (RAC/ASM) and home directory where the command is located.

Commands Usage RAC ASM Home
cluvfy Cluster Verify Utility or cvu performs system checks in integrity of all nodes, integrity of OCR, connectivity between all nodes, compare properties of one node with that of other and space availability Y Y Grid Home and Oracle Home
olsnodes The olsnodes command provides the list of nodes, local node name, vip addresses of nodes, status of nodes and type of nodes in cluster. Y Oracle CRS Home, Run as root user
crsctl Starting and stopping Oracle Clusterware resources, Enabling and disabling Oracle Clusterware daemons, Checking the health of the cluster, Managing resources that represent third-party applications. Y Y* Voting disk Grid Home
srvctl Server Control Utility to manage the Oracle RAC database. SRVCTL commands to add, remove, start, stop, modify, enable, and disable a number of components/objects, such as databases, instances, listeners, SCAN listeners, services, grid naming service (GNS), and Oracle ASM.Some SRVCTL operations for instance, database, service and nodeapps modify the configuration data stored in the Oracle Cluster Registry (OCR).  Good link which shows various options for this command is here Y Y Oracle Home,Run as oracle user
asmcmd ASMCMD is a command-line utility that you can list the content, perform searches, create and remove directories and aliases, display space utilization within Automatic Storage Management (ASM) disk groups. Orafaq link for ASMCMD is here Y Preferably separate ASM home
ocrconfig The ocrconfig command is used to manage the OCR. Ocrconfig provides the ability to import, export, add, delete, restore, overwrite, backup, repair, replace, move, upgrade, or downgrade the OCR. Same command can be used for OLR using –local option. Y Y Grid Home,Run as root user
ocrdump The ocrdump utility allows you to dump the contents of the OCR or the OLR to a file or to stdout. You can then read the resulting output for diagnostic and administration purposes. Y Y Grid Home,Run as root user
ocrcheck The ocrcheck utility is used to validate the integrity of the OCR and the OLR. Use the ocrcheck command whenever you have made any changes to your cluster (such as changing the VIP for example) Y Y Grid Home,Run as root user
oifcfg The Oracle Interface Configuration Tool is used to define and administer network interfaces such as the public and private interfaces. Y Grid Home,Run as oracle User


I have tried to consolidate all processes which are available in Single Instance, RAC Instance and ASM Instance of Oracle 11gR2 Database.




Single RAC ASM
ACFS ASM Cluster File System CSS Process Y Y
ACMS Atomic Controlfile to Memory Service (ACMS). In an Oracle RAC environment Y
ARBn ASM Rebalance Process Y
ARBx Background Process Y Y
ARCH Archiver Process Y Y
ARCHMON Archive Monitor Y Y
ASMB Automatic Storage Management Y Y Y
BMRn Automatic Block Media Recovery Slave Pool Process Y
Bnnn ASM Blocking Slave Process for GMON Y
CJQ0 Job Queue Coordinator Process Y
CKPTxx Checkpoint Y Y
CTWR Change Tracking Writer Y Y
DBRM Database Resource Manager Process Y
DBWn Database Writer Y Y
DIAG Diagnostic Capture Process Y Y
DMnn Data Pump Master Process Y
Dnnn Dispatcher Y Y
DRnn ASM Disk Resynchronization Slave Process Y
EMNC EMON Coordinator Process Y Y
Ennn EMON Slave Process Y Y
EXTPROC Callout queues Y Y
FBDA Flashback Data Archiver Process Y Y
FMON File Mapping Monitor Process Y Y
GEN0 General Task Execution Process Y Y
GMON ASM Disk Group Monitor Process Y
GTX0-j Global Transaction Process. The GTX0-j process provides transparent support for XA global transactions in a RAC environment. The database autotunes the number of these processes based on the workload of XA global transactions. Y
Innn Disk and Tape I/O Slave Process Y
Jnnn Job Queue Slave Process Y
LCK0 Instance Enqueue Process. The LCK0 process manages non-Cache Fusion resource requests such as library and row cache requests. Y
LCKn Lock Process Y Y
LGWR Log Writer Y Y
LMD Global Enqueue Service Daemon. The LMD process manages incoming remote resource requests within each instance. Y
LMHB Global Cache/Enqueue Service Heartbeat Monitor Y Y Y
LMON Global Enqueue Service Monitor. The LMON process monitors global enqueues and resources across the cluster and performs global enqueue recovery operations. Y
LMS Global Cache Service Process. The LMS process maintains records of the datafile statuses and each cached block by recording information in a Global Resource Directory (GRD). Y
MARK Mark AU for Resynchronization Coordinator Process Y Y
MMAN Background Process Y Y
MMNL Memory Monitor Light Y Y
MMON Memory Monitor Y Y
Mnnn MMON Slave Process Y Y
ORBn Background Process Y Y
OSMB Background Process Y Y
PING Interconnect Latency Measurement Process Y Y Y
PMON Process Monitor Y Y
Pnnn Parallel Query Slave Y Y
PRnn Parallel Recovery Process Y
PSP0 Process Spawner Process Y Y
QMNxx Advanced Queueing Processes Y Y
RBAL Background Process Y Y
RCBG Result Cache Background Process Y Y
RECO Distributed Transaction Recoverer Y Y
RMSn Oracle RAC Management Processes (RMSn). Y
RSMN Remote Slave Monitor manages background slave process creation and communication on remote instances. Y
RVWR Recovery Writer Y Y
Rnnn ASM Block Remap Slave Process Y
SMON System Monitor Y Y
Snnn Servers Y Y
SNPxx Snapshot process queues Y Y
VKTM Virtual Keeper of Time Process Y Y
VMB0 Volume Membership Process Y
Vnnn ASM Volume I/O Slave Process Y
Xnnn ASM Disk Expel Slave Process Y


Workshop also touched Data Guard theory and Performance Tuning in RAC.

I have tried to share what I learnt in a very simple and concise manner. I hope you find this article useful and If you feel that something could have been done better kindly drop me mail at

Vijay Mahawar

Visit Vijay’s Oracle photo gallery here and know more about him here


Posted by on 2 March, 2013 in DBA, Oracle


Tags: , , , , , , , , , , ,

4 responses to “Workshop on Oracle 11gR2 DBA: RAC and ASM

  1. Abul

    13 March, 2013 at 5:17 pm

    Nice post..thanks vijay

    • Viju

      30 March, 2013 at 2:57 pm

      Thanks for the appreciation Abul 🙂

  2. naresh

    20 April, 2013 at 12:30 pm

    can you explain how select statement works in 4 node rac
    how update stament works in 4 node rac……..

    • Viju

      20 April, 2013 at 1:28 pm

      Hi Naresh,

      SELECT and UPDATE statement are processed in n-node RAC setup with the help of LMS (Global Cache Service Process) which maintains the GRD (Global Resource Directory) and a concept called CACHE Fusion.
      As you must be aware that in RAC environment each node has its own buffer cache and memory structure.
      Whenever an instance request a block either for READ (in case of SELECT) or WRITE (in case of UPDATE), the LMS checks the GRD for its availability, which keeps the details of the block, its current instance which controls it, mode in which its being held. There might be three scenarios in this case:

      Now requesting instance can get the block requested in two ways:
      1. Either from other instance using Cache fusion (Logical IO through Interconnect) depending on availability by checking GRD.
      2. Directly Read the block from the disk.

      Note both Select and Update follow the same steps except that Update would require the exclusive lock on the requested block and SELECT statement would require only the consistent read.

      In short following steps takes place, which I got from following source.

      * Process tries to read a data block
      * Finds an active transaction in the block
      * Then checks the UNDO segment to see if the transaction has been committed or not
      ** If the transaction has been committed, it creates the REDO records and reads the block
      ** If the transaction has not been committed, it creates a CR block for itself using the UNDO/ROLLBACK information.
      * Creating a CR image in RAC is a bit different and can come with some I/O overheads. This is because the UNDO could be spread across instances and hence to build a CR copy of the block, the instance might has to visit UNDO segments on other instances and hence perform certain extra I/O

      I hope this answers your query


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: