Reboot Exadata Machine
- Log in to the first Exadata compute node as as root user.
- Change to root home directory and create the following files
- Check if Grid Infrastructure is enabled for autostart:
- Disable the Grid Infrastructure for autostart on the compute nodes if it is currently enabled for autostart.
- Shutdown the Grid Infrastructure stack on the Exadata compute nodes
- Ensure that the Grid Infrastructure stack has shutdown successfully on all the Compute nodes.
- Disable email/SNMP alerts. The following command will show the current notification method:
- Now set the notification method to null to disable alerts:
- Shutdown cell services on all the cells:
- Power off all storage cells:
- Copy the /root/dbs_group to /root/dbs_group1:
- Remove the "first compute node name" from the /root/dbs_group1 file (using any editor, e.g. vi).
- Power off all the compute nodes (except compute node # 1):
- Power off the compute node #1:
- Exadata components are shutdown for maintenance.
- Software components:
- Operating system
- Exadata Storage server software
- Firmware
- Hardware components:
- Motherboard
- Hard Disk
- Flash Disk
- Processor
- Physical memory
- Power supply
- Fan
- Batteries and so on
- External Redundancy : ASM doesn't provide redundancy
- Normal Redundancy : Provides two-way data mirroring. Maintains two Copies of data blocks in separate failure groups
- High Redundancy : Provides three-way data mirroring. Maintains three Copies of data blocks in separate failure groups
- root user access
- This procedure does not require Database and ASM outage.
- Ensure that there no failed disks in the storage cells.
- Check DISK_REPAIR_TIME value for all the ASM disk group mounted
- If ASM disks to be offline for more than the default time of 3.6 hours then you can change the DISK_REPAIR_TIME parameter value by issuing the command:
- Ensure that ASM will be fine if the grid disks go OFFLINE. The output of the following command should return 'yes' for all the grid disks listed.
- Run the following command to Inactivate all grid disks on the cell you wish to power shutdown or reboot:
- Execute the following command to ensure that the griddisks are offline. The output should show either asmmodestatus=OFFLINE or asmmodestatus=UNUSED and asmdeactivationoutcome=Yes for all griddisks once the disks are offline in ASM.
- Confirm all the grid disks now show inactive
- You can now safely shutdown or reboot the cell.
- Verify Storage Cell came up successfully.
- Verify that Cell Services came up fine.
- Reactive the grid disks after the Storage cell comes back online.
- Ensure taht all disks show 'active', Execute the following command
- Verify all grid disks are online using the following command:
- Each disk goes to 'SYNCING' state first and then goes to 'ONLINE'. You must wait until asmmodestatus is ONLINE for all grid disks.
- Verify Disks are added back to ASM and are ONLINE
- All database and Oracle
Clusterware processes should be shut down prior to
shutting down more than one Exadata Storage Server. - Powering off one Exadata
Storage Server does not affect running database
processes or Oracle ASM. - Powering off or restarting
Exadata Storage Servers can impact database
availability. - The shutdown commands can be used to power off or restart database servers.
1) Log in to the
first database server as root.
2) Change to the
OneCommand directory
#
cd /opt/oracle.SupportTools/onecommand
3) Note whether
the Grid Infrastructure is currently enabled for autostart, so that this state
can be restored later:
#
dcli -g dbs_group -l root /u01/app/11.2.0/grid/bin/crsctl config crs
4) Disable the
Grid Infrastructure for autostart on the database servers if the previous step
indicated it is currently enabled for autostart.
#
dcli -g dbs_group -l root /u01/app/11.2.0/grid/bin/crsctl disable crs
Note: This
is step is [Optional] and it can required during maintenance operation like
“firmware patches” which requires to reboot the Compute Node several times.
5) Stop the Grid
Infrastructure stack on the database servers (compute nodes):
#
dcli -g dbs_group -l root /u01/app/11.2.0/grid/bin/crsctl stop crs
6) Verify that
the Grid Infrastructure stack has shutdown successfully on the database
servers. The following command should show no output if the GI stack has
shutdown:
#
dcli -g dbs_group -l root “ps -ef | grep diskmo[n]”
7) [Optional]
Disable email/SNMP alerts if applicable. If alerts will be enabled, first note
the notification method in use so that it can be restored later. The following
command will show the current notification method:
# dcli -g cell_group -l root “su – celladmin -c \”cellcli
-e list cell attributes name,notificationMethod\””
Now set the
notification method to null to disable alerts:
# dcli -g cell_group -l root “su – celladmin -c \”cellcli
-e alter cell notificationMethod=null\””
8) Shutdown all
the cells:
# dcli -g cell_group -l root “su – celladmin -c \”cellcli
-e alter cell shutdown services all \””
9) Power off all
cells:
#
dcli -g cell_group -l root poweroff
10) Copy the
/opt/oracle.SupportTools/onecommand/dbs_group to
/opt/oracle.SupportTools/onecommand/dbs_group2:
#
cp /opt/oracle.SupportTools/onecommand/dbs_group
/opt/oracle.SupportTools/onecommand/dbs_group2
11) Remove the
“first compute node name” from the
/opt/oracle.SupportTools/onecommand/dbs_group2 file (using any editor, e.g.
vi).
12) Power off
all the compute nodes (except compute node # 1):
#
dcli -g dbs_group2 -l root poweroff
13) Power off
the compute node #1:
#
poweroff
14) Now, both
Cell Servers & Compute nodes are down for maintenance.
15) Power
“compute node #1” on, by using the power button on the front panel of the
Exadata Storage Servers.
16) Power the
cells on, either by using the power button on the front panel of the Exadata
Storage Servers, or by executing the following script from the first database
server (from Compute node #1):
for
host in `cat cell_group`; do
echo
${host}: `ipmitool -H ${host}-ilom -U root -P welcome1 chassis power on`
done
17) Wait until
all the Cell servers are up, then power the compute nodes on, either by using
the power button on the front panel of the Exadata Storage Servers, or by
executing the following script from the first database server:
for
host in `cat dbs_group2`; do
echo
${host}: `ipmitool -H ${host}-ilom -U root -P welcome1 chassis power on`
done
18) Verify that
all Exadata Storage Servers have booted successfully:
#
dcli -g cell_group -l root hostname
19) Verify all
the cells are up (MS, RS & CELLSRV services must be running):
#
dcli -g cell_group -l root “su – celladmin -c \”cellcli -e list cell detail
\””
20) Reenable
email/SNMP alerts if they were disabled during the pre-maintenance steps. The
example below illustrates the command to use if both email and SMTP alerting
used, but adjust the command as necessary to restore the notification method
noted during the pre-maintenance steps.
# dcli -g cell_group -l root “su – celladmin -c \”cellcli
-e alter cell notificationMethod=\’mail,snmp\’\””
21) Start the
Grid Infrastructure stack on the first database server
#
/u01/app/11.2.0/grid/bin/crsctl start crs
22)
Wait until the Grid Infrastructure stack has started successfully on the first
database server. To check the status of the Grid Infrastructure stack, run the
following command and verify that the “ora.asm” instance
is started. Note that the command below will continue to report that it is
unable to communicate with the Grid Infrastructure software for several
minutes after issuing the “crsctl start crs” command above:
#
/u01/app/11.2.0/grid/bin/crsctl status resource -t
23) Start the
Grid Infrastructure stack on all remaining database servers with a 1-minute
delay between servers. Note that the command below will issue the start
command for all database servers, including the first database server where it
is already started. The output for the first server will therefore indicate
that the software is already started, but this message can be safely ignored:
for
h in `cat dbs_group`; do
echo
“Starting CRS on host $h”
ssh
$h /u01/app/11.2.0/grid/bin/crsctl start crs
echo
“Waiting 60 seconds…”
sleep
60
done
24) Monitor the
startup progress (this could take several minutes):
#
/u01/app/11.2.0/grid/bin/crsctl status resource -t
25) Reenable the
Grid Infrastructure for autostart if it was found to be enabled for autostart
at the beginning of the pre-maintenance steps.
#
dcli -g dbs_group -l root /u01/app/11.2.0/grid/bin/crsctl enable crs
Note: If dcli/ssh is not
enabled due to security reasons, then you will need to connect and execute the
above steps on each node at the time.
======================================================================================
As root perform the following on
each database server - one at a time:
1)Check to see if AUTOSTART is
enabled:
# $GRID_HOME/bin/crsctl config crs
2) Disable the Grid Infrastructure
for autostart on the database servers if the previous step indicated it is
currently enabled for autostart.
# $GRID_HOME/bin/crsctl disable
crs
3) Stop the Grid Infrastructure
stack on the first database server locally:
# $GRID_HOME/bin/crsctl stop crs
4) Verify that the Grid
Infrastructure stack has shutdown successfully on the database server.
The following command should show
no output if the GI stack has shutdown:
# ps -ef | grep diskmon
5) Confirm that the clusterware
resources are still up and running on the other nodes:
cd
/opt/oracle.SupportTools/onecommand
# dcli -g dbs_group -l root $GRID_HOME/bin/crsctl check crs
# dcli -g dbs_group -l root $GRID_HOME/bin/crsctl check crs
7) Before shutting down the DB
node, check /etc/fstab for any nfs mounts that should be unmounted.
8) Shut the db node down so you
can peform maintenance.
# poweroff
====================================================================================
Steps to Shutdown Exadata Database Machine
Overview
When
you perform power maintenance, data center maintenance or moving servers from
one data center to another you need to shutdown the
Clusterware/Database/Servers. If you have Exadata Database Machines sitting in
your data center then you need to shutdown it as well. The process of
shutting down Exadata Database machine is slightly different when
compared to servers or cluster. We should follow a sequence of steps to
shutdown Exadata components.
In
this article, we will demonstrate the steps to shutdown Exadata Database
Machine to carry out maintenance task.
Environment
Exadata
X5-2 Full Rack
[root@dm01db01
~]# cd /root
[root@dm01db01
~]# vi dbs_group
dm01db01
dm01db02
dm01db03
dm01db04
dm01db05
dm01db06
dm01db07
dm01db08
[root@dm01db01
~]# vi cell_group
dm01cel01
dm01cel02
dm01cel03
dm01cel04
dm01cel05
dm01cel06
dm01cel07
dm01cel08
dm01cel09
dm01cel10
dm01cel11
dm01cel12
dm01cel13
dm01cel14
[root@dm01db01
~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl config crs
dm01db01:
CRS-4622: Oracle High Availability Services autostart is enabled.
dm01db02:
CRS-4622: Oracle High Availability Services autostart is enabled.
dm01db03:
CRS-4622: Oracle High Availability Services autostart is enabled.
dm01db04:
CRS-4622: Oracle High Availability Services autostart is enabled.
dm01db05:
CRS-4622: Oracle High Availability Services autostart is enabled.
dm01db06:
CRS-4622: Oracle High Availability Services autostart is enabled.
dm01db07:
CRS-4622: Oracle High Availability Services autostart is enabled.
dm01db08:
CRS-4622: Oracle High Availability Services autostart is enabled.
[root@dm01db01
~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl disable crs
dm01db01:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db02:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db03:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db04:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db05:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db06:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db07:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db08:
CRS-4621: Oracle High Availability Services autostart is disabled.
[root@dm01db01
~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl config crs
dm01db01:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db02:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db03:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db04:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db05:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db06:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db07:
CRS-4621: Oracle High Availability Services autostart is disabled.
dm01db08:
CRS-4621: Oracle High Availability Services autostart is disabled.
This
will take some time to come out
[root@dm01db01
~]#dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl stop crs
[root@dm01db01
~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl check cluster
dm01db01:
CRS-4639: Could not contact Oracle High Availability Services
dm01db01:
CRS-4000: Command Check failed, or completed with errors.
dm01db02:
CRS-4639: Could not contact Oracle High Availability Services
dm01db02:
CRS-4000: Command Check failed, or completed with errors.
dm01db03:
CRS-4639: Could not contact Oracle High Availability Services
dm01db03:
CRS-4000: Command Check failed, or completed with errors.
dm01db04:
CRS-4639: Could not contact Oracle High Availability Services
dm01db04:
CRS-4000: Command Check failed, or completed with errors.
dm01db05:
CRS-4639: Could not contact Oracle High Availability Services
dm01db05:
CRS-4000: Command Check failed, or completed with errors.
dm01db06:
CRS-4639: Could not contact Oracle High Availability Services
dm01db06:
CRS-4000: Command Check failed, or completed with errors.
dm01db07:
CRS-4639: Could not contact Oracle High Availability Services
dm01db07:
CRS-4000: Command Check failed, or completed with errors.
dm01db08:
CRS-4639: Could not contact Oracle High Availability Services
dm01db08:
CRS-4000: Command Check failed, or completed with errors.
The
following command returns no output if the Grid Infrastructure is shutdown:
[root@dm01db01
~]# dcli -g dbs_group -l root "ps -ef | grep diskmo[n]"
[root@dm01db01
~]# dcli -g cell_group -l root "cellcli -e list cell attributes
name,notificationMethod"
dm01cel01:
dm01cel01 none
dm01cel02:
dm01cel02 none
dm01cel03:
dm01cel03 none
dm01cel04:
dm01cel04 none
dm01cel05:
dm01cel05 none
dm01cel06:
dm01cel06 none
dm01cel07:
dm01cel07 none
dm01cel08:
dm01cel08 none
dm01cel09:
dm01cel09 none
dm01cel10:
dm01cel10 none
dm01cel11:
dm01cel11 none
dm01cel12:
dm01cel12 none
dm01cel13:
dm01cel13 none
dm01cel14:
dm01cel14 none
[root@dm01db01
~]# dcli -g cell_group -l root "cellcli -e alter cell
notificationMethod=none"
dm01cel01:
Cell dm01cel01 successfully altered
dm01cel02:
Cell dm01cel02 successfully altered
dm01cel03:
Cell dm01cel03 successfully altered
dm01cel04:
Cell dm01cel04 successfully altered
dm01cel05:
Cell dm01cel05 successfully altered
dm01cel06:
Cell dm01cel06 successfully altered
dm01cel07:
Cell dm01cel07 successfully altered
dm01cel08:
Cell dm01cel08 successfully altered
dm01cel09:
Cell dm01cel09 successfully altered
dm01cel10:
Cell dm01cel10 successfully altered
dm01cel11:
Cell dm01cel11 successfully altered
dm01cel12:
Cell dm01cel12 successfully altered
dm01cel13:
Cell dm01cel13 successfully altered
dm01cel14:
Cell dm01cel14 successfully altered
[root@dm01db01
~]# dcli -g cell_group -l root "cellcli -e alter cell shutdown services
all"
dm01cel01:
dm01cel01:
Stopping the RS, CELLSRV, and MS services...
dm01cel01:
The SHUTDOWN of services was successful.
dm01cel02:
dm01cel02:
Stopping the RS, CELLSRV, and MS services...
dm01cel02:
The SHUTDOWN of services was successful.
dm01cel03:
dm01cel03:
Stopping the RS, CELLSRV, and MS services...
dm01cel03:
The SHUTDOWN of services was successful.
dm01cel04:
dm01cel04:
Stopping the RS, CELLSRV, and MS services...
dm01cel04:
The SHUTDOWN of services was successful.
dm01cel05:
dm01cel05:
Stopping the RS, CELLSRV, and MS services...
dm01cel05:
The SHUTDOWN of services was successful.
dm01cel06:
dm01cel06:
Stopping the RS, CELLSRV, and MS services...
dm01cel06:
The SHUTDOWN of services was successful.
dm01cel07:
dm01cel07:
Stopping the RS, CELLSRV, and MS services...
dm01cel07:
The SHUTDOWN of services was successful.
dm01cel08:
dm01cel08:
Stopping the RS, CELLSRV, and MS services...
dm01cel08:
The SHUTDOWN of services was successful.
dm01cel09:
dm01cel09:
Stopping the RS, CELLSRV, and MS services...
dm01cel09:
The SHUTDOWN of services was successful.
dm01cel10:
dm01cel10:
Stopping the RS, CELLSRV, and MS services...
dm01cel10:
The SHUTDOWN of services was successful.
dm01cel11:
dm01cel11:
Stopping the RS, CELLSRV, and MS services...
dm01cel11:
The SHUTDOWN of services was successful.
dm01cel12:
dm01cel12:
Stopping the RS, CELLSRV, and MS services...
dm01cel12:
The SHUTDOWN of services was successful.
dm01cel13:
dm01cel13:
Stopping the RS, CELLSRV, and MS services...
dm01cel13:
The SHUTDOWN of services was successful.
dm01cel14:
dm01cel14:
Stopping the RS, CELLSRV, and MS services...
dm01cel14:
The SHUTDOWN of services was successful.
[root@dm01db01
~]#
[root@dm01db01
~]# dcli -g cell_group -l root poweroff
or
[root@dm01db01
~]# dcli -g cell_group -l root 'shutdown -h now'
[root@dm01db01
~]# cp /root/dbs_group /root/dbs_group1
[root@dm01db01
~]# vi /root/dbs_group1
[root@dm01db01
~]# cat /root/dbs_group1
dm01db02
dm01db03
dm01db04
dm01db05
dm01db06
dm01db07
dm01db08
[root@dm01db01
~]# dcli -g dbs_group1 -l root poweroff
or
[root@dm01db01
~]# dcli -g dbs_group1 -l root 'shutdown -h now'
[root@dm01db01
~]# poweroff
Broadcast
message from root@dm01db01.netsoftmate.com
(/dev/pts/0) at 7:36 ...
The
system is going down for power off NOW!
Conclusion
In
this article we have learnt how to Shutdown Exadata Database machine Rack for
maintenance.
==================================================================================================
Exadata - Shutdown or Reboot Exadata Storage Cell
Without Affecting ASM
Introduction
Exadata
Storage cells consists of hardware and software components.
Exadata
Storage Server Software is responsible for satisfying database I/O requests
and implementing unique performance features, including Smart Scan, Smart
Flash Cache, Smart Flash Logging, Storage Indexes, IO Resource Management, and
Hybrid Columnar Compression.
The
combination of faster, balanced and high available hardware with unique
Exadata software allows Exadata to deliver extreme performance.
When
performing maintenance such as hardware replacement or patching on Exadata
Storage cells it may be necessary to shutdown or reboot Storage cell.
Care
must be taken when shutting down or rebooting storage cell when databases are
running. If you have databases running and you wish to shutdown the storage
cell, ensure that shutting down storage cell will not impact ASM disk group
and databases running. This depends on the ASM redundancy used for the ASM
disk group and hard disk status in the other storage cells that have the
mirror data copies.
In
this article, I will demonstrate how you can shutdown or reboot the Exadata
Storage cells without Affecting ASM.
Environment
Details
|
Exadata Model
|
X2 Full Rack HC
2TB
|
|
Exadata
Components
|
Storage Cell
(14), Compute node (8) & Infiniband Switch (2)
|
|
Exadata Storage
cells
|
DBM01CEL01
– DBM01CEL14
|
|
Exadata Compute
nodes
|
DBM01DB01
– DBM01DB08
|
|
Exadata Software
Version
|
12.1.2.2.0
|
|
Exadata DB
Version
|
12.2.0.2 BP2
|
Procedure
to Shutdown or Reboot Exadata Storage Cell without Affecting ASM
Exadata ASM Disk Group
A
Disk Group is a storage volume consists of multiple physical disks. In
Exadata, A Disk Group is created using Grid Disks, Grid Disks are created
within Cell Disks and Cell Disks represent physical disks. You can create a
single Grid Disk per Cell Disk, most typical configurations have multiple Grid
Disks per Cell Disk.
There
are three types of redundancy in ASM:
Oracle
recommends using high Redundancy for all disk groups as it provides maximum
availability against storage failures and operational simplicity during a
storage outage. If all disk groups are configured with normal redundancy and
if two partner disks fail, the entire cluster and database will fail resulting
in loss of data.
Prerequisites
Steps
Login
to the Exadata Compute node as the Grid Infrastructure software owner and
connect to ASM Instance.
The
default value for disk_repair_time is 3.6 hours. ASM drops a disk shortly
after it is taken offline. The default value of 3.6h should be sufficient in
most of the environment.
Note: If one or more disks does not return asmdeactivationoutcome='Yes', check the ASM diskgroup and restore the redundancy for that ASM
diskgroup. Once the ASM diskgroup redundancy is restored, repeat the previous
command. Once all disks return asmdeactivationoutcome='Yes', you can proceed
with the next step.
NOTE: OFFLINE grid disks can take longer based on the Exadata hardware in
use and depending on activity. On Exadata V2 it took 2 hours inactive the grid
disk and as low as 10-15 minutes on X5. Ensure that all the grid disks are
offline before shutting down the cell services. Inactivating the grid disks
will automatically OFFLINE the disks in the ASM instance.
To
shutdown Exadata Storage cell immediately, excute the following command as
root
To
reboot Exadata Storage cell immediately and force fsck on reboot, excute the
following command as root
#shutdown -F -r now
[root@dm01cel01
~]# service celld status
rsStatus:
running
msStatus:
running
cellsrvStatus: running
Oracle
ASM synchronization is only complete when all grid disks show
asmmodestatus=ONLINE.
Note:
Before taking another storage server offline, Oracle ASM synchronization must
complete on the Storage Cell restarted. If synchronization is NOT completed,
then the check performed on another storage server will fail.
Here
is the sample output:
CellCLI>
list griddisk attributes name where asmdeactivationoutcome != 'Yes'
DATA_CD_00_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_01_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_02_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_03_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_04_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_05_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_06_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_07_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_08_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_09_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_10_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
DATA_CD_11_dm01cel02
"Cannot de-activate due to other offline disks in the diskgroup"
Conclusion
In
this article we have learned different hardware and software components in
Exadata Storage cell and how to shutdown or reboot Exadata Storage cell
without Affecting ASM. The ability to shutdown or reboot the storage cell
without affecting database availability depends on the ASM redundancy and disk
status in other storage cells that have mirror copies of data.
==============================================================================================
Shutting Down a
Exadata DB Node [same on normal RAC]
While we need to shutdown any of exadata(same for any
RAC DB) DB Nodes I normally follow below checklist.
1. from DB & ASM node run below query and cahek
all the nodes are in open staus
select * from gv$instance;
2. login to the target DB node with oracle user and
set .oracrs.sh env
3. now shutdown the instance on the target node using
below command
/u01/app/11.2.0/grid/bin/srvctl stop
instance -d ORADB -i
4. FROM DB node run belwo quert to be sure that
target instance is now down
select * from gv$instance;
5. now login as root and run below command to shudown
CRS
/u01/app/11.2.0/grid/bin/crsctl stop crs
-- from DB node run below command and make sure that
crs is down [CRS-0184]
/u01/app/11.2.0/grid/bin/crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.
6. FROM ASM run below query to make sure that ASM on
tgaret hots is down
select * from gv$instance;
7. shutdown the DB machine
8. startup the DB machine
9. check with below command the all the cluster
services are online except .gsd and .oc4j
/u01/app/11.2.0/grid/bin/crs_stat -t|grep OFFLINE
--from DB node with oracle user and .oracrs.sh env
run below command to make sure that all the ASM disks are online
asmcmd
ASMCMD> ls
. oracrs.sh
sqlplus / as sysasm
10. if any disk is missing the from ASM instance of
the target machine run below command to make the perticular disk online
alter diskgroup mount;
11. chek that DB instance is online or not if not
automatically online use below command to make it online
srvctl start -d axdw -i
12. from ASM & DB instance user below query to
check all the ASM and DB instances are online
select * from gv$instance;
13. from ASM & DB instance user below query to
check all the ASM DISKS are online from all the instance
select
header_status,mode_status,state,failgroup,count(*) from gv$asm_disk group by
header_status,mode_status,state,failgroup;
14. check alertlog of both db & asm for the
target node
====================================================================================================
The power on
sequence is as follows:
1. Rack, including
switches.
2. Exadata Storage
Servers.
3. Database
servers.
Powering Off Oracle
Exadata Rack
The power off
sequence for Oracle Exadata Rack is as follows:
1. Database servers
(Oracle Exadata Database Machine only).
2. Exadata Storage
Servers.
3. Rack, including
switches.
Powering Off One
Database Servers
The following
procedure is the recommended shutdown procedure for database
servers:
1. Stop Oracle
Clusterware using the following command:
#
GRID_HOME/grid/bin/crsctl stop cluster
If any resources
managed by Oracle Clusterware are still running after issuing the
crsctl stop cluster
command, then the command fails. Use the -f option to
unconditionally
stop all resources, and stop Oracle Clusterware.
2. Shut down the
operating system using the following command:
# shutdown -h -y
now
Powering Off
One Exadata Storage Server
For Restart Kindly visit Steps to Restart
Exadata Storage Server
Shutdown
immediately:
# shutdown -h -y
now
Restart
immediately:
# shutdown -r -y
now
Note the following
when powering off Exadata Storage Servers:
■ All database
and Oracle Clusterware processes should be shut down prior to
shutting down more
than one Exadata Storage Server.
■ Powering off
one Exadata Storage Server does not affect running database
processes or Oracle
ASM.
■ Powering off
or restarting Exadata Storage Servers can impact database
availability.
■ The shutdown
commands can be used to power off or restart database servers.
Powering Off
Multiple Servers at the Same Time
# dcli -l root -g
cell_group shutdown -h -y now
Do not run the dcli utility from a server that will be shut
down. ;)
Powering Off Oracle
Exadata Rack
1. Stop Oracle
Clusterware on all database servers
#
GRID_HOME/grid/bin/crsctl stop cluster -all
2. Shut down all
remote database servers
# dcli -l root -g
remote_dbs_group shutdown -h -y now
remote_dbs_group is the file that contains a list of all database
servers.
except the server you are using to shutdown the remote servers
(step 4)
3. Shut down all
Exadata Storage Servers
# dcli -l root -g
cell_group shutdown -h -y now
cell_group is the file that contains a list of all Exadata
Storage Servers.
4. Shut down the
local database server using the following command:
shutdown -h -y now
5. Remove power
from the rack.
============================================================================================
How to Power On/off Oracle Exadata Machine
How to Power On/off Oracle Exadata Machine
Oracle
Exadata machines can be powered on/off either by pressing the power button on
front of the server or or by logging in to the ILOM interface.
Powering on servers using button on front of
the server
The
power on sequence is as follows.
1. Start
Rack, including switches
Note>
Ensure the switches have had power applied for a few minutes to complete
poweron
configuration
before starting Exadata Storage Servers
2.Start
Exadata Storage Servers
Note>
Ensure all Exadata Storage Servers complete the boot process before starting
the
database
servers
3. Start
Database Servers
Powering On Servers Remotely using ILOM
The ILOM
can be accessed using the Web console, the command-line interface (CLI),
IPMI, or
SNMP. For example, to apply power to server dm01cel01 using IPMI, where
dm01cel01-ilom
is the host name of the ILOM for the server to be powered on, run the
following
command from a server that has IPMItool installed
#ipmitool -I
lanplus -H dm01cel01-ilom -U root chassis power on
You can
access the Web console as follows
where
exl1cn01-ilom is the host name of ILOM you can also use IP address instead of
hostname. Login at ILOM using root user
Powering Off Oracle Exadata Rack
Follow
the sequence below to power off Exadata Rack
1.
Database servers (Oracle Exadata Database Machine only).
2.
Exadata Storage Servers.
3. Rack,
including switches.
Powering Off Database Servers
to power
off database servers you need to first stop the cluterware. Use below command
as root user to stop the clusterware. Issue the command from the server you
want to stop the clusterware on
cd
$GRID_HOME/bin
#crsctl stop
cluster
Shut
down the operating system
# shutdown
-h now
Powering Off Exadata Storage Servers
You can
use the following commands to shutdown and restart the Exadata Storage Servers
shutdown
# shutdown
-h now
restart
# shutdown
-r now
Some
Points worth to note
Powering Off Multiple Servers at the Same Time
use dcli
utility for this nobel cause. but but but dont run run dcli command from the
server which will go down
# dcli -l
root -g group_name shutdown -h now
group_name
is the file that contains a list of all Exadata Storage Servers, cell_group,
or database servers, dbs_group
following
command will shudown all the cell server together listed in cell_group
group file
# dcli -l
root -g cell_group shutdown -h now
Comments
Post a Comment