RHEL7+Oracle11.2 RAC集群-多路径（multipath+udev）安装步骤-EW帮帮网

微信交流群：https://732903.hlcode.cn/?id=NK1fWUR

RHEL7+Oracle11.2RAC集群-多路径（multipath+udev）安装

配置虚拟存储

使用StarWind Management Console软件，配置存储

dggrid1: 1g*3

Dggrid2: 1g*3

Dgsystem: 5g*1 系统表空间，临时表空间，UNDO，参数文件，控制文件，重做日志文件

Dgrecovery:2g=2G*1 归档日志，一周归档量

Dgdata01: 4g=2G*2 生产数据空间

环境说明：
#public ip
172.30.21.171 hferp71
172.30.21.172 hferp72
#priv ip
10.10.10.71 hferp71prv
10.10.10.72 fgerp72prv
#vip ip
172.30.21.173 hferp71vip
172.30.21.174 hferp72vip
#scan ip
172.30.21.175 hferpdbscan


db name：hferpdb
instance name：hferpdb1、hferpdb2

配置iscsi连接存储

配置虚拟机参数

disk.EnableUUID = “TRUE”

cat /etc/inittab 设置不使用图形化界面

systemctl set-default multi-user.target

挂载光驱

mount /dev/cdrom /mnt

安装iscsi软件

rpm -ivh /mnt/Packages/iscsi-initiator-utils-6.2.0.874-10.el7.x86_64.rpm

cat /etc/iscsi/initiatorname.iscsi

echo “options=–whitelisted --replace-whitespace” > /etc/scsi_id.config

启动iscsi–开机启动

systemctl enable iscsi

systemctl start iscsi

iscsiadm -m discovery -t st -p 172.30.21.176

iscsiadm -m discovery -t st -p 172.30.21.177

iscsiadm -m discovery -t st -p 172.30.21.178

iscsiadm -m discovery -t st -p 172.30.21.154

iscsiadm -m node -T iqn.2008-08.com.starwindsoftware:172.30.21.154-hferpdb -p 172.30.21.176:3260 -l

iscsiadm -m node -T iqn.2008-08.com.starwindsoftware:172.30.21.154-hferpdb -p 172.30.21.177:3260 -l

iscsiadm -m node -T iqn.2008-08.com.starwindsoftware:172.30.21.154-hferpdb -p 172.30.21.178:3260 -l

iscsiadm -m node -T iqn.2008-08.com.starwindsoftware:172.30.21.154-hferpdb -p 172.30.21.154:3260 -l

检查磁盘：

[root@hferp71 ~]# fdisk -l |grep "Disk /dev"
Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Disk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sde: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdf: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdg: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdi: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdj: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdk: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdh: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdl: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdo: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdp: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdr: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdq: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdm: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdn: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sds: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdu: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdw: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdx: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdt: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdz: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdaa: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdy: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdab: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdae: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdv: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdac: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdag: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdai: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdad: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdaf: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdaj: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdak: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdam: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdal: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdah: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdan: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdao: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdap: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sdaq: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdar: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdat: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdas: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdau: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdav: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdax: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sday: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdaw: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdba: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/asm-dggrid2: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdaz: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdbb: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/asm-dggrid1: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/mapper/asm-dggrid3: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/mapper/asm-dggrid5: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/mapper/asm-dggrid4: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/mapper/asm-dggrid6: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/mapper/asm-dgrecovery01: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/mapper/asm-dgsystem01: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/mapper/asm-data01: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/mapper/asm-data02: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/mapper/fs-backup02: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/fs-backup01: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/fs-backup03: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/backupvg-backuplv: 9437 MB, 9437184000 bytes, 18432000 sectors

重启服务器 reboot

检查配置：fdisk -l |grep “Disk /dev”

配置多路径及ASM磁盘

多路径软件安装：

rpm -qa |grep multipath

配置生效，多路径开机启动

modprobe dm-multipath

modprobe dm-round-robin

lsmod |grep dm_multipath

systemctl enable multipathd

systemctl start multipathd

multipath -v2

报错：/etc/multipath.conf does not exist, blacklisting all devices

cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf /etc/multipath.conf

multipath -ll |grep mpath

for i in `cat /proc/partitions | awk '{print$4}' |grep sd | grep [a-z]$`; do echo "### $i: `/lib/udev/scsi_id --whitelisted --device=/dev/$i`"; done

[root@hferp71 ~]# for i in `cat /proc/partitions | awk '{print$4}' |grep sd | grep [a-z]$`; do echo "### $i: `/lib/udev/scsi_id --whitelisted --device=/dev/$i`"; done
### sda: 36000c29b5effc0b462f8133744caf8c4
### sdb: 36000c297b9b69ad821ba52dd7a8103a1
### sdc: 269e599b56d920cd3
### sdd: 2c8dacd7143254b70
### sde: 266fc7bc69e23e94a
### sdf: 24b93ac5567327f2f
### sdg: 2fe5799e30982c881
### sdi: 269e599b56d920cd3
### sdj: 2c8dacd7143254b70
### sdk: 266fc7bc69e23e94a
### sdh: 205e8a87f943a383e
### sdl: 24b93ac5567327f2f
### sdo: 269e599b56d920cd3
### sdp: 2c8dacd7143254b70
### sdr: 269e599b56d920cd3
### sdq: 266fc7bc69e23e94a
### sdm: 259e04342c730046d
### sdn: 2fe5799e30982c881
### sds: 2c8dacd7143254b70
### sdu: 24b93ac5567327f2f
### sdw: 2fe5799e30982c881
### sdx: 266fc7bc69e23e94a
### sdt: 21f356012394a74f0
### sdz: 24b93ac5567327f2f
### sdaa: 259e04342c730046d
### sdy: 205e8a87f943a383e
### sdab: 21241b7334e5274d2
### sdae: 21f356012394a74f0
### sdv: 205e8a87f943a383e
### sdac: 2fe5799e30982c881
### sdag: 259e04342c730046d
### sdai: 2d0bc8bb5f18458cf
### sdad: 2d0bc8bb5f18458cf
### sdaf: 21241b7334e5274d2
### sdaj: 21f356012394a74f0
### sdak: 21241b7334e5274d2
### sdam: 2d0bc8bb5f18458cf
### sdal: 259e04342c730046d
### sdah: 205e8a87f943a383e
### sdan: 21f356012394a74f0
### sdao: 21241b7334e5274d2
### sdap: 2d0bc8bb5f18458cf
### sdaq: 2a6f8ecb92c2c7506
### sdar: 233bef421cb87759d
### sdat: 2a6f8ecb92c2c7506
### sdas: 23b4a22bf4954f0c6
### sdau: 233bef421cb87759d
### sdav: 2a6f8ecb92c2c7506
### sdax: 2a6f8ecb92c2c7506
### sday: 233bef421cb87759d
### sdaw: 23b4a22bf4954f0c6
### sdba: 23b4a22bf4954f0c6
### sdaz: 233bef421cb87759d
### sdbb: 23b4a22bf4954f0c6

[root@hferp72 ~]# for i in `cat /proc/partitions | awk '{print$4}' |grep sd | grep [a-z]$`; do echo "### $i: `/lib/udev/scsi_id --whitelisted --device=/dev/$i`"; done
### sda: 36000c29583d9ab34838ce7207521dd40
### sdb: 36000c29e988fcd7db059225b8e7c2813
### sdc: 269e599b56d920cd3
### sdd: 2c8dacd7143254b70
### sde: 266fc7bc69e23e94a
### sdf: 269e599b56d920cd3
### sdg: 2c8dacd7143254b70
### sdi: 24b93ac5567327f2f
### sdj: 24b93ac5567327f2f
### sdk: 2fe5799e30982c881
### sdm: 205e8a87f943a383e
### sdh: 266fc7bc69e23e94a
### sdn: 205e8a87f943a383e
### sdo: 259e04342c730046d
### sdp: 21f356012394a74f0
### sds: 269e599b56d920cd3
### sdu: 2d0bc8bb5f18458cf
### sdr: 21241b7334e5274d2
### sdt: 269e599b56d920cd3
### sdl: 2fe5799e30982c881
### sdq: 259e04342c730046d
### sdw: 21f356012394a74f0
### sdx: 2c8dacd7143254b70
### sdy: 21241b7334e5274d2
### sdaa: 2d0bc8bb5f18458cf
### sdz: 266fc7bc69e23e94a
### sdv: 2c8dacd7143254b70
### sdab: 266fc7bc69e23e94a
### sdac: 24b93ac5567327f2f
### sdae: 2fe5799e30982c881
### sdag: 205e8a87f943a383e
### sdad: 24b93ac5567327f2f
### sdaf: 2fe5799e30982c881
### sdaj: 2a6f8ecb92c2c7506
### sdai: 205e8a87f943a383e
### sdal: 233bef421cb87759d
### sdam: 23b4a22bf4954f0c6
### sdak: 259e04342c730046d
### sdan: 2a6f8ecb92c2c7506
### sdap: 21f356012394a74f0
### sdao: 233bef421cb87759d
### sdah: 259e04342c730046d
### sdar: 21f356012394a74f0
### sdat: 21241b7334e5274d2
### sdas: 23b4a22bf4954f0c6
### sdau: 2d0bc8bb5f18458cf
### sdav: 2a6f8ecb92c2c7506
### sdaw: 233bef421cb87759d
### sdaq: 21241b7334e5274d2
### sdax: 23b4a22bf4954f0c6
### sday: 2d0bc8bb5f18458cf
### sdaz: 2a6f8ecb92c2c7506
### sdba: 233bef421cb87759d
### sdbb: 23b4a22bf4954f0c6

[root@hferp71 ~]# more /etc/multipath.conf   --blacklist 黑名单 ，排查下面的UUID ，是系统盘，不做多路径
blacklist {
wwid 36000c297b9b69ad821ba52dd7a8103a1
wwid 36000c29b5effc0b462f8133744caf8c4
# devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
# devnode "^hd[a-z]"
# devnode "^sd[a-b]"
}

defaults {
path_selector "round-robin 0"
path_grouping_policy group_by_prio
rr_min_io 100
max_fds 8192
rr_weight uniform
failback immediate
user_friendly_names yes
find_multipaths yes
}

multipaths {
multipath {
wwid 269e599b56d920cd3
alias asm-dggrid1
}
multipath {
wwid 2c8dacd7143254b70
alias asm-dggrid2
}
multipath {
wwid 266fc7bc69e23e94a
alias asm-dggrid3
}
multipath {
wwid 24b93ac5567327f2f
alias asm-dggrid4
}
multipath {
wwid 2fe5799e30982c881
alias asm-dggrid5
}
multipath {
wwid 205e8a87f943a383e
alias asm-dggrid6
}
multipath {
wwid 259e04342c730046d
alias asm-dgsystem01
}
multipath {
wwid 21f356012394a74f0
alias asm-dgrecovery01
}
multipath {
wwid 21241b7334e5274d2
alias asm-data01
}
multipath {
wwid 2d0bc8bb5f18458cf
alias asm-data02
}
multipath {
wwid 2a6f8ecb92c2c7506
alias fs-backup01
}
multipath {
wwid 233bef421cb87759d
alias fs-backup02
}
multipath {
wwid 23b4a22bf4954f0c6
alias fs-backup03
}
}

[root@hferp72 ~]# more /etc/multipath.conf     --blacklist 黑名单 ，排查下面的UUID ，是系统盘，不做多路径
blacklist {
wwid 36000c29583d9ab34838ce7207521dd40
wwid 36000c29e988fcd7db059225b8e7c2813
# devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
# devnode "^hd[a-z]"
# devnode "^sd[a-b]"
}

defaults {
path_selector "round-robin 0"
path_grouping_policy group_by_prio
rr_min_io 100
max_fds 8192
rr_weight uniform
failback immediate
user_friendly_names yes
find_multipaths yes
}

multipaths {
multipath {
wwid 269e599b56d920cd3
alias asm-dggrid1
}
multipath {
wwid 2c8dacd7143254b70
alias asm-dggrid2
}
multipath {
wwid 266fc7bc69e23e94a
alias asm-dggrid3
}
multipath {
wwid 24b93ac5567327f2f
alias asm-dggrid4
}
multipath {
wwid 2fe5799e30982c881
alias asm-dggrid5
}
multipath {
wwid 205e8a87f943a383e
alias asm-dggrid6
}
multipath {
wwid 259e04342c730046d
alias asm-dgsystem01
}
multipath {
wwid 21f356012394a74f0
alias asm-dgrecovery01
}
multipath {
wwid 21241b7334e5274d2
alias asm-data01
}
multipath {
wwid 2d0bc8bb5f18458cf
alias asm-data02
}
multipath {
wwid 2a6f8ecb92c2c7506
alias fs-backup01
}
multipath {
wwid 233bef421cb87759d
alias fs-backup02
}
multipath {
wwid 23b4a22bf4954f0c6
alias fs-backup03
}
}

检查路径 multipath -ll

[root@hferp71 ~]#  multipath -ll
asm-dgrecovery01 (21f356012394a74f0) dm-6 ROCKET  ,IMAGEFILE
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:7 sdt  65:48  active ready running
  |- 34:0:0:7 sdaj 66:48  active ready running
  |- 35:0:0:7 sdae 65:224 active ready running
  `- 36:0:0:7 sdan 66:112 active ready running
asm-dgsystem01 (259e04342c730046d) dm-7 ROCKET  ,IMAGEFILE
size=5.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:6 sdm  8:192  active ready running
  |- 35:0:0:6 sdaa 65:160 active ready running
  |- 34:0:0:6 sdag 66:0   active ready running
  `- 36:0:0:6 sdal 66:80  active ready running
asm-data02 (2d0bc8bb5f18458cf) dm-9 ROCKET  ,IMAGEFILE
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:9 sdad 65:208 active ready running
  |- 34:0:0:9 sdam 66:96  active ready running
  |- 35:0:0:9 sdai 66:32  active ready running
  `- 36:0:0:9 sdap 66:144 active ready running
fs-backup03 (23b4a22bf4954f0c6) dm-12 ROCKET  ,IMAGEFILE
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 37:0:0:2 sdas 66:192 active ready running
  |- 38:0:0:2 sdaw 67:0   active ready running
  |- 40:0:0:2 sdbb 67:80  active ready running
  `- 39:0:0:2 sdba 67:64  active ready running
asm-data01 (21241b7334e5274d2) dm-8 ROCKET  ,IMAGEFILE
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:8 sdab 65:176 active ready running
  |- 34:0:0:8 sdak 66:64  active ready running
  |- 36:0:0:8 sdao 66:128 active ready running
  `- 35:0:0:8 sdaf 65:240 active ready running
fs-backup02 (233bef421cb87759d) dm-10 ROCKET  ,IMAGEFILE
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 37:0:0:1 sdar 66:176 active ready running
  |- 38:0:0:1 sdau 66:224 active ready running
  |- 39:0:0:1 sday 67:32  active ready running
  `- 40:0:0:1 sdaz 67:48  active ready running
fs-backup01 (2a6f8ecb92c2c7506) dm-11 ROCKET  ,IMAGEFILE
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 37:0:0:0 sdaq 66:160 active ready running
  |- 38:0:0:0 sdat 66:208 active ready running
  |- 39:0:0:0 sdav 66:240 active ready running
  `- 40:0:0:0 sdax 67:16  active ready running
asm-dggrid6 (205e8a87f943a383e) dm-5 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:5 sdh  8:112  active ready running
  |- 35:0:0:5 sdy  65:128 active ready running
  |- 34:0:0:5 sdv  65:80  active ready running
  `- 36:0:0:5 sdah 66:16  active ready running
asm-dggrid5 (2fe5799e30982c881) dm-3 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:4 sdg  8:96   active ready running
  |- 34:0:0:4 sdn  8:208  active ready running
  |- 35:0:0:4 sdw  65:96  active ready running
  `- 36:0:0:4 sdac 65:192 active ready running
asm-dggrid4 (24b93ac5567327f2f) dm-4 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:3 sdf  8:80   active ready running
  |- 34:0:0:3 sdl  8:176  active ready running
  |- 35:0:0:3 sdu  65:64  active ready running
  `- 36:0:0:3 sdz  65:144 active ready running
asm-dggrid3 (266fc7bc69e23e94a) dm-2 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:2 sde  8:64   active ready running
  |- 34:0:0:2 sdk  8:160  active ready running
  |- 35:0:0:2 sdq  65:0   active ready running
  `- 36:0:0:2 sdx  65:112 active ready running
asm-dggrid2 (2c8dacd7143254b70) dm-0 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:1 sdd  8:48   active ready running
  |- 34:0:0:1 sdj  8:144  active ready running
  |- 35:0:0:1 sdp  8:240  active ready running
  `- 36:0:0:1 sds  65:32  active ready running
asm-dggrid1 (269e599b56d920cd3) dm-1 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:0 sdc  8:32   active ready running
  |- 34:0:0:0 sdi  8:128  active ready running
  |- 35:0:0:0 sdo  8:224  active ready running
  `- 36:0:0:0 sdr  65:16  active ready running

[root@hferp72 ~]# multipath -ll
asm-dgrecovery01 (21f356012394a74f0) dm-6 ROCKET  ,IMAGEFILE
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:7 sdp  8:240  active ready running
  |- 34:0:0:7 sdw  65:96  active ready running
  |- 36:0:0:7 sdar 66:176 active ready running
  `- 35:0:0:7 sdap 66:144 active ready running
asm-dgsystem01 (259e04342c730046d) dm-7 ROCKET  ,IMAGEFILE
size=5.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 34:0:0:6 sdq  65:0   active ready running
  |- 33:0:0:6 sdo  8:224  active ready running
  |- 36:0:0:6 sdah 66:16  active ready running
  `- 35:0:0:6 sdak 66:64  active ready running
asm-data02 (2d0bc8bb5f18458cf) dm-8 ROCKET  ,IMAGEFILE
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:9 sdu  65:64  active ready running
  |- 34:0:0:9 sdaa 65:160 active ready running
  |- 36:0:0:9 sdau 66:224 active ready running
  `- 35:0:0:9 sday 67:32  active ready running
fs-backup03 (23b4a22bf4954f0c6) dm-12 ROCKET  ,IMAGEFILE
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 37:0:0:2 sdam 66:96  active ready running
  |- 38:0:0:2 sdas 66:192 active ready running
  |- 40:0:0:2 sdbb 67:80  active ready running
  `- 39:0:0:2 sdax 67:16  active ready running
asm-data01 (21241b7334e5274d2) dm-9 ROCKET  ,IMAGEFILE
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 34:0:0:8 sdy  65:128 active ready running
  |- 33:0:0:8 sdr  65:16  active ready running
  |- 36:0:0:8 sdat 66:208 active ready running
  `- 35:0:0:8 sdaq 66:160 active ready running
fs-backup02 (233bef421cb87759d) dm-11 ROCKET  ,IMAGEFILE
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 37:0:0:1 sdal 66:80  active ready running
  |- 38:0:0:1 sdao 66:128 active ready running
  |- 40:0:0:1 sdba 67:64  active ready running
  `- 39:0:0:1 sdaw 67:0   active ready running
fs-backup01 (2a6f8ecb92c2c7506) dm-10 ROCKET  ,IMAGEFILE
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 37:0:0:0 sdaj 66:48  active ready running
  |- 38:0:0:0 sdan 66:112 active ready running
  |- 39:0:0:0 sdav 66:240 active ready running
  `- 40:0:0:0 sdaz 67:48  active ready running
asm-dggrid6 (205e8a87f943a383e) dm-3 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 34:0:0:5 sdn  8:208  active ready running
  |- 33:0:0:5 sdm  8:192  active ready running
  |- 36:0:0:5 sdag 66:0   active ready running
  `- 35:0:0:5 sdai 66:32  active ready running
asm-dggrid5 (2fe5799e30982c881) dm-5 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 34:0:0:4 sdl  8:176  active ready running
  |- 33:0:0:4 sdk  8:160  active ready running
  |- 36:0:0:4 sdae 65:224 active ready running
  `- 35:0:0:4 sdaf 65:240 active ready running
asm-dggrid4 (24b93ac5567327f2f) dm-2 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 34:0:0:3 sdj  8:144  active ready running
  |- 33:0:0:3 sdi  8:128  active ready running
  |- 36:0:0:3 sdac 65:192 active ready running
  `- 35:0:0:3 sdad 65:208 active ready running
asm-dggrid3 (266fc7bc69e23e94a) dm-4 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 34:0:0:2 sdh  8:112  active ready running
  |- 33:0:0:2 sde  8:64   active ready running
  |- 36:0:0:2 sdz  65:144 active ready running
  `- 35:0:0:2 sdab 65:176 active ready running
asm-dggrid2 (2c8dacd7143254b70) dm-0 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 33:0:0:1 sdd  8:48   active ready running
  |- 34:0:0:1 sdg  8:96   active ready running
  |- 36:0:0:1 sdx  65:112 active ready running
  `- 35:0:0:1 sdv  65:80  active ready running
asm-dggrid1 (269e599b56d920cd3) dm-1 ROCKET  ,IMAGEFILE
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 34:0:0:0 sdf  8:80   active ready running
  |- 33:0:0:0 sdc  8:32   active ready running
  |- 36:0:0:0 sds  65:32  active ready running
  `- 35:0:0:0 sdt  65:48  active ready running

AMS磁盘配置

创建用户和组

/usr/sbin/groupadd -g 5001 oinstall
/usr/sbin/groupadd -g 5002 dba
/usr/sbin/groupadd -g 5003 oper
/usr/sbin/groupadd -g 5004 asmadmin
/usr/sbin/groupadd -g 5005 asmoper
/usr/sbin/groupadd -g 5006 asmdba
/usr/sbin/useradd -u 6001 -g oinstall -G dba,asmdba,oper oracle
/usr/sbin/useradd -u 6002 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid
mkdir -p /oracle/app/grid
mkdir -p /oracle/app/12.1/grid
chown -R grid:oinstall /oracle
mkdir -p /oracle/app/oraInventory
chown -R grid:oinstall /oracle/app/oraInventory
mkdir -p /oracle/app/oracle
chown -R oracle:oinstall /oracle/app/oracle
chmod -R 775 /oracle

第一种：6.X和7.X：/etc/udev/rules.d/99-oracle.rules

KERNEL==“dm-*”, OWNER=“grid”, GROUP=“asmadmin”, MODE=“0660”

第二种6.X和7.X：vi /etc/udev/rules.d/99-oracle-asmdevices.rules

KERNEL==“dm-*”, NAME=“asm-dgrecovery01”,OWNER=“grid”, GROUP=“asmadmin”,MODE=“0660”

KERNEL==“dm-*”, NAME=“asm-dgsystem01”,OWNER=“grid”, GROUP=“asmadmin”,MODE=“0660”

第三种：7.X推荐

dmsetup ls |grep asm

[root@hferp71 ~]# dmsetup ls |grep asm
asm-dgrecovery01        (253:6)
asm-dgsystem01  (253:7)
asm-data02      (253:9)
asm-data01      (253:8)
asm-dggrid6     (253:5)
asm-dggrid5     (253:3)
asm-dggrid4     (253:4)
asm-dggrid3     (253:2)
asm-dggrid2     (253:0)
asm-dggrid1     (253:1)

[root@hferp71 ~]# for i in asm-dggrid1 asm-dggrid2 asm-dggrid3 asm-dggrid4 asm-dggrid5 asm-dggrid6 asm-dgsystem01 asm-dgrecovery01 asm-data01 asm-data02;
> do
> printf "%s %s\n" "$i" "$(udevadm info --query=all --name=/dev/mapper/$i |grep -i dm_uuid)";
> done
asm-dggrid1 E: DM_UUID=mpath-269e599b56d920cd3
asm-dggrid2 E: DM_UUID=mpath-2c8dacd7143254b70
asm-dggrid3 E: DM_UUID=mpath-266fc7bc69e23e94a
asm-dggrid4 E: DM_UUID=mpath-24b93ac5567327f2f
asm-dggrid5 E: DM_UUID=mpath-2fe5799e30982c881
asm-dggrid6 E: DM_UUID=mpath-205e8a87f943a383e
asm-dgsystem01 E: DM_UUID=mpath-259e04342c730046d
asm-dgrecovery01 E: DM_UUID=mpath-21f356012394a74f0
asm-data01 E: DM_UUID=mpath-21241b7334e5274d2
asm-data02 E: DM_UUID=mpath-2d0bc8bb5f18458cf

[root@hferp71 ~]# more  /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-269e599b56d920cd3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-2c8dacd7143254b70",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-266fc7bc69e23e94a",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-24b93ac5567327f2f",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-2fe5799e30982c881",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-205e8a87f943a383e",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-259e04342c730046d",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-21f356012394a74f0",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-21241b7334e5274d2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-2d0bc8bb5f18458cf",OWNER="grid",GROUP="asmadmin",MODE="0660"

[root@hferp72 ~]# more  /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-269e599b56d920cd3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-2c8dacd7143254b70",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-266fc7bc69e23e94a",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-24b93ac5567327f2f",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-2fe5799e30982c881",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-205e8a87f943a383e",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-259e04342c730046d",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-21f356012394a74f0",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-21241b7334e5274d2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-2d0bc8bb5f18458cf",OWNER="grid",GROUP="asmadmin",MODE="0660"

重新加载策略：/sbin/udevadm control --reload-rules

激活更改：sbin/udevadm trigger --type=devices --action=change

查看磁盘ls -lsa /dev/dm*

[root@hferp71 ~]# ls -lsa /dev/dm*
0 brw-rw---- 1 grid asmadmin 253,  0 Nov 26 09:16 /dev/dm-0
0 brw-rw---- 1 grid asmadmin 253,  1 Nov 26 09:16 /dev/dm-1
0 brw-rw---- 1 root disk     253, 10 Nov 26 09:16 /dev/dm-10
0 brw-rw---- 1 root disk     253, 11 Nov 26 09:16 /dev/dm-11
0 brw-rw---- 1 root disk     253, 12 Nov 26 09:16 /dev/dm-12
0 brw-rw---- 1 root disk     253, 13 Nov 26 09:16 /dev/dm-13
0 brw-rw---- 1 grid asmadmin 253,  2 Nov 26 09:17 /dev/dm-2
0 brw-rw---- 1 grid asmadmin 253,  3 Nov 26 09:17 /dev/dm-3
0 brw-rw---- 1 grid asmadmin 253,  4 Nov 26 09:17 /dev/dm-4
0 brw-rw---- 1 grid asmadmin 253,  5 Nov 26 09:16 /dev/dm-5
0 brw-rw---- 1 grid asmadmin 253,  6 Nov 26 09:17 /dev/dm-6
0 brw-rw---- 1 grid asmadmin 253,  7 Nov 26 09:17 /dev/dm-7
0 brw-rw---- 1 grid asmadmin 253,  8 Nov 26 09:17 /dev/dm-8
0 brw-rw---- 1 grid asmadmin 253,  9 Nov 26 09:17 /dev/dm-9
0 crw-rw---- 1 root audio     14,  9 Nov 26 09:16 /dev/dmmidi

系统参数设置

主机名不要使用大写字母，且主机名不能有-和_
vi /etc/hosts
#public ip
172.30.21.171 hferp71
172.30.21.172 hferp72
#priv ip
10.10.10.71 hferp71prv
10.10.10.72 fgerp72prv
#vip ip
172.30.21.173 hferp71vip
172.30.21.174 hferp72vip
#scan ip
172.30.21.175 hferpdbscan

--配置环境变量
echo "export LANG=en_US" >> ~/.bash_profile
cat ~/.bash_profile
--创建用户、组、目录
/usr/sbin/groupadd -g 5001 oinstall
/usr/sbin/groupadd -g 5002 dba
/usr/sbin/groupadd -g 5003 oper
/usr/sbin/groupadd -g 5004 asmadmin
/usr/sbin/groupadd -g 5005 asmoper
/usr/sbin/groupadd -g 5006 asmdba
/usr/sbin/useradd -u 6001 -g oinstall -G dba,asmdba,oper oracle
/usr/sbin/useradd -u 6002 -g oinstall -Gasmadmin,asmdba,asmoper,oper,dba grid
--改密码
passwd grid
passwd oracle

mkdir -p /oracle/app/grid
mkdir -p /oracle/app/11.2.0/grid
chown -R grid:oinstall /oracle

mkdir -p /oracle/app/oraInventory
chown -R grid:oinstall /oracle/app/oraInventory

mkdir -p /oracle/app/oracle
chown -R oracle:oinstall /oracle/app/oracle
chmod -R 775 /oracle

配置yum
mount /dev/cdrom /mnt
cd /etc/yum.repos.d
mkdir bk
mv *.repo bk/
echo "[EL]" >> /etc/yum.repos.d/itpux.repo
echo "name =Linux 7.x DVD" >> /etc/yum.repos.d/itpux.repo
echo "baseurl=file:///mnt" >> /etc/yum.repos.d/itpux.repo
echo "gpgcheck=0" >> /etc/yum.repos.d/itpux.repo
echo "enabled=1" >> /etc/yum.repos.d/itpux.repo
cat /etc/yum.repos.d/itpux.repo

yum -y install autoconf
yum -y install automake
yum -y install binutils
yum -y install binutils-devel
yum -y install bison
yum -y install cpp
yum -y install dos2unix
yum -y install ftp
yum -y install gcc
yum -y install gcc-c++
yum -y install lrzsz
yum -y install python-devel
yum -y install compat-db*
yum -y install compat-gcc-34
yum -y install compat-gcc-34-c++
yum -y install compat-libcap1
yum -y install compat-libstdc++-33
yum -y install compat-libstdc++-33.i686
yum -y install glibc-*
yum -y install glibc-*.i686
yum -y install libXpm-*.i686
yum -y install libXp.so.6
yum -y install libXt.so.6
yum -y install libXtst.so.6
yum -y install libXext
yum -y install libXext.i686
yum -y install libXtst
yum -y install libXtst.i686
yum -y install libX11
yum -y install libX11.i686
yum -y install libXau
yum -y install libXau.i686
yum -y install libxcb
yum -y install libxcb.i686
yum -y install libXi
yum -y install libXi.i686
yum -y install libgcc_s.so.1
yum -y install libstdc++.i686
yum -y install libstdc++-devel
yum -y install libstdc++-devel.i686
yum -y install libaio
yum -y install libaio.i686
yum -y install libaio-devel
yum -y install libaio-devel.i686
yum -y install ksh
yum -y install libXp
yum -y install libaio-devel
yum -y install numactl
yum -y install numactl-devel
yum -y install make -y
yum -y install sysstat -y
yum -y install unixODBC
yum -y install unixODBC-devel
yum -y install elfutils-libelf-devel-0.97
yum -y install elfutils-libelf-devel
yum -y install redhat-lsb-core
yum -y install unzip

修改系统相关参数

修改资源限制参数
vi /etc/security/limits.conf
#ORACLE SETTING
grid soft nproc 16384
grid hard nproc 16384
grid soft nofile 65536
grid hard nofile 65536
grid soft stack 32768
grid hard stack 32768
oracle soft nproc 16384
oracle hard nproc 16384
oracle soft nofile 65536
oracle hard nofile 65536
oracle soft stack 32768
oracle hard stack 32768
oracle hard memlock 2000000
oracle soft memlock 2000000  #-- 单位是KB，大于SGA，小于物理内存

ulimit -a

修改nproc参数

#Amend the "/etc/security/limits.d/20-nproc.conf" file as described below.
See MOS Note [ID 1487773.1]
控制给用户分配的资源
echo "* - nproc 16384" > /etc/security/limits.d/20-nproc.conf

控制给用户分配的资源（使上面生效）

echo "session required pam_limits.so" >> /etc/pam.d/login
cat /etc/pam.d/login

修改内核参数

vi /etc/sysctl.conf

#ORACLE SETTING
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
kernel.panic_on_oops = 1
vm.nr_hugepages = 1000 --2500
kernel.shmmax = 1610612736   --5033164800
kernel.shmall = 393216  --1228800
kernel.shmmni = 4096

 sysctl -p

关闭透明页

cat /sys/kernel/mm/transparent_hugepage/defrag
 [always] madvise never
cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

vi /etc/rc.d/rc.local

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

chmod +x /etc/rc.d/rc.local

关闭numa功能

numactl --hardware

vi /etc/default/grub

numa=off

重新编译grub2-mkconfig -o /etc/grub2.cfg

图形界面启动级别从5到3

systemctl set-default multi-user.target

共享内存段

echo “none /dev/shm tmpfs defaults,size=3072m 0 0” >>/etc/fstab 6144
mount -o remount /dev/shm

修改时区

ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
Hwclock

禁用SELINUX

echo “SELINUX=disabled” > /etc/selinux/config
echo "#SELINUXTYPE=targeted " >> /etc/selinux/config
setenforce 0

关闭防火墙

systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl status firewalld.service

配置NOZEROCONF

echo “NOZEROCONF=yes” >> /etc/sysconfig/network

修改nsswitch.conf 保存的是域名检索顺序

vi /etc/nsswitch.conf
hosts: files dns myhostname nis

avahi-daemon

avahi-daemon守护进程配合缓存用户程序的答复，以帮助减少因答复而产生的网络流量；根据官方文档列出来的BUG，建议关闭这个进程。
Linux7中：
systemctl stop avahi-daemon.socket avahi-daemon.service
systemctl disable avahi-daemon.socket avahi-daemon.service

两台机时间要一样，禁止NTP

date -s “Wed Jan 23 23:06:03 CST 2019”

systemctl stop ntpd
systemctl disabled ntpd
systemctl status ntpd

配置环境变量

第一台：

su – grid
vi ~/.bash_profile

PS1="[`whoami`@`hostname`:"'$PWD]$'
export PS1
umask 022
#alias sqlplus="rlwrap sqlplus"
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
ORACLE_BASE=/oracle/app/grid; export ORACLE_BASE
ORACLE_HOME=/oracle/app/11.2.0/grid; export ORACLE_HOME
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

su – oracle
vi ~/.bash_profile

PS1="[`whoami`@`hostname`:"'$PWD]$'
#alias sqlplus="rlwrap sqlplus"
#alias rman="rlwrap rman"
export PS1
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
export ORACLE_UNQNAME=hferpdb
ORACLE_BASE=/oracle/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME
ORACLE_SID=hferpdb1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG
PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/11.2.0/db_1/bin:$ORACLE_HOME/b
in; export PATH
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

第二台：

su - grid
vi ~/.bash_profile

PS1="[`whoami`@`hostname`:"'$PWD]$'
export PS1
umask 022
#alias sqlplus="rlwrap sqlplus"
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
ORACLE_SID=+ASM2; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
ORACLE_BASE=/oracle/app/grid; export ORACLE_BASE
ORACLE_HOME=/oracle/app/11.2.0/grid; export ORACLE_HOME
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

su - oracle
PS1="[`whoami`@`hostname`:"'$PWD]$'
#alias sqlplus="rlwrap sqlplus"
#alias rman="rlwrap rman"
export PS1
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
export ORACLE_UNQNAME=hferpdb
ORACLE_BASE=/oracle/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME
ORACLE_SID=hferpdb2; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG
PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/ll.2.0/db_1/bin:$ORACLE_HOME/b
in; export PATH
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

配置SSH信任关系-在第一台机

./sshUserSetup.sh -user grid -hosts "hferp71 hferp72" -advanced -exverify –confirm
./sshUserSetup.sh -user oracle -hosts "hferp71 hferp72" -advanced -exverify -confirm

Grid集群安装

查看系统参数

查看内存： free -m 或 cat /proc/meninfo

查看cpu：cat /proc/cpuinfo

查看硬件：deidecode |more

修改权限(chown -R grid:oinstall /backup chmod -R 777 /backup/)

进入grid用户的backup：su - grid

解压软件包：[grid@hferp71:/backup]$unzip p13390677_112040_Linux-x86-64_3of7.zip

解压软件包：[grid@hferp71:/backup]$unzip p19404309_112040_Linux-x86-64.zip

复制文件（解决验证问题）：[grid@hferp71:/backup]$cp b19404309/grid/cvu_prereq.xml grid/stage/cvu/

在root下安装（解决磁盘验证问题）[root@hferp71 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm

安装vnc：[root@hferp71 rpm]# mount /dev/cdrom /mnt/

安装grid：[grid@hferp71 grid]$ ./runInstaller -jreLoc /etc/alternatives/jre_1.8.0

第二条在执行会报错 --【Rhel7安装RAC 11gR2时运行root.sh报错找不到ohas服务妙招_redhat7安装oracle 11g rac root.sh-CSDN博客】

解决：

--[grid@hferp71:/backup]$unzip p18370031_112040_Linux-x86-64.zip
--[grid@hferp71:/backup/18370031]$/oracle/app/11.2.0/grid/OPatch/opatch apply
--两个节点一起在71上面打了，没必要在拷贝了
--[grid@hferp72:/oracle/app/11.2.0]$unzip p18370031_112040_Linux-x86-64.zip
--[grid@hferp72:/home/grid]$scp hferp71:/backup/p183*zip /oracle/app/11.2.0/

查看补丁有没有打上去

[grid@hferp71:/backup/18370031]$/oracle/app/11.2.0/grid/OPatch/opatch lsinventory

Asm磁盘组安装

在grid下：asmca

[grid@hferp71 grid]$ asmca

设置root环境变量两台机

[root@hferp71 ~]# vi ~/.bash_profile

[root@hferp72 ~]# vi ~/.bash_profile

PATH= $P A T H : / or a c l e / a pp /11.2.0/ g r i d / bin :$ HOME/bin

/oracle/app/11.2.0/grid/bin:

用grid用户进去，检查下磁盘组

crs_stat –t

用root用户：crsctl check crs

查看监听：[grid@hferp71:/home/grid]$lsnrctl status

数据库软件安装

用oracle用户登录，解压数据包

[oracle@hferp71:/backup]$unzip p13390677_112040_Linux-x86-64_1of7.zip

[oracle@hferp71:/backup]$unzip p13390677_112040_Linux-x86-64_2of7.zip

[oracle@hferp71:/backup]$vncserver

[oracle@hferp71 database]$ ./runInstaller -jreLoc /etc/alternatives/jre_1.8.0

报错：Error in invoking target ‘agent nmhs’ of makefile ……

Oracle用户进入：ORACLE_HOME；/oracle/app/oracle/product/11.2.0/db_1 ；cd sysman/lib

cp ins_emagent.mk ins_emagent.mk.bak –将文件备份

vi ins_emagent.mk 搜索NMECTL 在后面加上-lnnz11

安装数据库集群

[oracle@hferp71 database]$ dbca

Oracle11gR2 RAC 数据库补丁安装

补丁上传解压

停止应用和服务

生产环境一定要备份（停止数据库与集群，备份安装代码，备份数据库）

安装补丁工具Opatch

建议重启下两台主机

在线打补丁（数据库停止，dbconsole服务停掉，但集群一定打开）

验证补丁

启动应用测试

需要虚拟机安装环境的，文章后留言或私信，发网盘链接，另外博主有公网的高性能服务器可对外出租，可以用xshell工具直接连接，用于学习测试，有需要的可以私信。

RHEL7+Oracle11.2 RAC集群-多路径（multipath+udev）安装步骤