如何在云环境下部署oracle18c RAC(京东云公有云)

理论篇

安装oracle RAC 主要需要几个条件

  • 私有网络和公共网络隔离(不在同一网段)
  • 共享存储
  • 网络需要支持mulitcast

我们可以利用京东云提供的基础设施解决以上问题

  • 通过创建"创建无预设CIDR私有网络"的vpc进行网络隔离,实现public和private网段隔离
  • 通过创建"多点挂载存储"实现ASM所需的共享存储
  • 通过n2n点对点实现multicast支持

 

实施篇

 

网络配置

  • 创建“创建无预设CIDR私有网络”vpc
    控制台=>网络=>私有网络(vpc)=>创建=>选择"创建无预设CIDR私有网络"
  • 创建子网
    分别创建两个不同网段的子网
    10.0.1.0/24
    192.168.1.0/24

 

主机创建

 

创建共享磁盘

控制台=>弹性计算=>云硬盘=>创建=>选择"多点挂载"(目前仅华东有此选项)

 

创建弹性网卡

192.168.1.10 rac1private
192.168.1.20 rac2private

 

修改hosts文件

vi /etc/hosts

 

#oracle rac public ip
10.0.0.10 rac1
10.0.0.20 rac2

#oracle rac vip
10.0.0.11 rac1vip
10.0.0.21 rac2vip

#oracle rac scanip
10.0.0.30 scan-rac

#oracle rac private ip
192.168.2.10 rac1private
192.168.2.20 rac2private

这里可以利用弹性网卡来绑定多个ip地址来实现public ip和vip

 

创建用户

https://docs.oracle.com/en/database/oracle/oracle-database/18/cwlin/identifying-an-oracle-software-owner-user-account.html#GUID-0A95F4B1-1045-455D-9897-A23012E4E27F

 

用户和组:
/usr/sbin/groupadd -g 54321 oinstall
/usr/sbin/groupadd -g 54322 dba
/usr/sbin/groupadd -g 54323 oper
/usr/sbin/groupadd -g 54324 backupdba
/usr/sbin/groupadd -g 54325 dgdba
/usr/sbin/groupadd -g 54326 kmdba
/usr/sbin/groupadd -g 54327 asmdba
/usr/sbin/groupadd -g 54328 asmoper
/usr/sbin/groupadd -g 54329 asmadmin
/usr/sbin/groupadd -g 54330 racdba

/usr/sbin/useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle
/usr/sbin/useradd -u 54331 -g oinstall -G dba,asmdba,asmoper,asmadmin grid

echo oracle | passwd --stdin oracle
echo oracle | passwd --stdin grid

11、目录:
mkdir -p /u01/app/18.3.0/grid
mkdir -p /u01/app/grid
mkdir -p /u01/app/oracle
chown -R grid:oinstall /u01
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/

12、环境变量

节点1:
--grid
export ORACLE_SID=+ASM1
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/18.3.0/grid
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

--oracle用户
export ORACLE_SID=db1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/18.3.0/db_1
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib


节点2:
--gird
export ORACLE_SID=+ASM2
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/18.3.0/grid
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

--oracle用户
export ORACLE_SID=db2
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/18.3.0/db_1
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

  • 验证用户组

 

$ grep "oinstall" /etc/group
oinstall:x:54321:grid,oracle
You can then use the ID command to verify that the Oracle installation owners you intend to use have the Oracle Inventory group as their primary group. For example:

$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba), 
54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)

$ id grid
uid=54331(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),
54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)

 

操作系统参数

  • vi /etc/sysctl.conf
    sysctl -p /etc/sysctl.d/*.conf

 

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 67108864
kernel.shmmax = 274877906944
kernel.shmmni = 4096
#kernel.sem = 250 32000 100 128
kernel.sem = 5010 641280 5010 128
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
  • 限制文件:vi /etc/security/limits.conf

 

grid soft nproc 2047 
grid hard nproc 16384 
grid soft nofile 1024 
grid hard nofile 65536 
grid soft stack 10240 
oracle soft nproc 2047 
oracle hard nproc 16384 
oracle soft nofile 1024 
oracle hard nofile 65536 
oracle soft stack 10240
* soft memlock 524288000
* hard memlock 524288000
  • vi /etc/pam.d/login 文件
    session required pam_limits.so

  • vi /etc/fstab
    tmpfs /dev/shm tmpfs size=5000m 0 0

  • /etc/sysconfig/network
    NOZEROCONF=yes

  • vi /etc/profile

 

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
    if [ $SHELL = "/bin/ksh" ]; then
        ulimit -p 16384
        ulimit -n 65536
    else
        ulimit -u 16384 -n 65536
    fi
    umask 022
fi

 

安装必要包

 

yum install -y gcc-c++
yum install -y bc
yum install -y binutils
yum install -y compat-libcap1
yum install -y compat-libstdc++-33.i686    
yum install -y compat-libstdc++-33.x86_64
yum install -y glibc-2.17-36.el7.i686
yum install -y glibc-2.17-36.el7.x86_64
yum install -y glibc-devel.i686
yum install -y glibc-devel.x86_64
yum install -y ksh
yum install -y libaio.i686
yum install -y libaio.x86_64
yum install -y libaio-devel.i686
yum install -y libaio-devel.x86_64
yum install -y libX11.i686
yum install -y libX11.x86_64
yum install -y libXau.i686
yum install -y libXau.x86_64
yum install -y libXi.i686
yum install -y libXi.x86_64
yum install -y libXtst.i686
yum install -y libXtst.el7.x86_64
yum install -y libgcc.i686
yum install -y libgcc.x86_64
yum install -y libstdc++.i686
yum install -y libstdc++
yum install -y libstdc++-devel.i686
yum install -y libstdc++-devel.x86_64
yum install -y libxcb.i686
yum install -y libxcb.x86_64
yum install -y make
yum install -y nfs-utils 
yum install -y net-tools
yum install -y python
yum install -y python-configshell
yum install -y python-rtslib
yum install -y python-six
yum install -y smartmontools
yum install -y sysstat
yum install -y targetcli
  • 为保证xwindow正常显示的必要安装

 

yum groupinstall "X Window System" "Fonts" -y 

 

准备asm磁盘组(所有节点操作)

  • asmlib方式
    参考文档:https://docs.microsoft.com/zh-tw/azure/virtual-machines/workloads/oracle/configure-oracle-asm

    • 从 https://www.oracle.com/technetwork/server-storage/linux/asmlib/rhel7-2773795.html下载rpm
    • 安装
      rpm -Uvh oracleasm-support-2.1.11-2.el7.x86_64.rpm
      yum -y install kmod-oracleasm.x86_64
      rpm -ivh oracleasmlib-2.0.12-1.el7.x86_64.rpm
    • 配置asm
    /usr/sbin/oracleasm configure -i
    
    Configuring the Oracle ASM library driver.
    
    This will configure the on-boot properties of the Oracle ASM library
    driver. The following questions will determine whether the driver is
    loaded on boot and what permissions it will have. The current values
    will be shown in brackets ('[]'). Hitting  without typing an
    answer will keep that current value. Ctrl-C will abort.
    
    Default user to own the driver interface []: grid
    Default group to own the driver interface []: asmadmin
    Start Oracle ASM library driver on boot (y/n) [n]: y
    Scan for Oracle ASM disks on boot (y/n) [y]: y
    Writing Oracle ASM library driver configuration: done
    
    

    此处需要reboot

    • 创建asm磁盘组
    oracleasm createdisk ocr /dev/vdb1
    oracleasm createdisk data /dev/vdc1
    oracleasm scandisks
    oracleasm listdisks
    
    • 清除磁盘数据
    dd if=/dev/zero of=/dev/vdb1 bs=1M count=1000
    

 

n2n安装及配置

  • 安装

    yum install -y gcc
    yum install -y openssl-devel
    
    git clone https://github.com/ntop/n2n.git
    cd n2n
    git checkout 2.4-stable
    
    make
    make PREFIX=/opt/n2n install
    
  • 安装完毕后启动supernode服务

    nohup /opt/n2n/sbin/supernode -l 65530 &
    
  • 在RAC节点安装ntop软件(计划加入网络的节点都需要安装ntop)

    scp -r /opt/n2n 10.0.0.10:/opt
    scp -r /opt/n2n 10.0.0.20:/opt
    
    • 节点1
    /opt/n2n/sbin/edge -d edge1 -a 192.168.2.10 -s 255.255.255.0 -c dtstack -k  dtstack -l 192.168.1.40:65530 -E -r
    
    • 节点2
    /opt/n2n/sbin/edge -d edge1 -a 192.168.2.20 -s 255.255.255.0 -c dtstack -k  dtstack -l 192.168.1.40:65530 -E -r
    
  • 测试multicast(非必要,主要是验证multicast无异常)

    ./mcasttest.pl -n rac1,rac2 -i edge1
    

 

ssh互信

  • 分别在grid和oracle用户,每个节点上执行。

    • 创建ssh目录
    mkdir .ssh
    chmod 700 ~/.ssh
    cd ~/.ssh
    
    • 生成秘钥
    ssh-keygen -t rsa
    ssh-keygen -t dsa
    
  • 在rac1上将密钥添加到授权文件中,grid和oracle用户分别执行

    • 本机的拷贝
    cat id_dsa.pub >> authorized_keys
    cat id_rsa.pub >> authorized_keys
    
    • 拷贝另一台主机的秘钥
    ssh rac2 cat ~/.ssh/id_dsa.pub >> authorized_keys
    ssh rac2 cat ~/.ssh/id_rsa.pub >> authorized_keys
    
    • 将本机的authorized_keys的文件拷贝到另一台主机上
    scp authorized_keys rac2:/home/grid/.ssh/
    scp authorized_keys rac2:/home/oracle/.ssh/
    
  • 修改两个主机的授权密钥的权限

    chmod 600 ~/.ssh/authorized_keys
    
  • 测试

    • 获取服务器公钥
    ssh-keyscan rac1  >> ~/.ssh/known_hosts
    ssh-keyscan rac2  >> ~/.ssh/known_hosts
    ssh-keyscan rac1private  >> ~/.ssh/known_hosts
    ssh-keyscan rac2private  >> ~/.ssh/known_hosts
    
    • 测试
    ssh rac1 date
    ssh rac2 date
    ssh rac1private date
    ssh rac2private date
    

 

安装前检查

 

$ORACLE_HOME/runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose
  • 错误处理
    • PRVE-0421
      修改/etc/fstab
    tmpfs  /dev/shm tmpfs  size=5000m   0 0
    

 

安装GI--grid用户下

  • 解压安装压缩包
    cd /u01/app/12.2.0/grid/grid
    unzip /oraclesoftware/linuxx64_12201_grid_home.zip
    
  • 安装cvudisk RPM包
    root 用户安装
    cvuqdisk存于oracle GI安装介质的cv/rpm目录下
    cd /u01/app/18.3.0/grid
    export CVUQDISK_GRP=asmadmin
    rpm -ivh cv/rpm/cvuqdisk-1.0.10-1.rpm
    
    scp cv/rpm/cvuqdisk-1.0.10-1.rpm 10.0.0.20:/root
    export CVUQDISK_GRP=asmadmin
    rpm -ivh /root/cvuqdisk-1.0.10-1.rpm
    
  • grid用户执行$HOME_ORACLE/gridSetup.sh

 

安装DB--oracle用户下

  • 解压安装压缩包
    cd /u01/app/oracle/product/18.3.0/db_1
    unzip /oraclesoftware/LINUX.X64_180000_db_home.zip
    
  • oracle用户执行$HOME_ORACLE/runInstaller.sh

 

查看crs服务状态

 

crsctl status res -t

 

你可能感兴趣的:(Oracle数据库,云计算)