md:
mdadm: 将任何块设备做成RAID
模式化的命令：
1.创建模式
-C 专用选项： -l: 级别 -n #: 设备个数 -a {yes|no}: 是否自动为其创建设备文件 -c: CHUNK大小, 2^n，默认为64KB,最好为块的倍数 ,为了提高raid 0/5 的性能 mke2fs -E stride=16 -b 4096 /dev/md0 -x #: 指定空闲盘个数
3.管理模式
--add, --remove, --fail
mdadm /dev/md# --fail /dev/sda7
4.监控模式
-F
5.增长模式
-G
6.装配模式
-A
查看RAID阵列的详细信息
mdadm -D /dev/md#
--detail
停止阵列：
mdadm -S /dev/md#
--stop
删除设备文件
rm /dev/md#
创建功能:
1.创建一个空间大小为6G的RAID0设备
RAID0 6G:
2: 3GB
首先创建2个3G大小的分区,并把类型调成read的类型
root@VM_160_34_centos:~> fdisk /dev/xvdb WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended /dev/xvdb5 1 262 2104452 83 Linux /dev/xvdb6 263 655 3156741 83 Linux /dev/xvdb7 656 1048 3156741 83 Linux /dev/xvdb8 1049 1441 3156741 83 Linux /dev/xvdb9 1442 1834 3156741 83 Linux Command (m for help): d Partition number (1-9): 4 Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System Command (m for help): n Command action e extended p primary partition (1-4) e Partition number (1-4): 4 First cylinder (1-6527, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-6527, default 6527): Using default value 6527 Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended Command (m for help): n Command action l logical (5 or over) p primary partition (1-4) l First cylinder (1-6527, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-6527, default 6527): +3G Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended /dev/xvdb5 1 393 3156709+ 83 Linux Command (m for help): n Command action l logical (5 or over) p primary partition (1-4) l First cylinder (394-6527, default 394): Using default value 394 Last cylinder, +cylinders or +size{K,M,G} (394-6527, default 6527): +3G Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended /dev/xvdb5 1 393 3156709+ 83 Linux /dev/xvdb6 394 786 3156741 83 Linux Command (m for help): t Partition number (1-6): 5 Hex code (type L to list codes): l 0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris 1 FAT12 39 Plan 9 82 Linux swap / So c1 DRDOS/sec (FAT- 2 XENIX root 3c PartitionMagic 83 Linux c4 DRDOS/sec (FAT- 3 XENIX usr 40 Venix 80286 84 OS/2 hidden C: c6 DRDOS/sec (FAT- 4 FAT16 <32M 41 PPC PReP Boot 85 Linux extended c7 Syrinx 5 Extended 42 SFS 86 NTFS volume set da Non-FS data 6 FAT16 4d QNX4.x 87 NTFS volume set db CP/M / CTOS / . 7 HPFS/NTFS 4e QNX4.x 2nd part 88 Linux plaintext de Dell Utility 8 AIX 4f QNX4.x 3rd part 8e Linux LVM df BootIt 9 AIX bootable 50 OnTrack DM 93 Amoeba e1 DOS access a OS/2 Boot Manag 51 OnTrack DM6 Aux 94 Amoeba BBT e3 DOS R/O b W95 FAT32 52 CP/M 9f BSD/OS e4 SpeedStor c W95 FAT32 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi eb BeOS fs e W95 FAT16 (LBA) 54 OnTrackDM6 a5 FreeBSD ee GPT f W95 Ext'd (LBA) 55 EZ-Drive a6 OpenBSD ef EFI (FAT-12/16/ 10 OPUS 56 Golden Bow a7 NeXTSTEP f0 Linux/PA-RISC b 11 Hidden FAT12 5c Priam Edisk a8 Darwin UFS f1 SpeedStor 12 Compaq diagnost 61 SpeedStor a9 NetBSD f4 SpeedStor 14 Hidden FAT16 <3 63 GNU HURD or Sys ab Darwin boot f2 DOS secondary 16 Hidden FAT16 64 Novell Netware af HFS / HFS+ fb VMware VMFS 17 Hidden HPFS/NTF 65 Novell Netware b7 BSDI fs fc VMware VMKCORE 18 AST SmartSleep 70 DiskSecure Mult b8 BSDI swap fd Linux raid auto 1b Hidden W95 FAT3 75 PC/IX bb Boot Wizard hid fe LANstep 1c Hidden W95 FAT3 80 Old Minix be Solaris boot ff BBT 1e Hidden W95 FAT1 Hex code (type L to list codes): fd Changed system type of partition 5 to fd (Linux raid autodetect) Command (m for help): t Partition number (1-6): 6 Hex code (type L to list codes): fd Changed system type of partition 6 to fd (Linux raid autodetect) Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended /dev/xvdb5 1 393 3156709+ fd Linux raid autodetect /dev/xvdb6 394 786 3156741 fd Linux raid autodetect
确认一下分区已被内核识别
root@VM_160_34_centos:~> cat /proc/partitions major minor #blocks name 202 0 8388608 xvda 202 1 8385898 xvda1 202 16 52428800 xvdb 202 20 1 xvdb4 202 21 3156709 xvdb5 202 22 3156741 xvdb6 202 32 2097152 xvdc
执行创建命令
root@VM_160_34_centos:~> mdadm -C /dev/md0 -a yes -l 0 -n 2 /dev/xvdb{5,6} mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started.
我们 看一下 /proc/mdstat 文件, 这个文件里面记录着系统中所有启动在的raid 设备
root@VM_160_34_centos:~> cat /proc/mdstat Personalities : [raid0] md0 : active raid0 xvdb6[1] xvdb5[0] 6310912 blocks super 1.2 512k chunks unused devices:
我们把它们格式化,
root@VM_160_34_centos:~> mke2fs -j /dev/md0 mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=128 blocks, Stripe width=256 blocks 395136 inodes, 1577728 blocks 78886 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1619001344 49 block groups 32768 blocks per group, 32768 fragments per group 8064 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 37 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
然后挂载到mnt下
root@VM_160_34_centos:~> mount /dev/md0 /mnt/ root@VM_160_34_centos:~> ll /mnt/ total 16 drwx------ 2 root root 16384 Nov 21 15:30 lost+found
到此这个 raid 成功可以使用
创建1G的 raid1
我们首先创建2个1G的分区,并调整它的类型
root@VM_160_34_centos:~> fdisk /dev/xvdb WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended /dev/xvdb5 1 393 3156709+ fd Linux raid autodetect /dev/xvdb6 394 786 3156741 fd Linux raid autodetect Command (m for help): n Command action l logical (5 or over) p primary partition (1-4) l First cylinder (787-6527, default 787): Using default value 787 Last cylinder, +cylinders or +size{K,M,G} (787-6527, default 6527): +1G Command (m for help): n Command action l logical (5 or over) p primary partition (1-4) l First cylinder (919-6527, default 919): Using default value 919 Last cylinder, +cylinders or +size{K,M,G} (919-6527, default 6527): +1G Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended /dev/xvdb5 1 393 3156709+ fd Linux raid autodetect /dev/xvdb6 394 786 3156741 fd Linux raid autodetect /dev/xvdb7 787 918 1060258+ 83 Linux /dev/xvdb8 919 1050 1060258+ 83 Linux Command (m for help): t Partition number (1-8): 7 Hex code (type L to list codes): fd Changed system type of partition 7 to fd (Linux raid autodetect) Command (m for help): t Partition number (1-8): 8 Hex code (type L to list codes): fd Changed system type of partition 8 to fd (Linux raid autodetect) Command (m for help): p Disk /dev/xvdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc77f92d7 Device Boot Start End Blocks Id System /dev/xvdb4 1 6527 52428096 5 Extended /dev/xvdb5 1 393 3156709+ fd Linux raid autodetect /dev/xvdb6 394 786 3156741 fd Linux raid autodetect /dev/xvdb7 787 918 1060258+ fd Linux raid autodetect /dev/xvdb8 919 1050 1060258+ fd Linux raid autodetect Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks.
确保内核已经识别了我们的分区,如果没有执行 partx -a /dev/xvdb 来识别(可能需要umount 该盘其他挂载点) ,
root@VM_160_34_centos:~> mdadm -C /dev/md1 -a yes -l 1 -n 2 /dev/xvdb{7,8} mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90
执行创建命令
root@VM_160_34_centos:~> mdadm -C /dev/md1 -a yes -l 1 -n 2 /dev/xvdb{7,8} mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started.
执行的时候它会问你是否需要创建, 这个是毫无疑问的.
我们查看一下它的状态
root@VM_160_34_centos:~> cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 xvdb8[1] xvdb7[0] 1059222 blocks super 1.2 [2/2] [UU] [==========>..........] resync = 54.6% (579524/1059222) finish=0.0min speed=144881K/sec md0 : active raid0 xvdb6[1] xvdb5[0] 6310912 blocks super 1.2 512k chunks unused devices:
格式化,
root@VM_160_34_centos:~> mke2fs -j /dev/md1 mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 66240 inodes, 264805 blocks 13240 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=272629760 9 block groups 32768 blocks per group, 32768 fragments per group 7360 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Writing inode tables: done Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 25 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
挂载
root@VM_160_34_centos:~> mount /dev/md1 /storage/ root@VM_160_34_centos:~> ll /storage/ total 16 drwx------ 2 root root 16384 Nov 21 15:51 lost+found
我们查看一下这个阵列的属性:
root@VM_160_34_centos:~> mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Nov 21 15:48:21 2014 Raid Level : raid1 Array Size : 1059222 (1034.57 MiB 1084.64 MB) Used Dev Size : 1059222 (1034.57 MiB 1084.64 MB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Nov 21 15:53:44 2014 State : clean Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Name : VM_160_34_centos:1 (local to host VM_160_34_centos) UUID : 4e1ea263:6297d669:9d5877af:6e5fc1e0 Events : 17 Number Major Minor RaidDevice State 0 202 23 0 active sync /dev/xvdb7 1 202 24 1 active sync /dev/xvdb8
然后拷贝个文件进去,模拟损坏一个设备,文件照常访问
root@VM_160_34_centos:~> mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Nov 21 15:48:21 2014 Raid Level : raid1 Array Size : 1059222 (1034.57 MiB 1084.64 MB) Used Dev Size : 1059222 (1034.57 MiB 1084.64 MB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Nov 21 15:57:29 2014 State : clean Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Name : VM_160_34_centos:1 (local to host VM_160_34_centos) UUID : 4e1ea263:6297d669:9d5877af:6e5fc1e0 Events : 17 Number Major Minor RaidDevice State 0 202 23 0 active sync /dev/xvdb7 1 202 24 1 active sync /dev/xvdb8 root@VM_160_34_centos:~> mdadm /dev/md1 -f /dev/xvdb8 mdadm: set /dev/xvdb8 faulty in /dev/md1 root@VM_160_34_centos:~> mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Nov 21 15:48:21 2014 Raid Level : raid1 Array Size : 1059222 (1034.57 MiB 1084.64 MB) Used Dev Size : 1059222 (1034.57 MiB 1084.64 MB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Nov 21 15:59:05 2014 State : clean, degraded Active Devices : 1 Working Devices : 1 Failed Devices : 1 Spare Devices : 0 Name : VM_160_34_centos:1 (local to host VM_160_34_centos) UUID : 4e1ea263:6297d669:9d5877af:6e5fc1e0 Events : 18 Number Major Minor RaidDevice State 0 202 23 0 active sync /dev/xvdb7 1 0 0 1 removed 1 202 24 - faulty spare /dev/xvdb8 root@VM_160_34_centos:~> cat /storage/passwd root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin sync:x:5:0:sync:/sbin:/bin/sync shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown halt:x:7:0:halt:/sbin:/sbin/halt mail:x:8:12:mail:/var/spool/mail:/sbin/nologin uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin operator:x:11:0:operator:/root:/sbin/nologin games:x:12:100:games:/usr/games:/sbin/nologin gopher:x:13:30:gopher:/var/gopher:/sbin/nologin ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin nobody:x:99:99:Nobody:/:/sbin/nologin dbus:x:81:81:System message bus:/:/sbin/nologin vcsa:x:69:69:virtual console memory owner:/dev:/sbin/nologin abrt:x:173:173::/etc/abrt:/sbin/nologin haldaemon:x:68:68:HAL daemon:/:/sbin/nologin ntp:x:38:38::/etc/ntp:/sbin/nologin saslauth:x:499:76:"Saslauthd user":/var/empty/saslauth:/sbin/nologin postfix:x:89:89::/var/spool/postfix:/sbin/nologin sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin tcpdump:x:72:72::/:/sbin/nologin nscd:x:28:28:NSCD Daemon:/:/sbin/nologin nslcd:x:65:55:LDAP Client User:/:/sbin/nologin dd:x:500:500::/home/dd:/bin/bash mysql:x:501:501::/home/mysql:/sbin/nologin vsftpd:x:502:502::/home/vsftpd:/sbin/nologin www:x:503:503::/storage/www:/sbin/nologin hadoop:x:504:504::/home/hadoop:/bin/bash
我们替换1一个好的设备进去
root@VM_160_34_centos:~> mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Nov 21 15:48:21 2014 Raid Level : raid1 Array Size : 1059222 (1034.57 MiB 1084.64 MB) Used Dev Size : 1059222 (1034.57 MiB 1084.64 MB) Raid Devices : 2 Total Devices : 1 Persistence : Superblock is persistent Update Time : Fri Nov 21 16:01:42 2014 State : clean, degraded Active Devices : 1 Working Devices : 1 Failed Devices : 0 Spare Devices : 0 Name : VM_160_34_centos:1 (local to host VM_160_34_centos) UUID : 4e1ea263:6297d669:9d5877af:6e5fc1e0 Events : 25 Number Major Minor RaidDevice State 0 202 23 0 active sync /dev/xvdb7 1 0 0 1 removed
添加 xvdb 9
root@VM_160_34_centos:~> mdadm /dev/md1 -a /dev/xvdb9 mdadm: added /dev/xvdb9 root@VM_160_34_centos:~> mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Nov 21 15:48:21 2014 Raid Level : raid1 Array Size : 1059222 (1034.57 MiB 1084.64 MB) Used Dev Size : 1059222 (1034.57 MiB 1084.64 MB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Nov 21 16:04:46 2014 State : clean Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Name : VM_160_34_centos:1 (local to host VM_160_34_centos) UUID : 4e1ea263:6297d669:9d5877af:6e5fc1e0 Events : 46 Number Major Minor RaidDevice State 0 202 23 0 active sync /dev/xvdb7 2 202 25 1 active sync /dev/xvdb9
如果我们再 raid 中添加了3个设备, 那么空余那个会在其中一个设备坏了的之后制动替换掉坏掉的那个.
为了方便以后装载,我们把配置写入配置文件
将当前RAID信息保存至配置文件，以便以后进行装配：
mdamd -D --scan > /etc/mdadm.conf
创建一个 2G的 raid5(只要有冗余的都支持空闲盘,所以我们需要4个1G的设备,跟raid1的用法是一样的)