数据备份类型:
1、完全备份和部分备份(库或表)
2、完全备份、增量备份、差异备份
增量备份和差异备份区别在于:增量备份只会备份最近一次完全备份或增量备份(存在增量备份)后的变化的数据,而差异备份会备份完全备份以来的所有变化数据
3、热备、温备、冷备
热备:数据库正常运行时的备份,
温备:数据施加读锁后进行的备份
冷备:数据库完全停止后备份
4、物理备份、逻辑备份
物理备份:直接复制数据文件,从而实现备份
逻辑备份:从数据库中“导出”数据(sql语句)从而进行的备份(与存储引擎无关)
备份需要考虑因素:
1、当温备时,施加锁的时长
2、备份过程所用的时长
3、在线备份时系统的负载
4、恢复时过程所用的时长
备份时所考虑的内容:
1、数据
2、二进制日志,InnoDB事务日志
3、数据库的存储过程,存储函数,触发器,事件调度器
4、服务器的配置文件。这点容易被忽略,试想,服务器挂掉,而又记不起当初的配置参数,可能也会导致还原数据时出现故障。
备份策略:
1、时间:每天,每周,还是每小时
2、备份类型:完全备份+增量还是完全+差异
3、手段:物理备份,逻辑备份
4、备份工具(会受备份类型和手段限制):常见的有mysqldump,LVM快照,xtrabackup
mysqldump备份方式,但是只应用于下于G级别的备份方式
只备份库中一个表: [root@mariadb tmp]# mysqldump -uroot mysql db >/tmp/db.table #上面备份的方式不会create表,导入时,需要另外创建表 [root@mariadb tmp]# less db.table -- MySQL dump 10.14 Distrib 5.5.41-MariaDB, for Linux (x86_64) -- -- Host: localhost Database: mysql -- ------------------------------------------------------ -- Server version 5.5.41-MariaDB-log /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; /*!40101 SET NAMES utf8 */; /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */; /*!40103 SET TIME_ZONE='+00:00' */; /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */; /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */; /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */; /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */; -- -- Table structure for table `db` -- DROP TABLE IF EXISTS `db`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `db` ( `Host` char(60) COLLATE utf8_bin NOT NULL DEFAULT '', `Db` char(64) COLLATE utf8_bin NOT NULL DEFAULT '', `User` char(16) COLLATE utf8_bin NOT NULL DEFAULT '', `Select_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Insert_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Update_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Delete_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Create_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Drop_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Grant_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `References_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Index_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Alter_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Create_tmp_table_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Lock_tables_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Create_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Show_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Create_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Alter_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Execute_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Event_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', `Trigger_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N', PRIMARY KEY (`Host`,`Db`,`User`), KEY `User` (`User`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Database privileges'; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Dumping data for table `db` -- LOCK TABLES `db` WRITE; /*!40000 ALTER TABLE `db` DISABLE KEYS */; INSERT INTO `db` VALUES ('%','test','','Y','Y','Y','Y','Y','Y','N','Y','Y','Y','Y','Y','Y','Y','Y','N','N','Y','Y'),('%','test\\_%','','Y','Y','Y','Y','Y','Y','N' ,'Y','Y','Y','Y','Y','Y','Y','Y','N','N','Y','Y'); /*!40000 ALTER TABLE `db` ENABLE KEYS */; UNLOCK TABLES; /*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */; /*!40101 SET SQL_MODE=@OLD_SQL_MODE */; /*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */; /*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */; /*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */; /*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */; /*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */; /*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */; -- Dump completed on 2015-11-08 1:10:23 (END) ##只备份一个库 [root@mariadb tmp]# mysqldump -uroot --databases mysql >/tmp/user.db ##同时备份多个库 [root@mariadb tmp]# mysqldump -uroot --databases hellodb mysql >/tmp/bak.db ##备份所有库 [root@mariadb tmp]# mysqldump -uroot --all-databases >/tmp/all.db ##但是上面的方式备份,只能用于没有其他用户连接且没有数据处理时
在线备份时,需要施加锁,
如MyISAM只支持温备,施加读锁定方式:
--lock-tables:锁定单个数据库,备份某表
--lock-all-tables:锁定所有库的所有表
InnoDB也支持上述方式,但是还支持热备,需要施加事务锁
--single-transaction:施加单个事务锁
其他需要使用的参数
--master-data=2:备份时,记录二进制日志事件位置
-E,--events:备份指定数据库相关的所有event scheduler
-R,--routines:备份指定数据库相关的所有存储过程和存储函数
--triggers:备份表相关的触发器
--flush-logs:锁定表完成后,执行flush logs命令,实现日志滚动
常用到备份方式为:
[root@mariadb tmp]# mysqldump --all-databases --flush-logs --master-data=2 --lock-all-tables > all.sql
[root@mariadb tmp]# less all.sql #可以看到加了--master-data=2参数后,记录的binlog文件以及事件的位置
-- Position to start replication or point-in-time recovery from
--
-- CHANGE MASTER TO MASTER_LOG_FILE='bin-log.000004', MASTER_LOG_POS=245;
还原方式:
[root@centos7-4 tmp]# mysql < all.sql #这样导入会产生二进制日志
或者:MariaDB [(none)]> source /tmp/all.sql #这样可以先关闭二进制日志,再导入不会产生二进制日志
通过二进制日志实现即时点还原:
通过完全备份方式,已经还原到了备份节点数据库上了,现在模拟实时更新数据库的数据,如在数据中插入一行数据,在完全还原的数据库中,还原此条数据 MariaDB [hellodb]> INSERT INTO students (Name,Age,Gender) VALUES ("xiaowang",28,"M"); 查看使用的是哪个二进制日志 MariaDB [hellodb]> show master status; #如果与上一次完全备份不在同一个binlog上,需要分批在备份主机上还原 +----------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +----------------+----------+--------------+------------------+ | bin-log.000004 | 546 | | | +----------------+----------+--------------+------------------+ scp 到备份节点数据库上,在备份节点上,把其完全备份的事件位置后的二进制导出来 如上,已经查看了完全备份时,使用的二进制日志已经事件位置 CHANGE MASTER TO MASTER_LOG_FILE='bin-log.000004', MASTER_LOG_POS=245; [root@centos7-4 tmp]# mysqlbinlog --start-position=245 /tmp/bin-log.000004 >new.sql 即时点还原 MariaDB [(none)]> SET sql_log_bin=off MariaDB [(none)]> source /tmp/new.sql; MariaDB [hellodb]> SET sql_log_bin=on; MariaDB [hellodb]> select * from students; MariaDB [hellodb]> select * from students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | | 26 | xiaowang | 28 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 26 rows in set (0.01 sec) 小王依然恢复到了备份节点上
基于lvm备份
创建pv [root@mariadb ~]# pvcreate /dev/sda7 Physical volume "/dev/sda7" successfully created 创建vg [root@mariadb ~]# vgcreate testvg /dev/sda7 Volume group "testvg" successfully created 创建逻辑卷 [root@mariadb ~]# lvcreate -L 5G -n mydata testvg Logical volume "mydata" created. [root@mariadb ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert mydata testvg -wi-a----- 5.00g 格式化文件系统 [root@mariadb ~]# mkfs -t ext4 /dev/testvg/mydata [root@mariadb ~]# mount /dev/testvg/mydata /data/mydata/ [root@mariadb ~]# systemctl start mairadb MariaDB [(none)]> SET sql_log_bin=off MariaDB [(none)]> source /tmp/all.sql MariaDB [hellodb]> SET sql_log_bin=on; 请求施加全局锁,滚动日志,以及记录事件位置 MariaDB [hellodb]> FLUSH TABLES WITH READ LOCK; [root@mariadb ~]# mysql -e "FLUSH LOGS;" [root@mariadb ~]# mysql -e "SHOW MASTER STATUS;" >/root/pos.txt [root@mariadb ~]# cat pos.txt File Position Binlog_Do_DB Binlog_Ignore_DB bin-log.000004 245 创建快照卷 [root@mariadb ~]# lvcreate -L 2G -s -n mydata-snap -p r /dev/testvg/mydata 释放锁 MariaDB [hellodb]> UNLOCK TABLES; 备份快照数据,并保存到其他地方,这里是保存到/tmp/mydata下 [root@mariadb mydata]# mount /dev/testvg/mydata-snap /mnt [root@mariadb mydata]# cd /mnt [root@mariadb mnt]# ls aria_log.00000001 aria_log_control hellodb ibdata1 ib_logfile0 ib_logfile1 mysql performance_schema test [root@mariadb mnt]# cp -a * /tmp/mydata/ 删除快照卷 [root@mariadb ~]# umount /mnt/ [root@mariadb ~]# lvremove /dev/testvg/mydata-snap 插入新数据 MariaDB [hellodb]> INSERT INTO students (Name,Age,Gender) VALUES ("GBlaowang",28,"M"); 假设此时数据库崩溃了,数据文件丢失了,但是二进制日志是完好的 systemct stop mariadb.service rm -rf /data/mydata/* 首先要做的是: 1、把二进制日志保存出来 mv -a /data/binlogs/ /tmp/ 2、把备份文件拷贝到mariadb的datadir下,启动数据库 cp -a /tmp/mydata/* /data/mydata/ systemct start mariadb.service 3、快照前的数据依然恢复。但是快照后所插入的数据丢失,则需要通过二进制日志找回 [root@mariadb ~]# cat pos.txt File Position Binlog_Do_DB Binlog_Ignore_DB bin-log.000004 245 4、在二进制日志把245后发生的数据找到。 [root@mariadb binlogs]# ls bin-log.000001 bin-log.000002 bin-log.000003 bin-log.000004 bin-log.index [root@mariadb binlogs]# mysqlbinlog --start-position=245 bin-log.000004 > new1.sql 5、即时点还原 MariaDB [hellodb]> SET sql_log_bin=off; MariaDB [hellodb]> source /tmp/binlogs/new1.sql; MariaDB [hellodb]> SET sql_log_bin=on; 找到快照后数据 MariaDB [hellodb]> select * from students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | | 26 | xiaowang | 28 | M | NULL | NULL | | 27 | GBlaowang | 28 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 27 rows in set (0.01 sec)
基于xtrabackup
备份innodb热备的前提:是每表都有自己的表空间文件 innodb_file_per_table = on
完全备份 [root@mariadb ~]# innobackupex --user=root /tmp/ 生成的文件 [root@mariadb ~]# ls /tmp/2015-11-07_23-26-21/ backup-my.cnf hellodb ibdata1 mysql performance_schema test xtrabackup_binlog_info xtrabackup_checkpoints xtrabackup_info xtrabackup_logfile 复制到备份主机上 scp -r /tmp/2015-11-07_23-26-21 10.0.0.41:/tmp/ 整理备份文件 [root@centos7-4 tmp]# innobackupex --apply-log 2015-11-07_23-26-21/ 停止mariadb [root@centos7-4 tmp]# systemctl stop mariadb.service 删除mariadb的数据文件 [root@centos7-4 tmp]# rm -rf /data/mydata/* 还原数据 [root@centos7-4 tmp]# innobackupex --copy-back /tmp/2015-11-07_23-26-21 [root@centos7-4 tmp]# ls /data/mydata/ hellodb ibdata1 ib_logfile0 ib_logfile1 mysql performance_schema test xtrabackup_binlog_pos_innodb xtrabackup_info [root@centos7-4 tmp]# chown -R mysql.mysql mydata/ [root@centos7-4 tmp]# rm -rf /data/mydata/ib_logfile* #拷贝中日志大于数据库设定的值 数据完整 MariaDB [hellodb]> select * from students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | | 26 | xiaowang | 28 | M | NULL | NULL | | 27 | GBlaowang | 28 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 27 rows in set (0.01 sec) 增量备份 先删除掉备份主机上的数据库数据,再通过完全备份和增量备份还原,以及二进制日志还原 插入一些数据 MariaDB [hellodb]> use hellodb; MariaDB [hellodb]> CREATE TABLE test (id int); MariaDB [hellodb]> INSERT INTO test VALUES (1),(2),(3); MariaDB [hellodb]> select * from test; +------+ | id | +------+ | 1 | | 2 | | 3 | +------+ 备份增量 [root@mariadb ~]# innobackupex --incremental /tmp/ --incremental-basedir=/tmp/2015-11-07_23-26-21 把完全和增量全部copy到备份主机上 [root@mariadb tmp]# scp -r 2015-11-07_23-26-21/ 2015-11-08_00-22-07/ 10.0.0.41:/tmp/ 在备份主机上整理完全备份数据和增量数据 [root@centos7-4 tmp]# innobackupex --apply-log --redo-only 2015-11-07_23-26-21/ [root@centos7-4 tmp]# innobackupex --apply-log --redo-only 2015-11-07_23-26-21/ --incremental-dir=/tmp/2015-11-08_00-22-07/ 还原数据 [root@centos7-4 tmp]# innobackupex --copy-back /tmp/2015-11-07_23-26-21/ [root@centos7-4 tmp]# chown -R mysql.mysql /data/mydata/ 启动数据查看结果 [root@centos7-4 ~]# systemctl start mariadb.service MariaDB [hellodb]> select * from test; +------+ | id | +------+ | 1 | | 2 | | 3 | +------+ 3 rows in set (0.01 sec) 再通过二进制日志即时点还原 先在主数据主机上,的test表中插入一些数据 MariaDB [hellodb]> use hellodb; MariaDB [hellodb]> CREATE TABLE test (id int); MariaDB [hellodb]> INSERT INTO test VALUES (4),(5),(6); MariaDB [hellodb]> select * from test; +------+ | id | +------+ | 1 | | 2 | | 3 | | 4 | | 5 | | 6 | +------+ 通过查看增量备份日志可以看到增量备份时事务位置 [root@centos7-4 2015-11-08_00-22-07]# cat xtrabackup_binlog_info bin-log.000004 1051 把1051后新增的二进制日志导出 [root@mariadb tmp]# mysqlbinlog --start-position=1051 /data/binlogs/bin-log.000004 >new2.sql 把二进制日志拷贝到备份主机上 [root@mariadb tmp]# scp new2.sql 10.0.0.41:/root/ 在备注主机上做即时点还原 MariaDB [hellodb]> SET sql_log_bin=off; MariaDB [hellodb]> source /root/new2.sql; MariaDB [hellodb]> SET sql_log_bin=on; MariaDB [hellodb]> select * from test; +------+ | id | +------+ | 1 | | 2 | | 3 | | 4 | | 5 | | 6 | +------+