本文主要翻译SkySQL 中的文章,详细讲述了如何恢复InnoDB独立表空间数据;
原文地址:http://www.chriscalender.com/?p=28
有时候需要只有在.ibd文件的情况下恢复数据。我们尽力将它load到新的实例或者其他实例当中,可能会遇到 table id 的错误。
我这里有两种方式来恢复单个ibd数据。
前提是:你需要.ibd的文件,和对应该表的 CREATE TABLE 语句。
第一种:模拟Innodb internal table id计数方式。启用innodb_file_per_table,建立work table,直至internal table id 等于 (要恢复表的table id -1)
第二种:手动修改16进制的.ibd文件来改变table id
最后,由于innodb元数据的中内部结构,我们需要对dump import 我们已经恢复的表。
方法1、创建 work table.
1、新建立一个MySQL 实例,并启用innodb_file_per_table
2、找到work table 在新实例中的table id,和需要恢复表的table id,
note:
对于第二步(2a--2f)详细过程是找出实例中各个.ibd文件对应的table id,我已经写了php 脚本来做这个事情,
2a:创建测试库:
mysql> CREATE DATABASE test1; mysql> USE test1;
2b:在test1中创建对应.ibd(需要恢复的表) 文件的数据表
mysql> CREATE TABLE `product` ( `PRODUCT_ID` bigint(20) unsigned NOT NULL auto_increment, `BRAND_ID` int(10) unsigned default NULL, `PRODUCT_TYPE_ID` int(10) unsigned default NULL, `GROUP_ID` int(10) unsigned default NULL, `PRODUCT_NAME` varchar(500) NOT NULL, `DEFAULT_EMAIL_ID` varchar(48) default NULL, `PRODUCT_STATUS` tinyint(1) NOT NULL, `CLIENT_ID` bigint(20) unsigned default NULL, `LAST_MODIFIED_BY` varchar(45) NOT NULL, `LAST_MODIFIED_DATE` datetime NOT NULL, PRIMARY KEY (`PRODUCT_ID`) ) ENGINE=InnoDB;
2c:删除表空间
mysql> ALTER TABLE product DISCARD TABLESPACE;
2d: 拷贝原先的.ibd 文件(需要恢复的表的数据文件)到 test1的数据库目录下
2e:导入表空间
mysql> ALTER TABLE product IMPORT TABLESPACE;
该步骤通常情况下会报错,除非导入的表空间对应的table id 等于新建表的table id
报错信息;
ERROR 1030 (HY000): Got error -1 from storage engine
2f:检查error.log ,我们能够找到该.ibd文件的 table id
081010 11:47:40 InnoDB: Error: tablespace id in file '.test1product.ibd' is 1193, but in the InnoDB InnoDB: data dictionary it is 1
我们知道 internal table id 是1,.ibd file对应的 table id 是1193
3、清空 test1
3a、手动移动 .ibd 文件到 到其他位置(一会儿还会需要)
3b、删除表
mysql> DROP TABLE product;
这个步骤不会重设 internal table 计数器
4、创建相应数量的表来使 internal table id 的值增加
在这个案例中,我们需要在 test1中创建 1191个 innodb table,(table id 1已经被占用,需要比对应.ibd 文件的table id 小1, 所以是 1193-2=1191 )
执行下面的程序:
for ($1=1; $i<=1191; $1++) { CREATE TABLE t# (id int) ENGINE=InnoDB; }
注:我已经用Php 程序搞定
5、完成以上步骤后,删除所有的db 和 table。
drop database test1;
6、重新执行步骤 2a-2e
mysql> CREATE DATABASE test1; mysql> USE test1; mysql> CREATE TABLE `product` ( ... ) ENGINE=InnoDB; mysql> ALTER TABLE product DISCARD TABLESPACE;
拷贝对应的.ibd file 到对应的 test1数据库目录
mysql> ALTER TABLE product IMPORT TABLESPACE;
Success!
mysql> show tables; +-----------------+ | Tables_in_test1 | +-----------------+ | product | +-----------------+ 1 row in set (0.00 sec)
7、用mysqldump来备份该表(这一步必须执行),然后可以在任何一个实例中进行恢复。
以上的情况常常发生在数据库 crash 或者表空间损坏的情况下。
如果发生以上 情况,先尝试force innodb recovery 并dump出数据。 从 1开始innodb_force_recovery=1 (and try 2,3,4,5,6) 直到能够dump出数据。
对于以上的例子,我是设置 innodb_force_recovery=5 来解决问题。
以下是我的操作记录:
C:Program FilesMySQLmysql-5.0.68bin> mysqldump -uroot -P3385 test1 product > product_dump1.txt mysqldump: Couldn't execute 'show table status like 'product'': Lost connection to MySQL server during query (2013) C:Program FilesMySQLmysql-5.0.68bin> mysqldump -uroot -P3385 test1 product > product_dump2.txt mysqldump: Couldn't execute 'show table status like 'product'': Lost connection to MySQL server during query (2013) C:Program FilesMySQLmysql-5.0.68bin> mysqldump -uroot -P3385 test1 product > product_dump3.txt mysqldump: Couldn't execute 'show table status like 'product'': Lost connection to MySQL server during query (2013) C:Program FilesMySQLmysql-5.0.68bin> mysqldump -uroot -P3385 test1 product > product_dump4.txt mysqldump: Couldn't execute 'SELECT /*!40001 SQL_NO_CACHE */ * FROM `product`': Lost connection to MySQL server during query (2013) C:Program FilesMySQLmysql-5.0.68bin> mysqldump -uroot -P3385 test1 product > product_dump5.txt C:Program FilesMySQLmysql-5.0.68bin> mysqladmin -u root -P 3385 shutdown C:Program FilesMySQLmysql-5.0.68bin> mysqldump -uroot -P3385 test1 product > product_dump6.txt
设置为5的原因是由于以下error中的信息:
InnoDB: Error: trying to access update undo rec field 19 in index PRIMARY of table test1/product InnoDB: but index has only 12 fields
以上是 undo log中的信息,文档中设置为5的解释是:
"Do not look at undo logs when starting the database: InnoDB treats even incomplete transactions as committed"
方法二 修改.ibd file
在此之前先备份数据(ibdata file,ib_logfile, data)
按照下面的 1-5步来 操作:
http://dev.mysql.com/doc/refman/5.0/en/adding-and-removing.html
Let me post them here for completeness, however:
重复 2a-2f的过程,来获得 该.ibd file的table id,
在windows上使用 Freeware Hex Editor XVI32 (http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm)
以下是修改部分:
For me, and I assume it should be the same for you, but just look at the values to be sure, I see the tablespace id values listed at position 37 and 41 (positions 25 and 29 in hex). In the actual hex column, if you're previous tablespace id was 2, then in positions 37 and 41, you'd see 02 and 02.
(Note these positions can change. For instance, I tested on a table with an internal id of 1193. This in hex is 04A9. However, when searching the file, for the first instance of the table id, I found the '04' in position 39 and 'A9' in position 40. Then, for the second instance of the table id, the '04' was at position 43 and the 'A9' was at position 44. So, you'll have to convert the table id to hex, and then search for that value, near the beginning of the file.)
Note that this value (02) may vary depending on what your actual tablespace id is.
Then, simply modify both of those fields to 01, and save the file.
再执行下面的部分:
1. ALTER TABLE tbl_name DISCARD TABLESPACE; 2. Put the newly saved .ibd file back in the proper database directory 3. ALTER TABLE tbl_name IMPORT TABLESPACE;
以下是涉及到的 php 脚本:
$dbhost = "localhost:3385"; $dbname = "test1"; $dbuser = "root"; $dbpwd = ""; mysql_connect($dbhost,$dbuser,$dbpwd) or die(mysql_error()); for ($i = 1033; $i <= 1190; $i++) { $dbquery = "CREATE TABLE test1.t" . $i . " (id int) ENGINE=InnoDB"; echo "" . $dbquery . ""; $result = mysql_db_query($dbname,$dbquery) or die(mysql_error()); $j = 0; while($row = mysql_fetch_array($result)) { $j++; echo $row[0]; } } mysql_close();
PHP Internal Table ID Finder - Used to determine the internal Table ID from the binary .ibd file:
/* Tested with tables from 4.1.23, 5.0.68, 5.1.28, and 6.0.7. */ // Set the filename $filename = "C:\Users\Chris\Desktop\mysql\working\ibds\z1.ibd"; // Read 2 bytes in at a time $offset = 2; // Echo filename and path echo "filename = $filename "; // Open the filename - need 'rb' for binary file on Windows $handle = fopen($filename, "rb"); // Define redundant, local variables for possible later functionality and/or checks $ibd_id_bin = 0; $ibd_id_hex = 0; $ibd_id_dec = 0; $ibd_id_bin2 = 0; $ibd_id_hex2 = 0; $ibd_id_dec2 = 0; // Find the filesize (note: below command messes up script) //$filesize = filesize($filename)); // Only loop through first 21 bytes - as table is is in $array[18] and $array[20] for ($z = 0; $z <= 20; $z++) { // Set variable $contents equal to 2 ($offset) bytes of binary data $contents = fread($handle, $offset); // Convert $contents from binary data to hex data $contents2 = bin2hex($contents); // Convert $contents2 from hex data to decimal data $contents3 = hexdec($contents2); // Debug Output //echo "contents[$z] = " . $contents . ""; //echo "contents2[$z] = " . $contents2 . " "; //echo "contents3[$z] = " . $contents3 . " "; // If position 19, array position [18], then store the values if ($z == 18) { $ibd_id_bin = $contents; $ibd_id_hex = $contents2; $ibd_id_dec = $contents3; } // If position 21, array position [20], then store the values if ($z == 20) { $ibd_id_bin2 = $contents; $ibd_id_hex2 = $contents2; $ibd_id_dec2 = $contents3; } } fclose($handle); // More Debug output //echo " The table id is $ibd_id_dec "; //echo " The table id is $ibd_id_dec2 "; // Check to see if both values are equal. If so, then it's // most certain this is the correct value. // If not, then there's a chance the positions are off for // this table (due to versions, etc.). if ($ibd_id_dec == $ibd_id_dec2) { echo " The table id is $ibd_id_dec "; } else { echo "The values from positions [18] and [20] did not match,"; echo "so please enable debug output, and check for proper positions."; }