恢复InnoDB独立表空间数据

     本文主要翻译SkySQL 中的文章，详细讲述了如何恢复InnoDB独立表空间数据；

     原文地址：http://www.chriscalender.com/?p=28

     有时候需要只有在.ibd文件的情况下恢复数据。我们尽力将它load到新的实例或者其他实例当中，可能会遇到 table id 的错误。

     我这里有两种方式来恢复单个ibd数据。

     前提是：你需要.ibd的文件，和对应该表的 CREATE TABLE 语句。

     第一种：模拟Innodb internal  table id计数方式。启用innodb_file_per_table，建立work  table，直至internal table id 等于 （要恢复表的table id  -1）

     第二种：手动修改16进制的.ibd文件来改变table  id

     最后，由于innodb元数据的中内部结构，我们需要对dump import 我们已经恢复的表。

     方法1、创建 work table.

     1、新建立一个MySQL 实例，并启用innodb_file_per_table

     2、找到work table 在新实例中的table id,和需要恢复表的table id,

     note:

     对于第二步（2a--2f）详细过程是找出实例中各个.ibd文件对应的table id，我已经写了php 脚本来做这个事情，

     2a：创建测试库：

mysql> CREATE DATABASE test1;
mysql> USE test1;

     2b:在test1中创建对应.ibd（需要恢复的表） 文件的数据表

mysql> CREATE TABLE `product` (
  `PRODUCT_ID` bigint(20) unsigned NOT NULL auto_increment,
  `BRAND_ID` int(10) unsigned default NULL,
  `PRODUCT_TYPE_ID` int(10) unsigned default NULL,
  `GROUP_ID` int(10) unsigned default NULL,
  `PRODUCT_NAME` varchar(500) NOT NULL,
  `DEFAULT_EMAIL_ID` varchar(48) default NULL,
  `PRODUCT_STATUS` tinyint(1) NOT NULL,
  `CLIENT_ID` bigint(20) unsigned default NULL,
  `LAST_MODIFIED_BY` varchar(45) NOT NULL,
  `LAST_MODIFIED_DATE` datetime NOT NULL,
  PRIMARY KEY  (`PRODUCT_ID`)
  ) ENGINE=InnoDB;

     2c：删除表空间

mysql> ALTER TABLE product DISCARD TABLESPACE;

     2d: 拷贝原先的.ibd 文件（需要恢复的表的数据文件）到 test1的数据库目录下

     2e：导入表空间 

mysql> ALTER TABLE product IMPORT TABLESPACE;

        该步骤通常情况下会报错，除非导入的表空间对应的table id 等于新建表的table id

        报错信息；

ERROR 1030 (HY000): Got error -1 from storage engine

     2f：检查error.log ,我们能够找到该.ibd文件的 table id

081010 11:47:40  InnoDB: Error: tablespace id in file
'.test1product.ibd' is 1193, but in the InnoDB
InnoDB: data dictionary it is 1

        我们知道 internal  table id 是1，.ibd file对应的 table id 是1193

     3、清空 test1

     3a、手动移动 .ibd 文件到 到其他位置（一会儿还会需要）

     3b、删除表

mysql> DROP TABLE product;

        这个步骤不会重设 internal table 计数器

      4、创建相应数量的表来使 internal table id 的值增加

         在这个案例中，我们需要在 test1中创建 1191个 innodb table,(table id 1已经被占用，需要比对应.ibd 文件的table id 小1， 所以是 1193-2=1191 )

         执行下面的程序：     

for ($1=1; $i<=1191; $1++) {
  CREATE TABLE t# (id int) ENGINE=InnoDB;
}

       注：我已经用Php 程序搞定

      5、完成以上步骤后，删除所有的db 和 table。

         drop database test1;

      6、重新执行步骤 2a-2e

mysql> CREATE DATABASE test1;
mysql> USE test1;
mysql> CREATE TABLE `product` ( ... ) ENGINE=InnoDB;
mysql> ALTER TABLE product DISCARD TABLESPACE;

        拷贝对应的.ibd file 到对应的 test1数据库目录      

mysql> ALTER TABLE product IMPORT TABLESPACE;

Success!

mysql> show tables;
+-----------------+
| Tables_in_test1 |
+-----------------+
| product         |
+-----------------+
1 row in set (0.00 sec)

      7、用mysqldump来备份该表（这一步必须执行），然后可以在任何一个实例中进行恢复。

      以上的情况常常发生在数据库 crash 或者表空间损坏的情况下。

      如果发生以上 情况，先尝试force innodb recovery 并dump出数据。  从 1开始innodb_force_recovery=1 (and try 2,3,4,5,6) 直到能够dump出数据。

      对于以上的例子，我是设置 innodb_force_recovery=5 来解决问题。

      以下是我的操作记录：     

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump1.txt
mysqldump: Couldn't execute 'show table status like 'product'':
Lost connection to MySQL server during query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump2.txt
mysqldump: Couldn't execute 'show table status like 'product'':
Lost connection to MySQL server during query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump3.txt
mysqldump: Couldn't execute 'show table status like 'product'':
Lost connection to MySQL server during query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump4.txt
mysqldump: Couldn't execute 'SELECT /*!40001 SQL_NO_CACHE */
* FROM `product`': Lost connection to MySQL server during
query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump5.txt

C:Program FilesMySQLmysql-5.0.68bin>
mysqladmin -u root -P 3385 shutdown

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump6.txt

      设置为5的原因是由于以下error中的信息：

InnoDB: Error: trying to access update undo rec field 19
in index PRIMARY of table test1/product
InnoDB: but index has only 12 fields

      以上是 undo  log中的信息，文档中设置为5的解释是：

      "Do not look at undo logs when starting the database: InnoDB treats even incomplete transactions as committed"

      方法二 修改.ibd file

      在此之前先备份数据（ibdata  file,ib_logfile, data）

      按照下面的 1-5步来 操作：

      http://dev.mysql.com/doc/refman/5.0/en/adding-and-removing.html

      Let me post them here for completeness, however:

1. Use mysqldump to dump all your InnoDB tables.
2. Stop the server.
3. Remove all the existing tablespace files, including the ibdata and ib_log files. If you want to keep a backup copy of the information, then copy all the ib* files to another location before the removing the files in your MySQL installation.
4. Remove any .frm files for InnoDB tables.
5. Configure a new tablespace.
6. Restart the server.
7. Import the dump files.

   重复 2a-2f的过程，来获得 该.ibd  file的table id，

   在windows上使用 Freeware Hex Editor XVI32 （http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm）

   以下是修改部分：

   For me, and I assume it should be the same for you, but just look at the values to be sure, I see the tablespace id values listed at position 37 and 41 (positions 25 and 29 in hex). In the actual hex column, if you're previous tablespace id was 2, then in positions 37 and 41, you'd see 02 and 02.

   (Note these positions can change. For instance, I tested on a table with an internal id of 1193. This in hex is 04A9. However, when searching the file, for the first instance of the table id, I found the '04' in position 39 and 'A9' in position 40. Then, for the second instance of the table id, the '04' was at position 43 and the 'A9' was at position 44. So, you'll have to convert the table id to hex, and then search for that value, near the beginning of the file.)

   Note that this value (02) may vary depending on what your actual tablespace id is.

   Then, simply modify both of those fields to 01, and save the file.

   再执行下面的部分：

1. ALTER TABLE tbl_name DISCARD TABLESPACE;
2. Put the newly saved .ibd file back in the proper database directory
3. ALTER TABLE tbl_name IMPORT TABLESPACE;

   以下是涉及到的 php 脚本：

$dbhost = "localhost:3385";
$dbname = "test1";
$dbuser = "root";
$dbpwd  = "";

mysql_connect($dbhost,$dbuser,$dbpwd) or die(mysql_error());

for ($i = 1033; $i <= 1190; $i++) {
   $dbquery = "CREATE TABLE test1.t" . $i . " (id int) ENGINE=InnoDB";

   echo "" . $dbquery . "";

      $result = mysql_db_query($dbname,$dbquery) or die(mysql_error());

      $j = 0;

      while($row = mysql_fetch_array($result)) {
         $j++;
         echo $row[0];
      }
}

mysql_close();

PHP Internal Table ID Finder - Used to determine the internal Table ID from the binary .ibd file:

/*
Tested with tables from 4.1.23, 5.0.68, 5.1.28, and 6.0.7.
*/

// Set the filename
$filename = "C:\Users\Chris\Desktop\mysql\working\ibds\z1.ibd";

// Read 2 bytes in at a time
$offset = 2;

// Echo filename and path
echo "filename = $filename

";

// Open the filename - need 'rb' for binary file on Windows
$handle = fopen($filename, "rb");

// Define redundant, local variables for possible later functionality and/or checks
$ibd_id_bin = 0;
$ibd_id_hex = 0;
$ibd_id_dec = 0;
$ibd_id_bin2 = 0;
$ibd_id_hex2 = 0;
$ibd_id_dec2 = 0;

// Find the filesize (note: below command messes up script)
//$filesize = filesize($filename));

// Only loop through first 21 bytes - as table is is in $array[18] and $array[20]
for ($z = 0; $z <= 20; $z++) {

	// Set variable $contents equal to 2 ($offset) bytes of binary data
	$contents = fread($handle, $offset);

	// Convert $contents from binary data to hex data
	$contents2 = bin2hex($contents);

	// Convert $contents2 from hex data to decimal data
	$contents3 = hexdec($contents2);

	// Debug Output
	//echo "contents[$z] = " . $contents . "";
	//echo "contents2[$z] = " . $contents2 . "

";
	//echo "contents3[$z] = " . $contents3 . "

";

	// If position 19, array position [18], then store the values
	if ($z == 18) {
		$ibd_id_bin = $contents;
		$ibd_id_hex = $contents2;
		$ibd_id_dec = $contents3;
	}

	// If position 21, array position [20], then store the values
	if ($z == 20) {
		$ibd_id_bin2 = $contents;
		$ibd_id_hex2 = $contents2;
		$ibd_id_dec2 = $contents3;
	}
}
fclose($handle);

// More Debug output
//echo "

The table id is $ibd_id_dec

";
//echo "

The table id is $ibd_id_dec2

";

// Check to see if both values are equal.  If so, then it's
// most certain this is the correct value.
// If not, then there's a chance the positions are off for
// this table (due to versions, etc.).
if ($ibd_id_dec == $ibd_id_dec2) {
	echo "

The table id is $ibd_id_dec

";
} else {
	echo "The values from positions [18] and [20] did not match,";
             echo "so please enable debug output, and check for proper positions.";
}