moving InnoDB
Impossible - possible, moving InnoDB tables between servers
http://www.mysqlperformanceblog.com/2009/06/08/impossible-possible-moving-innodb-tables-between-servers/
Impossible - possible, moving InnoDB tables between servers
Posted by Vadim
This is probably the feature I missed most from early days when I started to use InnoDB instead of MyISAM. Since that I figured out how to survive without it, but this is first question I hear from customers who migrated from MyISAM to InnoDB - can I just copy .ibd files from one server to another and answer “use mysqldump” is quite disappointed.
Jokes aside, I see real needs in this:
- when we need to restore only single table from backup (sometimes developers kill only single table, not whole database
)
- to copy single table from production to QA environment. It may sound not so important, but I see needs in this quite often. QA boxes may have their own setup, not so powerful and with not enough space, but QA still needs to have some tables in fresh status.
- resharding databases, moving some shards from one server to another.
So long story short end - we made new mode for XtraBackup, now it can copy and prepare InnoDB/XtraDB tables, which later can be imported into XtraDB.
To be objective the process is still not so easy as with MyISAM tables when you just copy table.frm, table.MYD, table.MYI files, but so nice here - just can copy InnoDB tables in fully non-blocking mode, you do not need to shutdown neither source nor destination server (if destination server runs XtraDB).
In short, process looks like (ah, yes you also need innodb_file_table mode, which ones love, and ones hate)
1) do backup of specific tables on sources server with xtrabackup (server can run InnoDB or XtraDB, it does not matter)
2) do prepare of tables with –export options
3) create table on destination server with exactly the same CREATE TABLE statement as on source. (on destination you should have XtraDB with innodb_expand_import extension and with innodb_expand_import=1 setting)
4) run alter table IMPORTEDTABLE discard tablespace;
5) copy .exp and .ibd files to destination server
6) run alter table IMPORTEDTABLE import tablespace;
7) enjoy progress in error.log like:
InnoDB: import: extended import of test/img_out59
InnoDB: import: 3 indexes are detected.
InnoDB: Progress in %: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 done.
Some more information available on
http://www.percona.com/docs/wiki/percona-xtradb:patch:innodb_expand_import
If you want some internals: I expected the biggest problem with implementation would be merging insert buffer or handling undo records, stored in ibdata file, but it was easy - you just need to shutdown instance with innodb_fast_shutdown=1 (done in xtrabackup –prepare –export call). The more complex problem was that InnoDB stores pointers on root leafs of secondary indexes also in system tablespace. And that’s why we maintain .exp files - to keep information from system tablespace that related to exported table.
Both xtrabackup –export mode and XtraDB innodb_expand_import available only in source code for now, you can get them from Launchpad projects https://launchpad.net/percona-xtradb and https://launchpad.net/percona-xtrabackup. Binary releases will be ready as soon as we have done with testing of this feature. You are welcome to try!
http://www.mysqlperformanceblog.com/2009/06/08/impossible-possible-moving-innodb-tables-between-servers/
Impossible - possible, moving InnoDB tables between servers
Posted by Vadim
This is probably the feature I missed most from early days when I started to use InnoDB instead of MyISAM. Since that I figured out how to survive without it, but this is first question I hear from customers who migrated from MyISAM to InnoDB - can I just copy .ibd files from one server to another and answer “use mysqldump” is quite disappointed.
Jokes aside, I see real needs in this:
- when we need to restore only single table from backup (sometimes developers kill only single table, not whole database
)
- to copy single table from production to QA environment. It may sound not so important, but I see needs in this quite often. QA boxes may have their own setup, not so powerful and with not enough space, but QA still needs to have some tables in fresh status.
- resharding databases, moving some shards from one server to another.
So long story short end - we made new mode for XtraBackup, now it can copy and prepare InnoDB/XtraDB tables, which later can be imported into XtraDB.
To be objective the process is still not so easy as with MyISAM tables when you just copy table.frm, table.MYD, table.MYI files, but so nice here - just can copy InnoDB tables in fully non-blocking mode, you do not need to shutdown neither source nor destination server (if destination server runs XtraDB).
In short, process looks like (ah, yes you also need innodb_file_table mode, which ones love, and ones hate)
1) do backup of specific tables on sources server with xtrabackup (server can run InnoDB or XtraDB, it does not matter)
2) do prepare of tables with –export options
3) create table on destination server with exactly the same CREATE TABLE statement as on source. (on destination you should have XtraDB with innodb_expand_import extension and with innodb_expand_import=1 setting)
4) run alter table IMPORTEDTABLE discard tablespace;
5) copy .exp and .ibd files to destination server
6) run alter table IMPORTEDTABLE import tablespace;
7) enjoy progress in error.log like:
InnoDB: import: extended import of test/img_out59
InnoDB: import: 3 indexes are detected.
InnoDB: Progress in %: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 done.
Some more information available on
http://www.percona.com/docs/wiki/percona-xtradb:patch:innodb_expand_import
If you want some internals: I expected the biggest problem with implementation would be merging insert buffer or handling undo records, stored in ibdata file, but it was easy - you just need to shutdown instance with innodb_fast_shutdown=1 (done in xtrabackup –prepare –export call). The more complex problem was that InnoDB stores pointers on root leafs of secondary indexes also in system tablespace. And that’s why we maintain .exp files - to keep information from system tablespace that related to exported table.
Both xtrabackup –export mode and XtraDB innodb_expand_import available only in source code for now, you can get them from Launchpad projects https://launchpad.net/percona-xtradb and https://launchpad.net/percona-xtrabackup. Binary releases will be ready as soon as we have done with testing of this feature. You are welcome to try!