关于4K Block Size的Device和 Aligned IO
背景：最近采购了一批新的服务器，底层的存储设备的默认physical sector size从原有的 512B 改为了 4K。
装完系统以后，在做数据库物理备份恢复时xtrabackup报了这么一个错。但是同样的备份在512B sector size的老系统上却可以恢复。
报错如下：
InnoDB: Error: tried to read 2048 bytes at offset 0 0.
InnoDB: Was only able to read 0.
140304 18:48:12 InnoDB: Operating system error number 22 in a file operation.
InnoDB: Error number 22 means 'Invalid argument'.
InnoDB: Some operating system error numbers are described at
InnoDB:http://dev.mysql.com/doc/refman/5.1/en/operating-system-error-codes.html
InnoDB: File operation call: 'read'.
InnoDB: Cannot continue operation.
innobackupex-1.5.1: Error:
innobackupex-1.5.1: ibbackup failed at /usr/bin/innobackupex-1.5.1 line 386.我们先不讨论原因，先看一下解决方案：http://bazaar.launchpad.net/~akopytov/percona-xtrabackup/bug1190779-2.0/revision/561#src/xtrabackup.cc
升级到2.0.7以上的xtrabackup即可。
1. 什么是Block（Sector）
为什么同样的程序在512B的block size 和在4K的block size上的行为结果不一样呢？
我们先来看一下什么是device block (sector) size：block(又叫sector) 是一个块设备的最小读写单位。也就是说对于一个512B block size的设备。即使上层调用只需要读10个Byte的数据，它也会从设备读取512B的数据，然后再把多余的剔除，返回给上层调用者。
在device block size的上层是filesystem block size：对于filesystem来说一个block也是最小的读写单位。也即只有一个字节的文件，在底层device上也会占一个block的大小。
更多对于block size的解释，见链接
2. 什么是Aligned IO
有了block size以后，自然就出现了对齐(align)的概念。所谓对齐就是IO请求的边界和底层block的边界重合。也就是说上层IO请求的起始点和偏移量是下层设备block size的整数倍。同样读取512B的数据，对齐后的请求只需要下层设备的一次IO，而非对齐的请求就需要下层设备的两次IO再加上前后数据截断。也因为如此，aligned IO的性能要比unaligned IO的性能好很多
下面就是从上自下（从DB到Disk）严格对齐的一张事例图
然而，Linux操作系统和MySQL并不严格要求IO对齐。unaligned IO只会造成IO请求性能略低，但并不应该出现访问报错。
那是什么样的原因导致xtrabackup在4K sector size的设备上报错了呢？
3. O_DIRECT 和 unaligned IO
查阅Linux文档以后我们发现，文件系统在O_DIRECT模式下打开的文件有IO对齐的限制。而xtrabackup在使用了O_DIRECT方式open file的情况，发起了unaligned IO。这种情况下，文件系统会拒绝IO请求。
具体文档摘抄如下：
Users must always take care to use properly aligned and sized IO. This
is especially important for Direct I/O access. Direct I/O should be
aligned on a 'logical_block_size' boundary and in multiples of the
'logical_block_size'. With native 4K devices (logical_block_size is 4K)
it is now critical that applications perform Direct I/O that is a
multiple of the device's 'logical_block_size'. This means that applications that do not perform 4K aligned I/O, but 512-byte aligned I/O, will break with native 4K devices. Applications may consult a
device's "I/O Limits" to ensure they are using properly aligned and
sized I/O. The "I/O Limits" are exposed through both sysfs and block
device ioctl interfaces (also see: libblkid).
而查看xtrabackup 2.0.7 对于这个bug的描述，我们也可以发现这个bug的修复实际上就是简单的把 O_DIRECT的文件打开属性去除。具体change log摘抄如下：
The problem was in an length-unaligned I/O request issued while
manipulating xtrabackup_logfile with O_DIRECT enabled.We don't actually need O_DIRECT in those cases, so the fix was to
disable O_DIRECT.. The patch also removes userspace buffer alignment
code and implements other minor cleanups.
4. 相关文档
http://www.orczhou.com/index.php/2009/08/innodb_flush_method-file-io/
https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1055547
https://bugs.launchpad.net/percona-xtrabackup/+bug/902567
https://bugs.launchpad.net/percona-server/+bug/1033051
http://www.linuxintro.org/wiki/Blocks,_block_devices_and_block_sizes
http://www.mysqlperformanceblog.com/2011/06/09/aligning-io-on-a-hard-disk-raid-the-theory/
http://people.redhat.com/msnitzer/docs/io-limits.txt