linux file system
Linux各种文件系统(ext3,ReiserFS,jfs,xfs)的性能
2006-07-28 08:55
以下文章是我自己翻译的，后面有英文的原文。不当之处，敬请指教.
应该不是太新的文章，但是我我是2006-07-12的上午才看到的。哎........
2006-07-12 15:00 翻译完成
--------------------------------------------------------------------------------------------------------------
肠子都悔青了，昨天刚刚新加的硬盘上面的文件系统还是被我搞成了ext3。如果我能造一天注意到这篇文章的话........
----------------------------------------------------------------------------------------------------------------
Debian Administration
System Administration Tips and Resources
现在还可以得到的许多Linux filesystems 比较，但是他们中大多数是古老的，基于为人任务的或者在更老的情况下完成。 这篇基准测试文基于与老一代的适合一台文件服务器的11项硬件(奔腾II/III，EIDE硬盘)。
从最初编制到出版，文章已经产生许多变化，意见和建议改进。我目前正努力进行一些新的试验。(回答在原文范围的问题)。
结果将在大约在(2006年5月8日)可提供
汉斯
为什么要做基准测试？
我发现quantitative and reproductible benchmark基准使用2.6.x kernel。
Benoit在2003年在有512 MB RAM 的PIII 500服务器上使用大文件(1 + GB)实现12次试验。 这次试验信息十分丰富，但是结果对2.6.x kernel开始，主要适用于底座， 专门操作大的锉(例如，多媒体，科学，数据库)。
Piszcz 在2006年实现21项任务(有768 MB RAM 和一个400GBEIDE-133硬盘在PIII-500 模拟多种文件操作)。到目前为止，这测试看起来是在2.6.x kernel上的最全面的工作。 但是， 很多任务是人造的(例如， 复制和删除10，000个空目录，新建10，000个文件，递归分割文件)，把这些结论应用到现实世界可能是无意义的。
因此， 这里测试的基准的目标是验证一些Piszcz(2006)的结论， 通过专门应用于现实世界在小型企业文件服务器(看见任务描述)里找到。
测试基础
* Hardware Processor : Intel Celeron 533
* RAM : 512MB RAM PC100
* Motherboard : ASUS P2B
* Hard drive : WD Caviar SE 160GB (EIDE 100, 7200 RPM, 8MB Cache)
* Controller : ATA/133 PCI (Silicon Image)
* OS Debian Etch (kernel 2.6.15), distribution upgraded on April 18, 2006
* All optional daemons killed (cron,ssh,saMBa,etc.)
* Filesystems Ext3 (e2fsprogs 1.38)
* ReiserFS (reiserfsprogs 1.3.6.19)
* JFS (jfsutils 1.1.8)
* XFS (xfsprogs 2.7.14)
选择的测试任务描述
*在一个大文件(ISO 镜像文件，700 MB)的从第2个磁盘复制到这个试验磁盘
*再从在另一个位置再复制这个 ISO 一次
*删除这个ISO 的两个副本
*操作一文件树(有7500 文件，900 目录，1.9GB)，从第2 磁盘复制到这个试验磁盘
*再从在另一个位置再复制这个文件树 一次
*删除这个文件树的两个副本
*用递归的方法遍历文件树目录和文件树的全部内容，复制到这个试验磁盘
*匹配通配符，在文件树查找具体的文件
*用(mkfs) 建立filesystem(全部FS都使用默认值)
*mount filesystem
*Umount filesystem
上述11项任务(从建立filesystem到umounting filesystem)的顺序，编写为Bash script运行完成3 次(报告平均成绩)。 每个顺序花费大约7分种，完成任务的时间用秒计算， GNU time utility (version 1.7) 记录任务时的CPU 的利用百分比。
结果
分区能力
(在filesystem 创造之后)初始化分区并重新划分block的过程里，Ext3有最差的初始利用率(92.77%)， 其它的filesystem 几乎可是使用全部的容量(ReiserFS = 99.83%，JFS = 99.82%，XFS = 99.95%)。
结论： 为了使用你的分区的的最大容量，选择ReiserFS，JFS或者XFS。
建立文件系统，mount和unmounting
在20GB的分区创造filesystem测试，划分为Ext3带14.7秒， 与为相比其他filesystem多2秒或更少。(ReiserFS = 2.2，JFS = 1.3，XFS = 0.7)。
不过，挂载ReiserFS 要比其他的FS多花费5-15倍时间(2.3秒)(Ext3 = 0.2, JFS = 0.2, XFS = 0.5)，umount以及要比其他的FS多花费2 倍时间(0.4秒)。
所有的FS都花费差不多CPU占用来创建FS(59%（ReiserFS） -74%（JFS）)，挂载FS(在6和9%之间)。 不过，Ext3 和XFS多用2倍的CPU占用 给umount(37% 和45%)， ReiserFS和JFS(14% 和27%)。
结论： 对创建FS性能和mounting/unmounting来说，选择JFS或者XFS。
大文件操作性能(ISO image，700 MB)
把大文件复制到Ext3(38.2秒)和ReiserFS(41.8)， 要比JFS和XFS(35.1和34.8)需要更多时间。使用XFS有助于提高在相同的磁盘上复制相同的文件(XFS=33.1,Ext3 = 37.3，JFS = 39.4，ReiserFS = 43.9)相比。 在JFS和XFS上删除那些ISO 大约要快100 倍(0.02秒)，（ReiserFS1.5秒，Ext32.5秒）！所有FS复制时的CPU利用(在46和51%之间)，再复制时(在38%到50%之间)。当其他FS使用大约10%时，ReiserFS使用49%的CPU。 比他FS大约少5到10%)，JFS使用较少的CPU。
结论： 对大文件操作性能来说，最好选择JFS或者XFS。 如果你需要使CPU利用减到最小，更推荐JFS。
目录树(7500个文件，900份目录，1.9GB)的操作
最初复制目录树时，Ext3(158.3秒)和XFS(166.1秒)更迅速， ReiserFS和JFS(172.1和180.1)。在第二次复制的时候有相似的结果。（Ext3=120秒，XFS = 135.2，ReiserFS = 136.9 和JFS = 151)。但是， 移动目录树时Ext3(22秒)相比ReiserFS(8.2秒， XFS(10.5秒)和JFS(12.5秒))大约多2倍时间！所有FS在复制和再复制目录树时都使用较多的CPU (复制在27和36%之间)，(再复制在29%-JFS和45%-ReiserFS之间)。
令人吃惊，ReiserFS 和XFS使用更多的CPU 删除目录树(86% 和65%)， 而其他FS只使用大约15%的(Ext3和JFS)。再次，与任何其他FS相比较，JFS的明显使用较少CPU。 当有较多的数量较小页面时适合ReiserFS。这个差别在目录树的再复制和移动里的ReiserFS将有更高的速度。
结论：对在大容量的目录树操作来说，选择Ext3或者XFS。 来自其他作者的基准测试已经证明如果使用ReiserFS，对大量的小文件更为合适。但是，目前包括各种各样尺寸(10KB在5 MB)数千文件的目录树操作上，建议使用Ext3或者XFS可能获得更好的性能。如果JFS的CPU占用减到最小，这种FS带着相当高的性能。
目录列表和文件查找
递归显示目录的目录列表，ReiserFS(1.4秒)和XFS更迅速的1.8)， Ext3和JFS(2.5和3.1)。文件查找有着相同的结果。在文件查找项目， ReiserFS(0.8秒)相比XFS(2.8)和Ext3(4.6秒)和JFS(5秒)更迅速。 Ext3和JFS有更好CPU占用：目录列表(35%)，文件查找(6%)。 XFS目录列表(70%)使用更多的CPU，文件查找(10%)。 ReiserFS 看起来是占用CPU最多的FS，目录列表71%，文件查找36% 。
结论： 结果表明那， 对于这些CPU占用任务来说，(ext3和JFS)filesystems 能更少的使用CPU的。 XFS作为好备选，带有相对适中的性能和CPU的占用。
总的结论
这些结果从Piszcz(2006)关于分解的Ext3，ReiserFS的磁盘能力报告一样。 这两篇文章两篇已经显示JFS是最低的CPU利用的FS。 最后，这份报告看起来没有显示出ReiserFS的high page faults activity
由于一个分区只能有一个filesystem，认识每种filesystem的优缺点很重要。如果，以这篇文章的全部测试为基准， XFS看起来是家庭或者小型企业最适合的应用于文件服务器的filesystem：
*它使用你的服务器硬盘(s)的拥有最大的容量
*创建FS，mount和unmount很迅速
*操作大文件最迅速的FS(>500 MB)
*这FS得第二名地方给经营关于许多在适度尺寸文件和目录小
*在CPU占用和目录列表或者文件搜寻性能之间比较平衡，
*没有最小CPU要求，老一代硬件也可十分接受
Piszcz(2006)当时没有明确推荐XFS，他只是说："就个人来说，我仍然愿意选择同时具备性能和可伸缩性的XFS"。 现在我只能支持这个结论。
参考
贝努瓦，M.(2003)。 Linux 文件系统基准。
Piszcz，J.(2006)。 基准问题测试Filesystems第二部分。 Linux Gazette, 122 (January 2006)。
以下是原文
-----------------------------------------------------------------------
Debian Administration
System Administration Tips and Resources
[ About | Archive | FAQ | Hall of Fame | Search | Tagged Articles |
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch
There are a lot of Linux filesystems comparisons available but most of them are anecdotal, based on artificial tasks or completed under older kernels. This benchmark essay is based on 11 real-world tasks appropriate for a file server with older generation hardware (Pentium II/III, EIDE hard-drive).
Since its initial publication, this article has generated
a lot of questions, comments and suggestions to improve it.
Consequently, I'm currently working hard on a new batch of tests
to answer as many questions as possible (within the original scope
of the article).
Results will be available in about two weeks (May 8, 2006)
Many thanks for your interest and keep in touch with
Debian-Administration.org!
Hans
Why another benchmark test?
I found two quantitative and reproductible benchmark testing studies using the 2.6.x kernel (see References). Benoit (2003) implemented 12 tests using large files (1+ GB) on a Pentium II 500 server with 512MB RAM. This test was quite informative but results are beginning to aged (kernel 2.6.0) and mostly applied to settings which manipulate exclusively large files (e.g., multimedia, scientific, databases).
Piszcz (2006) implemented 21 tasks simulating a variety of file operations on a PIII-500 with 768MB RAM and a 400GB EIDE-133 hard disk. To date, this testing appears to be the most comprehensive work on the 2.6 kernel. However, since many tasks were "artificial" (e.g., copying and removing 10 000 empty directories, touching 10 000 files, splitting files recursively), it may be difficult to transfer some conclusions to real-world settings.
Thus, the objective of the present benchmark testing is to complete some Piszcz (2006) conclusions, by focusing exclusively on real-world operations found in small-business file servers (see Tasks description).
Test settings
* Hardware Processor : Intel Celeron 533
* RAM : 512MB RAM PC100
* Motherboard : ASUS P2B
* Hard drive : WD Caviar SE 160GB (EIDE 100, 7200 RPM, 8MB Cache)
* Controller : ATA/133 PCI (Silicon Image)
* OS Debian Etch (kernel 2.6.15), distribution upgraded on April 18, 2006
* All optional daemons killed (cron,ssh,saMBa,etc.)
* Filesystems Ext3 (e2fsprogs 1.38)
* ReiserFS (reiserfsprogs 1.3.6.19)
* JFS (jfsutils 1.1.8)
* XFS (xfsprogs 2.7.14)
Description of selected tasks
* Operations on a large file (ISO image, 700MB) Copy ISO from a second disk to the test disk
* Recopy ISO in another location on the test disk
* Remove both copies of ISO
* Operations on a file tree (7500 files, 900 directories, 1.9GB) Copy file tree from a second disk to the test disk
* Recopy file tree in another location on the test disk
* Remove both copies of file tree
* Operations into the file tree List recursively all contents of the file tree and save it on the test disk
* Find files matching a specific wildcard into the file tree
* Operations on the file system Creation of the filesystem (mkfs) (all FS were created with default values)
* Mount filesystem
* Umount filesystem
The sequence of 11 tasks (from creation of FS to umounting FS) was run as a Bash script which was completed three times (the average is reported). Each sequence takes about 7 min. Time to complete task (in secs), percentage of CPU dedicated to task and number of major/minor page faults during task were computed by the GNU time utility (version 1.7).
RESULTS
Partition capacity
Initial (after filesystem creation) and residual (after removal of all files) partition capacity was computed as the ratio of number of available blocks by number of blocks on the partition. Ext3 has the worst inital capacity (92.77%), while others FS preserve almost full partition capacity (ReiserFS = 99.83%, JFS = 99.82%, XFS = 99.95%). Interestingly, the residual capacity of Ext3 and ReiserFS was identical to the initial, while JFS and XFS lost about 0.02% of their partition capacity, suggesting that these FS can dynamically grow but do not completely return to their inital state (and size) after file removal.
Conclusion : To use the maximum of your partition capacity, choose ReiserFS, JFS or XFS.
File system creation, mounting and unmounting
The creation of FS on the 20GB test partition took 14.7 secs for Ext3, compared to 2 secs or less for other FS (ReiserFS = 2.2, JFS = 1.3, XFS = 0.7). However, the ReiserFS took 5 to 15 times longer to mount the FS (2.3 secs) when compared to other FS (Ext3 = 0.2, JFS = 0.2, XFS = 0.5), and also 2 times longer to umount the FS (0.4 sec). All FS took comparable amounts of CPU to create FS (between 59% - ReiserFS and 74% - JFS) and to mount FS (between 6 and 9%). However, Ex3 and XFS took about 2 times more CPU to umount (37% and 45%), compared to ReiserFS and JFS (14% and 27%).
Conclusion : For quick FS creation and mounting/unmounting, choose JFS or XFS.
Operations on a large file (ISO image, 700MB)
The initial copy of the large file took longer on Ext3 (38.2 secs) and ReiserFS (41.8) when compared to JFS and XFS (35.1 and 34.8). The recopy on the same disk advantaged the XFS (33.1 secs), when compared to other FS (Ext3 = 37.3, JFS = 39.4, ReiserFS = 43.9). The ISO removal was about 100 times faster on JFS and XFS (0.02 sec for both), compared to 1.5 sec for ReiserFS and 2.5 sec for Ext3! All FS took comparable amounts of CPU to copy (between 46 and 51%) and to recopy ISO (between 38% to 50%). The ReiserFS used 49% of CPU to remove ISO, when other FS used about 10%. There was a clear trend of JFS to use less CPU than any other FS (about 5 to 10% less). The number of minor page faults was quite similar between FS (ranging from 600 - XFS to 661 - ReiserFS).
Conclusion : For quick operations on large files, choose JFS or XFS. If you need to minimize CPU usage, prefer JFS.
Operations on a file tree (7500 files, 900 directories, 1.9GB)
The initial copy of the tree was quicker for Ext3 (158.3 secs) and XFS (166.1) when compared to ReiserFS and JFS (172.1 and 180.1). Similar results were observed during the recopy on the same disk, which advantaged the Ext3 (120 secs) compared to other FS (XFS = 135.2, ReiserFS = 136.9 and JFS = 151). However, the tree removal was about 2 times longer for Ext3 (22 secs) when compared to ReiserFS (8.2 secs), XFS (10.5 secs) and JFS (12.5 secs)! All FS took comparable amounts of CPU to copy (between 27 and 36%) and to recopy the file tree (between 29% - JFS and 45% - ReiserFS). Surprisingly, the ReiserFS and the XFS used significantly more CPU to remove file tree (86% and 65%) when other FS used about 15% (Ext3 and JFS). Again, there was a clear trend of JFS to use less CPU than any other FS. The number of minor page faults was significantly higher for ReiserFS (total = 5843) when compared to other FS (1400 to 1490). This difference appears to come from a higher rate (5 to 20 times) of page faults for ReiserFS in recopy and removal of file tree.
Conclusion : For quick operations on large file tree, choose Ext3 or XFS. Benchmarks from other authors have supported the use of ReiserFS for operations on large number of small files. However, the present results on a tree comprising thousands of files of various size (10KB to 5MB) suggest than Ext3 or XFS may be more appropriate for real-world file server operations. Even if JFS minimize CPU usage, it should be noted that this FS comes with significantly higher latency for large file tree operations.
Directory listing and file search into the previous file tree
The complete (recursive) directory listing of the tree was quicker for ReiserFS (1.4 secs) and XFS (1.8) when compared to Ext3 and JFS (2.5 and 3.1). Similar results were observed during the file search, where ReiserFS (0.8 sec) and XFS (2.8) yielded quicker results compared to Ext3 (4.6 secs) and JFS (5 secs). Ext3 and JFS took comparable amounts of CPU for directory listing (35%) and file search (6%). XFS took more CPU for directory listing (70%) but comparable amount for file search (10%). ReiserFS appears to be the most CPU-intensive FS, with 71% for directory listing and 36% for file search. Again, the number of minor page faults was 3 times higher for ReiserFS (total = 1991) when compared to other FS (704 to 712).
Conclusion : Results suggest that, for these tasks, filesystems can be regrouped as (a) quick and more CPU-intensive (ReiserFS and XFS) or (b) slower but less CPU-intensive (ext3 and JFS). XFS appears as a good compromise, with relatively quick results, moderate usage of CPU and acceptable rate of page faults.
OVERALL CONCLUSION
These results replicate previous observations from Piszcz (2006) about reduced disk capacity of Ext3, longer mount time of ReiserFS and longer FS creation of Ext3. Moreover, like this report, both reviews have observed that JFS is the lowest CPU-usage FS. Finally, this report appeared to be the first to show the high page faults activity of ReiserFS on most usual file operations.
While recognizing the relative merits of each filesystem, only one filesystem can be install for each partition/disk. Based on all testing done for this benchmark essay, XFS appears to be the most appropriate filesystem to install on a file server for home or small-business needs :
* It uses the maximum capacity of your server hard disk(s)
* It is the quickest FS to create, mount and unmount
* It is the quickest FS for operations on large files (>500MB)
* This FS gets a good second place for operations on a large number of small to moderate-size files and directories
* It constitutes a good CPU vs time compromise for large directory listing or file search
* It is not the least CPU demanding FS but its use of system ressources is quite acceptable for older generation hardware
While Piszcz (2006) did not explicitly recommand XFS, he concludes that "Personally, I still choose XFS for filesystem performance and scalability". I can only support this conclusion.
References
Benoit, M. (2003). Linux File System Benchmarks.
Piszcz, J. (2006). Benchmarking Filesystems Part II. Linux Gazette, 122 (January 2006).