No FileSystem for scheme: hdfs,No FileSystem for scheme: file

原文：http://blog.newitfarmer.com/tags/hadoop

hadoop  打成jar 打包插件assembly的问题，目前hadoop 2.4.0的应用有两个jar。

这两个jar分别 hadoop-common和hadoop-hdfs这两个包，assembly 打包的时候对应每个jar里面的META-INFO/services/org.apache.hadoop.fs.FileSystem

在新包里面不是追加而是覆盖导致的。

  解决办法：

 

hadoopConfig.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName() );  

hadoopConfig.set("fs.file.impl", org.apache.hadoop.fs.LocalFileSystem.class.getName() );

When we use maven-assembly, it merges all our JARs into one, and all META-INFO/services/org.apache.hadoop.fs.FileSystem overwrite each-other. Only one of these files remains (the last one that was added). In this case, the Filesystem list from hadoop-commons overwrites the list from hadoop-hdfs, so DistributedFileSystem was no longer declared.