hadoop api java 访问conf.set("fs.default",hdfs)

转载:http://www.codeweblog.com/hdfs-api%E6%96%87%E4%BB%B6%E4%B8%8A%E4%BC%A0-fs-default-name%E9%85%8D%E7%BD%AE%E7%9A%84%E5%85%B3%E9%94%AE%E6%80%A7/

HDFS API文件上传

fs.default.name配置的关键性


package step1;

import java.io.IOException;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

public class MyMR1 {
    private static String inPatha = "/matrix/step1_input/matrix2.txt";
    private static String inPath = "hdfs://bigdata.fun.com:9000/matrix/step1_input/matrix2.txt";
    private static String hdfs = "hdfs://bigdata.fun.com:9000";
    public static void main(String[] args) {
        Configuration conf = new Configuration();
        Path inputPath = new Path(inPatha);
        conf.set("fs.default.name", hdfs);
        System.out.println(conf.toString());
        FileSystem fsFileSystem;
        try {
                fsFileSystem = FileSystem.get(conf);
                System.out.println(fsFileSystem.exists(inputPath));
                if (fsFileSystem.exists(inputPath)) {
                    System.out.println("success");
                } else {
                    System.out.println(inputPath);
                    System.out.println("input not exists!!!!");
            }
        } catch (IOException e1) {
            // TODO Auto-generated catch block
            e1.printStackTrace();
            System.out.println("2  ");
        }
    }
}

注意:conf.set(“fs.default.name”, hdfs);

这里的key经测试,hadoop 1.2.1 用“fs.default.name”,没测试过其他版本,据说2.x版本可以试试“fs.defaultFS”或“dfs.defaultFS”,这需要根据各自的版本的hadoop api设置。
api:http://hadoop.apache.org/docs/r1.2.1/api/overview-summary.html
ps:百度了一晚上,根本没想起来api这回事儿,不写代码java真是太久了……

你可能感兴趣的:(hadoop,经验教训)