编程完成输出HDFS中指定文件的文本到终端中

题目:查看Java帮助手册或其它资料,用“java.net.URL”和“org.apache.hadoop.fs.FsURLStreamHandlerFactory”编程完成输出HDFS中指定文件的文本到终端中。

java代码:

1.	import java.io.BufferedReader;
2.	import java.io.InputStreamReader;
3.	import java.net.URL;
4.	import org.apache.hadoop.conf.Configuration;
5.	import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;
6.	
7.	public class ReadHdfsFileFromURL {
8.	
9.	  public static void main(String[] args) throws Exception {
10.	    // Register the Hadoop URL stream handler factory with the JVM
11.	    URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
12.	
13.	    // Set the configuration object for the Hadoop filesystem
14.	    Configuration conf = new Configuration();
15.	    conf.set("fs.defaultFS", "hdfs://localhost:9000");
16.	
17.	    // Construct the URL for the HDFS file
18.	    URL url = new URL("hdfs:///user/hadoop/input/file.txt");
19.	
20.	    // Open an input stream from the URL
21.	    BufferedReader reader = new BufferedReader(
22.	        new InputStreamReader(url.openStream()));
23.	
24.	    // Read the contents of the file and print to the console
25.	    String line;
26.	    while ((line = reader.readLine()) != null) {
27.	      System.out.println(line);
28.	    }
29.	
30.	    // Close the input stream
31.	    reader.close();
32.	  }
33.	}

命令:

1.	start-dfs.sh
2.	vim file.txt
3.	hadoop fs -mkdir /user/hadoop/input
4.	hadoop fs -put /root/file.txt /user/hadoop/input
5.	cd /usr/local/hadoop/share
6.	ls
7.	vim ReadHdfsFileFromURL.java
8.	ls
9.	javac -classpath /usr/local/hadoop/share/hadoop/common/hadoop-common-2.7.4.jar     ReadHdfsFileFromURL.java      
10.	ls
11.	HADOOP_CLASSPATH=. hadoop ReadHdfsFileFromURL 

结果:

编程完成输出HDFS中指定文件的文本到终端中_第1张图片

 

你可能感兴趣的:(hadoop,hdfs,大数据)