Development hadoop program on Windows

This post is about the development of hadoop program on windows. The detail configuration describes as follows:

Step 1: copy the hadoop plugin to the eclipse's plugin directory.

Step 2:restart the ecplise IDE.

Step 3:configure the mapreduce plugin in eclipse IDE.

Step 4:mapping the hostname to the ip address in C:\WINDOWS\system32\drivers\etc\hosts.

# e.g.

xxx.xxx.xxx.xxx master

Step 5: you will encounter the errors, when running the programs on the hadoop.

13/05/25 10:40:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/05/25 10:40:28 ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator1090172040\.staging to 0700
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator1090172040\.staging to 0700
	at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
	at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
	at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
	at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
	at WordCount.main(WordCount.java:69)

Reason: The job starts with the local JobRunner and the jobClient isn't initialized with the configure files on eclipse IDE.

Solution: add the below codes after Configuration conf = new Configuration();

conf.set("mapred.job.tracker", "master's ip address:9001");