hadoop解决问题集锦

• 
  qiulp@qiuluping:~/hadoop_work/hadoop-1.0.3_qiu$ ./bin/hadoop fs -ls /
  Exception in thread "Main Thread" java.lang.Error: Unresolved compilation problems: 
  The import org.apache.commons cannot be resolved
  The import org.apache.commons cannot be resolved
  The import org.codehaus cannot be resolved
  The import org.codehaus cannot be resolved
  Log cannot be resolved to a type
  LogFactory cannot be resolved
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  Log cannot be resolved to a type
  JsonFactory cannot be resolved to a type
  JsonFactory cannot be resolved to a type
  JsonGenerator cannot be resolved to a type
  
  
  at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:58)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:57)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
  at org.apache.hadoop.fs.FsShell.main(FsShell.java:1895)
  
  修改了主机目录后，出现问题，重新copy就好了。
• hadoop streaming
  日志级别问题

• 误用root启动hadoop时，
• 用root重启hadoop时，显示连接不上 查看日志..
  
• 帮波波装时候，删除localhost
• export HADOOP_HOME_WARN_SUPPRESS=1
  
• 网络访问不到某个节点 防火墙没有关闭 访问量很大时起作用
  网络访问不到 ：端口开了访问不到
         或者端口没开
  体现在MR就是程序会在某个地方卡很久
  reduce > copy (5 of 6 at 0.00 MB/s)
  
• mjiang@syvenus:~/program/eclipse/customer$ hadoop --config ~/hadoop_work/hadoop-1.0.3/conf fs -ls /
  Found 16 items
  -rw-r--r--   3 qiulp  supergroup     446615 2012-08-24 17:18 /CHANGES.txt
  drwxrwxrwx   - hadoop supergroup          0 2012-09-19 17:30 /data1
  drwxrwxrwx   - hadoop supergroup          0 2012-08-30 19:25 /fenxi_system
  drwxrwxrwx   - hadoop supergroup          0 2012-04-09 17:50 /hadoop
  drwxrwxrwx   - hadoop supergroup          0 2012-09-19 13:28 /home
  drwxrwxrwx   - hadoop supergroup          0 2012-07-26 17:45 /liujingyu_t1
  drwxrwxrwx   - hadoop supergroup          0 2012-07-26 17:45 /liujingyu_t2
  drwxrwxrwx   - hadoop supergroup          0 2012-07-26 13:42 /liujingyu_test
  drwxrwxrwx   - mjiang supergroup          0 2012-09-10 18:51 /o3
  drwxrwxrwx   - hadoop supergroup          0 2012-09-11 10:06 /o5
  drwxr-xr-x   - qiulp  supergroup          0 2012-08-28 16:48 /qiulp
  drwxrwxrwx   - hadoop supergroup          0 2012-09-05 13:40 /system
  drwxr-xr-x   - mjiang supergroup          0 2012-08-30 10:44 /t_fx_latitude_daily
  drwxrwxrwx   - hadoop supergroup          0 2012-02-14 10:51 /test
  drwxrwxrwx   - hadoop supergroup          0 2012-09-19 17:30 /tmp
  drwxrwxrwx   - hadoop supergroup          0 2012-09-19 17:28 /user
  
  mjiang@syvenus:~/program/eclipse/customer$ hadoop fs -ls  /
  Found 26 items
  dr-xr-xr-x   - root root          0 2012-09-17 14:41 /proc
  drwxr-xr-x   - root root       4096 2012-09-17 14:13 /home
  drwxr-xr-x   - root root       4096 2012-04-19 17:32 /mnt
  drwxr-xr-x   - root root          0 2012-09-17 14:41 /sys
  drwxr-xr-x   - root root        800 2012-09-18 22:46 /run
  drwxr-xr-x   - root root       4096 2012-09-17 14:25 /usr
  -rw-r--r--   1 root root   14189274 2012-07-24 20:56 /initrd.img
  drwxr-xr-x   - root root       4096 2012-04-26 00:04 /lib64
  drwx------   - root root      16384 2012-07-24 13:41 /lost+found
  drwxr-xr-x   - root root       4096 2012-09-14 19:32 /var
  drwxr-xr-x   - root root       4096 2012-07-24 13:43 /cdrom
  -rw-r--r--   1 root root    4965840 2012-04-26 08:13 /vmlinuz
  drwxr-xr-x   - root root       4096 2012-07-24 20:56 /lib
  drwxr-xr-x   - root root       4096 2012-04-26 00:04 /srv
  drwxr-xr-x   - root root       4096 2012-09-18 18:46 /media
  drwxr-xr-x   - root root       4096 2012-09-13 10:44 /sbin
  drwxrwxrwx   - root root      20480 2012-09-20 09:18 /tmp
  drwxr-xr-x   - root root       4096 2012-07-30 11:19 /opt
  drwxr-xr-x   - root root       4096 2012-07-24 20:56 /bin
  drwx------   - root root       4096 2012-09-17 14:25 /root
  drwxr-xr-x   - root root       4096 2012-09-12 18:15 /123
  drwxr-xr-x   - root root       4280 2012-09-19 14:55 /dev
  drwxr-xr-x   - root root      12288 2012-09-19 20:16 /etc
  drwxr-xr-x   - root root       4096 2012-09-14 18:01 /test1
  drwxr-xr-x   - root root       1024 2012-07-24 20:56 /boot
  drwxr-xr-x   - root root       4096 2012-03-06 01:54 /selinux
  
  mjiang@syvenus:~/hadoop_work/hadoop-1.0.3$ vi bin/hadoop-config.sh 
  mjiang@syvenus:~/hadoop_work/hadoop-1.0.3$ echo $HADOOP_CONF_DIR
  /home/yibo/hadoop/hadoop-1.0.3/conf