Hadoop集成BTrace

1.BTrace

1.1BTrace部署

1.下载BTrace工具包,官网地址:https://kenai.com/projects/btrace/downloads/directory/releases

2.解压btrace-bin.tar.gz,命令如下:

$ tar -zxvf btrace-bin.tar.gz

3.配置环境变量

BTRACE_HOME=/usr/btrace-bin
PATH=$PATH:$BTRACE_HOME/bin

4.执行btrace(注意:$BTRACE_HOME/binbtracebtracecbtracer文件权限)命令如下,表示部署成功:


1.2BTrace脚本

BTrace测试脚本代码:

importcom.sun.btrace.annotations.*; 
 
import static com.sun.btrace.BTraceUtils.*;
 
@BTrace
public class TraceMR
{
       @TLS privatestaticlongmapperSetupStartTime = 0l;
 
       @OnMethod(clazz="org.apache.hadoop.mapreduce.Mapper",
                       method="run",
                       location=@Location(where=Where.BEFORE, value=Kind.CALL,clazz="/.*/", method="setup"))
       public static void onMapper_run_Before_Call_setup() {
              mapperSetupStartTime =timeNanos();
              println(strcat("MAP\tSTARTUP_MEM\t",str(used(heapUsage()))));
       }
 
       @OnMethod(clazz="org.apache.hadoop.mapreduce.Mapper",
                       method="run",
                       location=@Location(where=Where.AFTER, value=Kind.CALL,clazz="/.*/", method="setup"))
       public static void onMapper_run_After_Call_setup() {
              println(strcat("MAP\tSETUP\t",str(timeNanos() -mapperSetupStartTime)));
              println(strcat("MAP\tSETUP_MEM\t",str(used(heapUsage()))));
       }
}


1.3编译BTrace脚本

 编译命令:

$ btracec TraceMR.java

2.Hadoop配置

2.1分发文件

将BTrace文件:$BTRACE_HOME/build目录下的btrace-agent.jar和btrace-boot.jar,以及编译Btrace脚本后生成的class文件分发到各个子节点上。建议使用脚本分发,代码如下:

#!/usr/bin/envbash
#Usage:
#  ./install_btrace.sh <slaves_file>
# 
# where:
#    slaves_file = File containing a list of slavemachines
#
#Example:
#  ./install_btrace.sh /root/SLAVE_NAMES.txt
#Make sure we have all the arguments
if [$# -ne 1 ]; then
   echo "Usage: $0<slaves_file>"
   echo " slaves_file = File containing a list of slave machines"
   echo ""
   exit -1
fi
 
#Get the slaves file
declareSLAVES_FILE=$1;
iftest ! -e $SLAVES_FILE; then
   echo "ERROR: The file '$SLAVES_FILE'does not exist. Exiting"
   exit -1
fi
#Get the slaves file
declareSLAVES_FILE=$1;
iftest ! -e $SLAVES_FILE; then
   echo "ERROR: The file '$SLAVES_FILE'does not exist. Exiting"
   exit -1
fi
 
MASTER_BTRACE_DIR=/home/wangpeng/zhiming/usr/btrace/build
SLAVES_BTRACE_DIR=/home/wangpeng/zhiming/usr
 
forslave in `cat "$SLAVES_FILE"`; do
{
   printf "Installing on host:$slave\n"
   ssh $slave "mkdir -p$SLAVES_BTRACE_DIR"
   scp ${MASTER_BTRACE_DIR}/btrace-agent.jar$slave:$SLAVES_BTRACE_DIR/.
   scp ${MASTER_BTRACE_DIR}/btrace-boot.jar$slave:$SLAVES_BTRACE_DIR/.
   scp ${MASTER_BTRACE_DIR}/TraceMR.class$slave:$SLAVES_BTRACE_DIR/.
}
done


2.2配置Hadoop文件

在所有子节点的$HADOOP_HOME安装目录下找到mapred-site.xml文件,进行如下设置:

代码:

<property>
         <name>mapred.map.child.java.opts</name>
         <value>
     -Xmx1024m -javaagent:/home/wangpeng/zhiming/usr/btrace/build/btrace-agent.jar=script=/home/wangpeng/zhiming/usr//TraceMR.class,stdout=true,noServer=true
         </value>
</property>


2.3运行MR程序

  1.  准备一个写好的MR程序,如WordCount.java程序;
  2. .执行hadoop jar WordCount.jar WordCount /user/wzm/input /user/wzm/output。

2.4查看输出文件

在所有子节点的$HADOOP_HOME安装目录下找到logs目录中的stdout文件,执行命令:cat stdout,且有如下输出表示配置成功:


你可能感兴趣的:(hadoop,BTrace)