GPS-Graph Processing System 修改源码经验总结 （四）
HamaWhite原创，转载请注明出处！欢迎大家加入Giraph 技术交流群： 228591158
本文目的：在修改GPS源码后，详细描述如何编译和分发到各Worker节点上。下面以Graph Coloring 算法为例进行讲解，本文基于GPS的前三篇文章。
1. 首先在Master(test150)上修改源码。Graph Coloring算法源码路径：gps.examples.coloring包，主要修改ColoringVertex.java类。该算法在Selection（MIS_1）阶段是按照顶点的出度大小概率性的选择UNDECIDED状态的顶点，源码如下：
if (ColoringVertexType.NOT_IN_SET == value.type
|| ColoringVertexType.IN_SET == value.type) {
return;
}
double probability = getNeighborsSize() > 0 ? 1.0 /
((double) 2*value.numRemainingNeighbors) : 1;
if (Math.random() <= probability) {
value.type = ColoringVertexType.SELECTED_AS_POSSIBLE_IN_SET;
if (value.numRemainingNeighbors > 0) {
ColoringMessage newSelectedAsPossibleMessage = ColoringMessage
.newNeighborSelectedAsPossibleMessage(getId());
for (int neighborId : getNeighborIds()) {
if (neighborId >= 0) {
sendMessage(neighborId, newSelectedAsPossibleMessage);
}
}
}
}
下面修改为：选择所有的UNDECIDED状态的顶点，而非概率性选择。修改后源码如下：
if (ColoringVertexType.NOT_IN_SET == value.type
|| ColoringVertexType.IN_SET == value.type) {
return;
}
//double probability = getNeighborsSize() > 0 ? 1.0 /
// ((double) 2*value.numRemainingNeighbors) : 1;
//if (Math.random() <= probability) {
// value.type = ColoringVertexType.SELECTED_AS_POSSIBLE_IN_SET;
if (value.numRemainingNeighbors > 0) {
ColoringMessage newSelectedAsPossibleMessage = ColoringMessage
.newNeighborSelectedAsPossibleMessage(getId());
for (int neighborId : getNeighborIds()) {
if (neighborId >= 0) {
sendMessage(neighborId, newSelectedAsPossibleMessage);
}
}
}
//}
2. 因为之前已编译过GPS源码，并且把Jar包等文件已分发到集群中各个Worker节点上，所以此步骤是删除之前的文件。删除Master节点trunk目录下的：gps-0.0.1-slave.tar.gz 、gps_node_runner.jar和classes文件夹，删除Worker节点trunk目录下的 conf 、gps-0.0.1-slave.tar.gz 、gps_node_runner.jar 、libs 、scripts，logs文件夹保留。
3. 参考 GPS-Graph Processing System集群安装笔记（一），重新编译和分发Jar包等文件。
下面附上我的脚本，因中间使用了我自己的脚本，故不可直接使用，但是可以参考。脚本所在目录：/home/gougou/GPS/trunk。
cd /home/gougou/GPS/trunk # delete master files rm -rf gps_node_runner.jar rm -rf classes rm -rf gps-0.0.1-slave.tar.gz # delete worker files. the Shell writed by myself. cd /home/gougou/ShellUtils ./deleteDirectory.sh /home/gougou/GPS/trunk/conf ./deleteDirectory.sh /home/gougou/GPS/trunk/gps-0.0.1-slave.tar.gz ./deleteDirectory.sh /home/gougou/GPS/trunk/gps_node_runner.jar ./deleteDirectory.sh /home/gougou/GPS/trunk/libs ./deleteDirectory.sh /home/gougou/GPS/trunk/scripts/ # compile GPS source code cd /home/gougou/GPS/trunk cd local-master-scripts # generate gps_node_runner.jar and classes under trunk directory ./make_gps_node_runner_jar.sh # generate gps-0.0.1-slave.tar.gz under trunk directory ./make_gps_tar_gz.sh cd ../master-scripts cp slaves temp cp slaves-12 slaves ./copy_and_untar_gps_tar_to_slaves.sh 12 mv temp slaves4. 在trunk/master-scripts目录下，运行Graph Coloring 算法，命令如下：
./start_gps_nodes.sh 2 GC-Test5-1 \ "-ifs /user/gougou/GC-Test5/gc-5.txt \ -hcf /home/gougou/hadoop-1.0.3/conf/core-site.xml \ -jc gps.examples.coloring.JobConfiguration \ -mcfg /machine-configs/test_machine_config_2.cfg \ -log4jconfig /home/gougou/GPS/trunk/conf/log4j.config"
完！