Spark生态之Tachyon学习1---单机版搭建和运行（Alluxio）
环境
ubuntu 14.04
Spark-1.5.2
Tachyon-0.7.1
1．由于最近用的Spark-1.5.2系统默认的tachyon为0.7.1，在Spark-core的pom.xml可以查看
另外虽然现在Tachyon改名为Alluxio ，但是不影响这个版本的使用
2.单机版搭建：
下载地址：
https://github.com/Alluxio/alluxio/releases/tag/v0.7.1
下载的是：tachyon-0.7.1-bin.tar.gz
3.安装
修改配置文件：
cp conf/alluxio-env.sh.template conf/alluxio-env.sh然后format和start：
./bin/alluxio format ./bin/alluxio-start.sh local
查看安装情况：
xubo@xubo:~/cloud/tachyon-0.7.1/bin$ ./tachyon format Connecting to localhost as xubo... Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts. Formatting Tachyon Worker @ xubo Connection to localhost closed. Formatting Tachyon Master @ localhost xubo@xubo:~/cloud/tachyon-0.7.1/bin$ jps 12576 SparkSubmit 13760 Jps xubo@xubo:~/cloud/tachyon-0.7.1/bin$ ls tachyon tachyon-mount.sh tachyon-start.sh tachyon-stop.sh tachyon-workers.sh xubo@xubo:~/cloud/tachyon-0.7.1/bin$ ./tachyon-start.sh local Killed 0 processes on xubo Killed 0 processes on xubo Connecting to localhost as xubo... Killed 0 processes on xubo Connection to localhost closed. [sudo] password for xubo: Formatting RamFS: /mnt/ramdisk (1gb) Starting master @ localhost Starting worker @ xubo xubo@xubo:~/cloud/tachyon-0.7.1/bin$ jps 12576 SparkSubmit 14088 Jps 14058 TachyonWorker 14030 TachyonMaster xubo@xubo:~/cloud/tachyon-0.7.1/bin$
可以看出来master和worker已经启动了
另外也可以从浏览器看：
http://localhost:19999/home
localhost为自己的ip
说明安装成功
4. 使用
（1）指令
xubo@xubo:~/cloud/tachyon-0.7.1/bin$ ./tachyon tfs -help
Usage: java TFsShell
[cat <path>]
[count <path>]
[ls <path>]
[lsr <path>]
[mkdir <path>]
[rm <path>]
[rmr <path>]
[tail <path>]
[touch <path>]
[mv <src> <dst>]
[copyFromLocal <src> <remoteDst>]
[copyToLocal <src> <localDst>]
[fileinfo <path>]
[location <path>]
[report <path>]
[request <tachyonaddress> <dependencyId>]
[pin <path>]
[unpin <path>]
[free <file path|folder path>]
[getUsedBytes]
[getCapacityBytes]
[du <path>]
（2）测试用例
xubo@xubo:~/cloud/tachyon-0.7.1$ ./bin/tachyon runTest Basic CACHE_THROUGH /default_tests_files/BasicFile_CACHE_THROUGH has been removed 2016-05-04 22:19:45,075 INFO (MasterClient.java:connect) - Tachyon client (version 0.7.1) is trying to connect with master @ localhost/127.0.0.1:19998 2016-05-04 22:19:45,104 INFO (MasterClient.java:connect) - User registered with the master @ localhost/127.0.0.1:19998; got UserId 4 2016-05-04 22:19:45,130 INFO (CommonUtils.java:printTimeTakenMs) - createFile with fileId 3 took 59 ms. 2016-05-04 22:19:45,153 INFO (WorkerClient.java:connect) - Trying to get local worker host : xubo 2016-05-04 22:19:45,166 INFO (WorkerClient.java:connect) - Connecting local worker @ xubo/219.219.220.222:29998 2016-05-04 22:19:45,230 INFO (BlockOutStream.java:get) - Writing with local stream. tachyonFile: /default_tests_files/BasicFile_CACHE_THROUGH, blockIndex: 0, opType: CACHE_THROUGH 2016-05-04 22:19:45,289 INFO (CommonUtils.java:createBlockPath) - Folder /mnt/ramdisk/tachyonworker/4 was created! 2016-05-04 22:19:45,294 INFO (LocalBlockOutStream.java:<init>) - /mnt/ramdisk/tachyonworker/4/3221225472 was created! tachyonFile: /default_tests_files/BasicFile_CACHE_THROUGH, blockIndex: 0, blockId: 3221225472, blockCapacityByte: 536870912 2016-05-04 22:19:45,370 INFO (CommonUtils.java:printTimeTakenMs) - writeFile to file /default_tests_files/BasicFile_CACHE_THROUGH took 239 ms. 2016-05-04 22:19:45,420 INFO (CommonUtils.java:printTimeTakenMs) - readFile file /default_tests_files/BasicFile_CACHE_THROUGH took 50 ms. Passed the test!
（3）从本地上传文件到remote：
xubo@xubo:~/cloud/tachyon-0.7.1$ ./bin/tachyon tfs copyFromLocal pom.xml / Copied pom.xml to /可以在浏览器中查看
（4）显示文件内容：
xubo@xubo:~/cloud/test/tachyon$ ../../tachyon-0.7.1/bin/tachyon tfs copyFromLocal 1.txt / Copied 1.txt to / xubo@xubo:~/cloud/test/tachyon$ ../../tachyon-0.7.1/bin/tachyon tfs cat /1.txt hello tachyon 1 2 3
（5）Spark调用tachyon文件：待完成，Spark出问题了，请见下一篇博文
（6）关闭tachyon
hadoop@Master:~/cloud/testByXubo/spark/tachyon/tachyon-0.7.1$ ./bin/tachyon-stop.sh Killed 1 processes on Master Killed 1 processes on Master Connecting to localhost as hadoop... jjKilled 0 processes on Master Connection to localhost closed.
附：将tachyon脚本cp到usr的bin下：（需要本地编译，直接start的不行）
xubo@xubo:~/cloud/tachyon-0.7.1/bin$ sudo cp tachyon /usr/ bin/ games/ include/ lib/ lib64/ local/ sbin/ share/ src/ xubo@xubo:~/cloud/tachyon-0.7.1/bin$ sudo cp tachyon /usr/local/bin/ xubo@xubo:~/cloud/tachyon-0.7.1/bin$ ta tabs tac tachyon tail tailf tali tangle tap2deb tap2rpm tapconvert tar tarcat targen taskset方便使用
参考
【1】http://alluxio.org/documentation/v0.7.1/
【2】https://github.com/xubo245/SparkLearning
【3】http://alluxio.org/documentation/v0.7.1/Running-Tachyon-Locally.html