1.下载:
git clone https://github.com/databricks/spark-avro.git
./build/sbt package
3,记录:
hadoop@Master:~/cloud/testByXubo/soft/spark-avro$ ./build/sbt package Using /usr/lib/jvm/jdk1.7.0 as default JAVA_HOME. Note, this will be overridden by -java-home if it is set. Attempting to fetch sbt Launching sbt from build/sbt-launch-0.13.9.jar [info] Loading project definition from /home/hadoop/cloud/testByXubo/soft/spark-avro/project [warn] Multiple resolvers having different access mechanism configured with same name 'sonatype-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`). [info] Updating {file:/home/hadoop/cloud/testByXubo/soft/spark-avro/project/}spark-avro-build... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] downloading https://oss.sonatype.org/content/repositories/releases/org/scala-lang/scala-actors/2.10.2/scala-actors-2.10.2.jar ... [info] [SUCCESSFUL ] org.scala-lang#scala-actors;2.10.2!scala-actors.jar (8654ms) [info] Done updating. Missing bintray credentials /home/hadoop/.bintray/.credentials. Some bintray features depend on this. [info] Set current project to spark-avro (in build file:/home/hadoop/cloud/testByXubo/soft/spark-avro/) [warn] Credentials file /home/hadoop/.bintray/.credentials does not exist [info] Updating {file:/home/hadoop/cloud/testByXubo/soft/spark-avro/}spark-avro... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] downloading http://maven.oschina.net/content/groups/public/org/apache/avro/avro/1.7.6/avro-1.7.6.jar ... [info] [SUCCESSFUL ] org.apache.avro#avro;1.7.6!avro.jar(bundle) (365ms) [info] downloading http://maven.oschina.net/content/groups/public/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar ... [info] [SUCCESSFUL ] com.thoughtworks.paranamer#paranamer;2.3!paranamer.jar (184ms) [info] downloading http://maven.oschina.net/content/groups/public/org/xerial/snappy/snappy-java/1.0.5/snappy-java-1.0.5.jar ... [info] [SUCCESSFUL ] org.xerial.snappy#snappy-java;1.0.5!snappy-java.jar(bundle) (402ms) [info] downloading http://maven.oschina.net/content/groups/public/org/slf4j/slf4j-api/1.6.4/slf4j-api-1.6.4.jar ... [info] [SUCCESSFUL ] org.slf4j#slf4j-api;1.6.4!slf4j-api.jar (182ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/spark/spark-sql_2.10/1.4.1/spark-sql_2.10-1.4.1.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-sql_2.10;1.4.1!spark-sql_2.10.jar (619ms) [info] downloading http://maven.oschina.net/content/groups/public/org/scalatest/scalatest_2.10/2.2.1/scalatest_2.10-2.2.1.jar ... [info] [SUCCESSFUL ] org.scalatest#scalatest_2.10;2.2.1!scalatest_2.10.jar(bundle) (1064ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/spark/spark-core_2.10/1.4.1/spark-core_2.10-1.4.1.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-core_2.10;1.4.1!spark-core_2.10.jar (1335ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/spark/spark-catalyst_2.10/1.4.1/spark-catalyst_2.10-1.4.1.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-catalyst_2.10;1.4.1!spark-catalyst_2.10.jar (706ms) [info] downloading http://maven.oschina.net/content/groups/public/com/twitter/parquet-column/1.6.0rc3/parquet-column-1.6.0rc3.jar ... [info] [SUCCESSFUL ] com.twitter#parquet-column;1.6.0rc3!parquet-column.jar (344ms) [info] downloading http://maven.oschina.net/content/groups/public/com/twitter/parquet-hadoop/1.6.0rc3/parquet-hadoop-1.6.0rc3.jar ... [info] [SUCCESSFUL ] com.twitter#parquet-hadoop;1.6.0rc3!parquet-hadoop.jar (284ms) [info] downloading http://maven.oschina.net/content/groups/public/org/jodd/jodd-core/3.6.3/jodd-core-3.6.3.jar ... [info] [SUCCESSFUL ] org.jodd#jodd-core;3.6.3!jodd-core.jar (217ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/spark/spark-launcher_2.10/1.4.1/spark-launcher_2.10-1.4.1.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-launcher_2.10;1.4.1!spark-launcher_2.10.jar (175ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/spark/spark-network-common_2.10/1.4.1/spark-network-common_2.10-1.4.1.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-network-common_2.10;1.4.1!spark-network-common_2.10.jar (448ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/spark/spark-network-shuffle_2.10/1.4.1/spark-network-shuffle_2.10-1.4.1.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-network-shuffle_2.10;1.4.1!spark-network-shuffle_2.10.jar (150ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/spark/spark-unsafe_2.10/1.4.1/spark-unsafe_2.10-1.4.1.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-unsafe_2.10;1.4.1!spark-unsafe_2.10.jar (211ms) [info] downloading http://maven.oschina.net/content/groups/public/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar ... [info] [SUCCESSFUL ] net.jpountz.lz4#lz4;1.2.0!lz4.jar (226ms) [info] downloading http://maven.oschina.net/content/groups/public/org/spark-project/akka/akka-remote_2.10/2.3.4-spark/akka-remote_2.10-2.3.4-spark.jar ... [info] [SUCCESSFUL ] org.spark-project.akka#akka-remote_2.10;2.3.4-spark!akka-remote_2.10.jar (470ms) [info] downloading http://maven.oschina.net/content/groups/public/org/spark-project/akka/akka-slf4j_2.10/2.3.4-spark/akka-slf4j_2.10-2.3.4-spark.jar ... [info] [SUCCESSFUL ] org.spark-project.akka#akka-slf4j_2.10;2.3.4-spark!akka-slf4j_2.10.jar (147ms) [info] downloading http://maven.oschina.net/content/groups/public/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar ... [info] [SUCCESSFUL ] org.scala-lang#scala-library;2.10.4!scala-library.jar (1155ms) [info] downloading http://maven.oschina.net/content/groups/public/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar ... [info] [SUCCESSFUL ] io.netty#netty-all;4.0.23.Final!netty-all.jar (373ms) [info] downloading http://maven.oschina.net/content/groups/public/io/dropwizard/metrics/metrics-core/3.1.0/metrics-core-3.1.0.jar ... [info] [SUCCESSFUL ] io.dropwizard.metrics#metrics-core;3.1.0!metrics-core.jar(bundle) (155ms) [info] downloading http://maven.oschina.net/content/groups/public/io/dropwizard/metrics/metrics-jvm/3.1.0/metrics-jvm-3.1.0.jar ... [info] [SUCCESSFUL ] io.dropwizard.metrics#metrics-jvm;3.1.0!metrics-jvm.jar(bundle) (216ms) [info] downloading http://maven.oschina.net/content/groups/public/io/dropwizard/metrics/metrics-json/3.1.0/metrics-json-3.1.0.jar ... [info] [SUCCESSFUL ] io.dropwizard.metrics#metrics-json;3.1.0!metrics-json.jar(bundle) (149ms) [info] downloading http://maven.oschina.net/content/groups/public/io/dropwizard/metrics/metrics-graphite/3.1.0/metrics-graphite-3.1.0.jar ... [info] [SUCCESSFUL ] io.dropwizard.metrics#metrics-graphite;3.1.0!metrics-graphite.jar(bundle) (148ms) [info] downloading http://maven.oschina.net/content/groups/public/org/tachyonproject/tachyon-client/0.6.4/tachyon-client-0.6.4.jar ... [info] [SUCCESSFUL ] org.tachyonproject#tachyon-client;0.6.4!tachyon-client.jar (152ms) [info] downloading http://maven.oschina.net/content/groups/public/org/spark-project/akka/akka-actor_2.10/2.3.4-spark/akka-actor_2.10-2.3.4-spark.jar ... [info] [SUCCESSFUL ] org.spark-project.akka#akka-actor_2.10;2.3.4-spark!akka-actor_2.10.jar (689ms) [info] downloading http://maven.oschina.net/content/groups/public/org/spark-project/protobuf/protobuf-java/2.5.0-spark/protobuf-java-2.5.0-spark.jar ... [info] [SUCCESSFUL ] org.spark-project.protobuf#protobuf-java;2.5.0-spark!protobuf-java.jar(bundle) (354ms) [info] downloading http://maven.oschina.net/content/groups/public/org/tachyonproject/tachyon/0.6.4/tachyon-0.6.4.jar ... [info] [SUCCESSFUL ] org.tachyonproject#tachyon;0.6.4!tachyon.jar (585ms) [info] downloading http://maven.oschina.net/content/groups/public/commons-codec/commons-codec/1.5/commons-codec-1.5.jar ... [info] [SUCCESSFUL ] commons-codec#commons-codec;1.5!commons-codec.jar (151ms) [info] downloading http://maven.oschina.net/content/groups/public/org/scala-lang/scala-compiler/2.10.4/scala-compiler-2.10.4.jar ... [info] [SUCCESSFUL ] org.scala-lang#scala-compiler;2.10.4!scala-compiler.jar (2113ms) [info] downloading http://maven.oschina.net/content/groups/public/com/twitter/parquet-common/1.6.0rc3/parquet-common-1.6.0rc3.jar ... [info] [SUCCESSFUL ] com.twitter#parquet-common;1.6.0rc3!parquet-common.jar (169ms) [info] downloading http://maven.oschina.net/content/groups/public/com/twitter/parquet-encoding/1.6.0rc3/parquet-encoding-1.6.0rc3.jar ... [info] [SUCCESSFUL ] com.twitter#parquet-encoding;1.6.0rc3!parquet-encoding.jar (408ms) [info] downloading http://maven.oschina.net/content/groups/public/com/twitter/parquet-generator/1.6.0rc3/parquet-generator-1.6.0rc3.jar ... [info] [SUCCESSFUL ] com.twitter#parquet-generator;1.6.0rc3!parquet-generator.jar (192ms) [info] downloading http://maven.oschina.net/content/groups/public/com/twitter/parquet-format/2.2.0-rc1/parquet-format-2.2.0-rc1.jar ... [info] [SUCCESSFUL ] com.twitter#parquet-format;2.2.0-rc1!parquet-format.jar (255ms) [info] downloading http://maven.oschina.net/content/groups/public/com/twitter/parquet-jackson/1.6.0rc3/parquet-jackson-1.6.0rc3.jar ... [info] [SUCCESSFUL ] com.twitter#parquet-jackson;1.6.0rc3!parquet-jackson.jar (356ms) [info] downloading http://maven.oschina.net/content/groups/public/commons-net/commons-net/3.1/commons-net-3.1.jar ... [info] [SUCCESSFUL ] commons-net#commons-net;3.1!commons-net.jar (219ms) [info] downloading https://oss.sonatype.org/content/repositories/releases/org/scoverage/scalac-scoverage-runtime_2.10/1.1.0/scalac-scoverage-runtime_2.10-1.1.0.jar ... [info] [SUCCESSFUL ] org.scoverage#scalac-scoverage-runtime_2.10;1.1.0!scalac-scoverage-runtime_2.10.jar (4206ms) [info] downloading http://maven.oschina.net/content/groups/public/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar ... [info] [SUCCESSFUL ] org.mortbay.jetty#jetty;6.1.26!jetty.jar (380ms) [info] downloading http://maven.oschina.net/content/groups/public/org/apache/velocity/velocity/1.7/velocity-1.7.jar ... [info] [SUCCESSFUL ] org.apache.velocity#velocity;1.7!velocity.jar (275ms) [info] Done updating. [info] Compiling 6 Scala sources to /home/hadoop/cloud/testByXubo/soft/spark-avro/target/scala-2.10/classes... [info] Packaging /home/hadoop/cloud/testByXubo/soft/spark-avro/target/scala-2.10/spark-avro_2.10-2.0.2-SNAPSHOT.jar ... [info] Done packaging. [success] Total time: 183 s, completed May 2, 2016 11:19:48 AM